Skip to content

Conversation

destrex271
Copy link
Contributor

@destrex271 destrex271 commented Mar 16, 2025

This introduces a duckdb.struct type to postgres which can be used to read the STRUCT type in DuckDB.

Fixes #599

@destrex271 destrex271 changed the title identified oid; branch still in draft state Support for STRUCT type Mar 16, 2025
@@ -250,6 +250,12 @@ ConvertTimeTzDatum(const duckdb::Value &value) {
return TimeTzADTPGetDatum(result);
}

inline Datum
ConvertDuckRowDatum(const duckdb::Value &value) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really have a separate duckdb.struct type and not reuse the duckdb.row type (but you can copy and slightly modify the implementation of duckdb.row to create this new duckdb.struct type). We need this for one important reason (and possibly a few other less important ones): When duckdb.row is part of a select list, we replace it with a * before sending it to duckdb. We don't want that for this struct type. Apart from that they can basically behave the same, but we need two separate types to make the distinction between a type that is really a multiple columns under the hood (and which thus needs * expansion) versus a type that actually is a single column with a composite type in it.

Copy link
Contributor Author

@destrex271 destrex271 Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JelteF do we also need to give the ability to create this type from postgres? like -

CREATE TABLE tester(a duckdb.struct);
INSERT INTO TABLE tester(a) VALUES('{a:12, b:12, c:Akshat}');

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also correct me if I am wrong but we are handling the star exapnsion in pgduckdb_ddl::DuckdbHandleDDL, right?
As far as I understood, pgduckdb_target_list_contains_unresolved_type_or_row function helps us in this decision, right?

@destrex271
Copy link
Contributor Author

destrex271 commented Mar 26, 2025

So I have tried out the following for this:

  1. Created a table on motherduck with a column of struct type and populated as follows:
    image
    image

  2. From postgres I am able to perform the following operations:

-- Select entire content of table
postgres=# SELECT * FROM t1;
         s         
-------------------
 {'v': a, 'i': 42}
(1 row)

-- Use subscripts to access individual elements
postgres=# SELECT s['i'] FROM t1;
 s  
----
 42
(1 row)

-- Perform operations on selected values
postgres=# SELECT (s['i'] + 1) as sum FROM t1;
 sum 
-----
  43
(1 row)

As of now we do not have:

  • Create new tables with struct type from postgres in duckdb
  • Create duckdb.struct type in general

PS: For some reason the json_functions test are failing, will look into it as well
Fixed. Needed to update the tests to work with STRUCT type

@destrex271 destrex271 requested a review from JelteF March 26, 2025 08:45
@destrex271 destrex271 marked this pull request as ready for review March 27, 2025 08:49
@@ -101,6 +106,14 @@ pgduckdb_var_is_duckdb_row(Var *var) {
return pgduckdb_is_duckdb_row(var->vartype);
}

bool
pgduckdb_var_is_duckdb_struct(Var *var) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Where is this utility be used?

Copy link
Contributor Author

@destrex271 destrex271 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for pointing that out! we won't need that for duckdb.struct. I'll get rid of pgduckdb_var_is_duckdb_struct and pgduckdb_is_duckdb_struct functions

@JelteF JelteF enabled auto-merge (squash) April 15, 2025 09:52
@JelteF
Copy link
Collaborator

JelteF commented Apr 15, 2025

Thanks for working on this. I made some relatively small improvements. The major one is to return the duckdb.struct type from the JSON functions that return a STRUCT. That way we actually have some tests for the subscript operator.

@JelteF JelteF requested a review from Y-- April 15, 2025 11:00
Copy link
Collaborator

@JelteF JelteF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me now

@destrex271
Copy link
Contributor Author

This looks good to me now

Thanks!

@JelteF JelteF merged commit 4411d3b into duckdb:main Apr 15, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for converting DuckDB STRUCT type to Postgres composite ROW type
4 participants