What happens?

When converting Postgres tuples to DuckDB chunk data, we reuse some PG functions (e.g., JsonbToCString and deconstruct_array). These functions internally allocate memory via palloc that isn't explicitly freed. This memory leaks until the end of query (when it is released with the ExecutorState).

Possible fixes:

1. Implement custom version of JsonbToCString and deconstruct_array in pgduckdb. However, this approach might be boverse and introduce compatibility issue.
2. Switch to a dedicated MemoryContext before calling these functions, then periodically reset memory context to reclaim memory (e.g., based on the MemoryContext's allocated memory)

To Reproduce

set duckdb.force_execution=on;

create table j1(c jsonb);
insert into j1 select '{"aasdkjhzxcuyaisdyasuidqwe": 1, "b12314jhkjahsdjkahsdjk": 2}'::jsonb from generate_series(1, 10000000);
-- memory keeps growing
select * from j1 order by 1 limit 1; 

create table a1(c text[]);
insert into a1 select array['aasdkjhzxcuyaisdyasuidqwe', 'b12314jhkjahsdjkahsdjk'] from generate_series(1, 10000000);
-- memory keeps growing
select * from a1 order by 1 limit 1; 

OS:

centos

pg_duckdb Version (if built from source use commit hash):

main branch

Postgres Version (if built from source use commit hash):

17

Hardware:

No response

Full Name:

Yuwei XIAO

Affiliation:

zbyte-inc

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a source build

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

• Yes, I have