Skip to content

malloc heap corruption when reading and writing to parquet #13914

@ncclementi

Description

@ncclementi

What happens?

I'm reading some data from overturemaps filtering and writing to parquet, and I get the following error

python3.11(19247,0x170583000) malloc: Heap corruption detected, free list is damaged at 0x600001d2df40
*** Incorrect guard value: 4736766904
python3.11(19247,0x170583000) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      ipython

To Reproduce

In [1]: import duckdb

In [2]: duckdb.__version__
Out[2]: '1.1.0'

In [3]: 

In [3]: duckdb.sql("install spatial;")

In [4]: duckdb.sql("load spatial;")

In [5]: sql = """COPY (SELECT *
   ...:     FROM read_parquet('s3://overturemaps-us-west-2/release/2024-07-22.0/theme=base/type=infrastructure/*')
   ...:     WHERE bbox.xmin > -125.0
   ...:     AND bbox.xmax < 24.8
   ...:     AND bbox.ymin > -65.8
   ...:     AND bbox.ymax < 49.2
   ...:     AND subtype = 'power') TO test_download_usa_infra.parquet;
   ...: """

In [6]: duckdb.sql(sql)

OS:

MacOS - Sonoma 14.6.1

DuckDB Version:

1.1.0

DuckDB Client:

Python

Hardware:

No response

Full Name:

Naty Clementi

Affiliation:

Voltron Data

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

  • Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

  • Yes, I have

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions