Skip to content

LIMIT queries on pyarrow tables no longer work #17638

@cpcloud

Description

@cpcloud

What happens?

I cannot add LIMIT n to a query on a PyArrow table when the query also contains an ORDER BY.

To Reproduce

import duckdb
import pyarrow as pa

con = duckdb.connect()

t = pa.Table.from_pydict({"a": [3]})

con.register("t", t)

print(con.sql("SELECT a FROM t ORDER BY a LIMIT 11"))  # noqa: T201

The error I get is:

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    print(con.sql("SELECT a FROM t ORDER BY a LIMIT 11"))  # noqa: T201
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
duckdb.duckdb.NotImplementedException: Not implemented Error: Pushdown Filter Type DYNAMIC_FILTER is not currently supported in PyArrow Scans

OS:

NixOS Linux x86_64

DuckDB Version:

1.3.0

DuckDB Client:

Python

Hardware:

No response

Full Name:

Phillip Cloud

Affiliation:

Voltron Data

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

  • Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

  • Yes, I have

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions