Skip to content

Conversation

samansmink
Copy link
Contributor

perhaps a little (too?) hacky, but this allows injecting state into the MultiFileReader through the table function

This is useful when you want to implement ATTACH on a delta tables:

The GetScanFunction for the tableentry can then store additional information (for example a DeltaSnapshot object) in the shared_ptr<TableFunctionInfo> TableFunction::function_info; which the delta extension can then store in the multifilereader. This allows pinning a delta snapshot during a transaction.

@Mytherin
Copy link
Collaborator

Mytherin commented Sep 25, 2024

Thanks for the PR!

The GetScanFunction for the tableentry can then store additional information (for example a DeltaSnapshot object) in the shared_ptr TableFunction::function_info; which the delta extension can then store in the multifilereader. This allows pinning a delta snapshot during a transaction.

Am I reading this right that this passes in a const TableFunction, but we then modify one of the members of the TableFunction?

How would that work if we have multiple connections all starting transactions at the same time over the same delta table?

@samansmink
Copy link
Contributor Author

@Mytherin Mytherin changed the base branch from main to feature September 25, 2024 13:55
@Mytherin
Copy link
Collaborator

Thanks, makes sense after discussion - good to go from my side after CI passes

@duckdb-draftbot duckdb-draftbot marked this pull request as draft September 26, 2024 09:25
@samansmink samansmink marked this pull request as ready for review September 26, 2024 09:25
@duckdb-draftbot duckdb-draftbot marked this pull request as draft October 1, 2024 18:55
@samansmink samansmink force-pushed the more-multifilereader-flexibility branch 2 times, most recently from 2987301 to 85786e4 Compare October 1, 2024 19:29
@samansmink samansmink force-pushed the more-multifilereader-flexibility branch from 85786e4 to 9ebe3c6 Compare October 1, 2024 19:41
@samansmink samansmink marked this pull request as ready for review October 2, 2024 08:25
@Mytherin Mytherin merged commit 2e0490b into duckdb:feature Oct 3, 2024
42 checks passed
@Mytherin
Copy link
Collaborator

Mytherin commented Oct 3, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants