Skip to content

Conversation

pdet
Copy link
Contributor

@pdet pdet commented Feb 17, 2023

No description provided.

@@ -180,4 +177,10 @@ struct DuckDBPyConnection : public std::enable_shared_from_this<DuckDBPyConnecti
static void DetectEnvironment();
};

template <class T>
static bool ModuleIsLoaded() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this still work properly now that we're lazily loading the Python modules?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a specific test in mind?

Copy link
Collaborator

@Mytherin Mytherin Feb 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is that if the pandas module is not already loaded, an entry in the replacement scan list cannot possibly be a pandas dataframe, hence we don't need to check if it is one. Otherwise every replacement scan would involve loading all modules for which we support replacement scans (which is expanding now - pandas, polars, pyarrow, pytorch, tensorflow, ...).

The alternative would be to do a string comparison on the type instead, which we can do without loading the module.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, understood. There might be a couple of rare edge cases but yeah this should work

@Mytherin Mytherin merged commit c4fe4db into duckdb:master Feb 17, 2023
@pdet pdet deleted the polars_ci branch June 27, 2024 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants