[Python] Fix crash related to file-like objects and fsspec
#7012
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When reading from file-like objects with the python client, we rely on
fsspec
to create an in-memory filesystem to store them.The import cache is utilized for this, but a recent change that allows importing optional attributes/modules is causing problems here. Because the import silently fails, the object.ptr() is left as
nullptr
.When the object's constructor is called, this causes a segfault.
I've fixed this here by checking if the object could not be imported before calling the constructor, but there are likely other places where this could become a problem.
In #7005 I've made changes to
py::isinstance
to support nullptr objects, maybe a better change would be to modifypy::isinstance
to take aPythonImportCacheItem
object instead, and throw from theoperator ()
of the item if it's called on an item that wasn't imported.Also, I can't add a test for this, I've tried monkeypatching, but in the
pyduckdb/filesystem.py
we import fsspec, overriding the monkeypatch.