-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Handle exceptions gracefully while loading plugins. #12460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
tests/plugins/test_manager.py
Outdated
caplog.set_level(logging.INFO, logger="conda.plugins.manager") | ||
plugin_manager.load_setuptools_entrypoints("conda") | ||
assert plugin_manager.get_plugins() == set() | ||
assert "load_setuptools_entrypoints ImportError" in caplog.text |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I bet you could add an ImportError
in a different way. See https://github.com/conda/conda/blob/main/tests/gateways/test_repodata_gateway.py#L113 or
conda/tests/test_deprecations.py
Line 170 in 196148c
mocker.patch.dict("sys.modules", values=modules, clear=True) |
The other thing we might want to do is audit our plugins for too-eager imports. Ideally scanning the entry point would avoid importing most of the plugin implementation. |
I'm not following, scanning the entry point? |
When a plugin is imported, just to be added to the list of A plugin author might want to arrange a plugin with separate folders myplugin/cli and myplugin/implementation so that "import myplugin.cli" doesn't import whatever dependencies "myplugin.implementation" uses to do its thing. Inside "myplugin.cli:do_the_plugin()", import "myplugin.implementation". Then unrelated commands remain fast. Firefox used to say "these extensions are slowing down your browser" and tell you how long each one took to load. Then you could decide whether to continue to use those extensions. It also has a safe mode to avoid loading all extensions. |
I see, lazy loading plugins, but I don't see how this works with pluggy as it uses importlib as the central system to load entrypoints based plugins which loads plugins with the import system. I'm not sure how to achieve this given these constraints. I'll take a look at it tomorrow. |
It depends on the type of the plugin. For conda_subcommands it is simple. Image some subcommand uses sqlalchemy. A. someplugin.py import sqlalchemy
def custom_command():
sqlalchemy.do_a_thing()
@plugins.hookimpl
def conda_subcommands(self):
yield CondaSubcommand(
name="custom",
summary="test custom command",
action=self.custom_command,
) B. anotherplugin.py def custom_command():
import sqlalchemy
sqlalchemy.do_a_thing()
@plugins.hookimpl
def conda_subcommands(self):
yield CondaSubcommand(
name="custom",
summary="test custom command",
action=self.custom_command,
) The second plugin is faster. Authors would be more likely to do this accidentally, by having an |
Yeah, I get that, import-time side effects have a downside. For subcommands we could allow to optionally pass a dotted import path as the action parameter and lazy-import it when we need it, to move the moment of loading to later in the stack. |
Thanks @jezdez, this is a much better experience than the older behavior. |
f"Error while loading conda plugins from entrypoints: {err}" | ||
) | ||
count = 0 | ||
for dist in list(distributions()): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the retconned "same thing setuptools did" equivalent function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, importlib.metadata is the pkg_resources equivalent
continue | ||
self.register(plugin, name=entry_point.name) | ||
count += 1 | ||
return count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do we get this line to run
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not following, this just returns the count, there is no return statement in the previous code. This contains code from https://github.com/pytest-dev/pluggy/blob/c18d5da87f33bc5359c6eeccf4e85a194842d907/src/pluggy/_manager.py#L352-L377 BTW
### Enhancements | ||
|
||
* Handle Python import errors gracefully when loading | ||
conda plugins from entrypoints. (#12460) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
solid description
Did we fake a working plugin too as long as we're in here? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should help.
Nah, but it's a good idea to do that in a separate PR. |
Description
Resolves #12455.
Checklist - did you ...
news
directory (using the template) for the next release's release notes?