Skip to content

Conversation

soapy1
Copy link
Contributor

@soapy1 soapy1 commented Jun 5, 2025

Description

This PR introduces a way for environment spec plugins to declare that they should not be automatically detected. So, these plugins would only be accessible to users if they explicitly request the plugin by cli arg or configuration.

For example, consider the environment.yml spec plugin. One could implement a new plugin based on it, with autodetection for their new plugin disabled:

class YamlNoAutoDetect(YamlFileSpec):
   detection_supported = False

@hookimpl
def conda_environment_specifiers():
    yield CondaEnvironmentSpecifier(
        name="secret-environment.yml",
        environment_spec=YamlNoAutoDetect,
    )

This plugin will never be selected if not explicitly requested. In order to use the YamlNoAutoDetect plugin, users must run a command like:

$ conda env create --file special.yml --env-spec secret-environment.yml

When no plugin is detected, conda will produce an error calling out plugins with autodetection disabled:

$ conda env create --file tests/env/support/simple.yml             

EnvironmentSpecPluginNotDetected: Environment at tests/env/support/simple.yml is not readable by any installed environment specifier plugins.

Available plugins: 
    - binstar
    - requirements.txt

Found compatible plugins but they must be explicitly selected.
Request conda to use these plugins by providing
the cli argument `--environment-spec PLUGIN_NAME`:

    - secret-environment.yml

depends on the changes to the plugin manager introduced in #14877

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch from e72a0b7 to 3ec8b82 Compare June 5, 2025 20:32
@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Jun 5, 2025
@github-project-automation github-project-automation bot moved this to 🆕 New in 🔎 Review Jun 5, 2025
Copy link

codspeed-hq bot commented Jun 5, 2025

CodSpeed Performance Report

Merging #14914 will not alter performance

Comparing soapy1:enable-autodetection-env-spec (18a4bff) with main (ea5728d)

Summary

✅ 21 untouched benchmarks

@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch 3 times, most recently from 1eddfda to c1166b1 Compare June 5, 2025 22:25
@jezdez
Copy link
Member

jezdez commented Jun 10, 2025

pre-commit.ci autofix

Comment on lines 568 to 582
if hook.enable_autodetection:
log.debug("EnvironmentSpec hook: checking %s", hook.name)
if hook.environment_spec(source).can_handle():
log.debug(
"EnvironmentSpec hook: %s can be %s",
source,
hook.name,
)
found.append(hook)
else:
log.debug(
"EnvironmentSpec hook: %s can NOT be handled by %s",
source,
hook.name,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaimergp @soapy1 I wonder if this enable_autodetection option is yet another predicate about whether a plugin "can handle", since the same information could simply be implemented on the env spec level instead of the pluggy interface level. I'd prefer the env spec level to all the env spec business logic covered there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that autodetection and the ability for a plugin to parse an environment spec are seperate ideas. Both of these ideas need to come together eventually for conda to decide if an environment spec plugin is going to be used. However, can_handle is about deteriming if a plugin is able to read/parse a given environment spec. And enable_autodetection is about a plugin author being able to opt out of the plugin autodetection process.

imo the whole auto detection process is the responsibility of the plugin manager. So, I think the most appropriate place for a plugin author to be able to set this information is in the bit where the plugin get's registered.

I'm definitely open to suggestions here. But, I would strongly recommend that the mechanism for opting in-to or out-of autodetection is very explicit. Like, it would be confusing if opting out of autodetection was done by having can_handle return a None value opposed to a True/False value.

Copy link
Member

@jezdez jezdez Jun 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree with your hypothesis that plugin authors should be allowed to create plugins that have two states, auto-detectable or not, I'd say those are fundamentally different behaviors, and it's visible by the error message below, which doesn't give users any recourse to resolve the problem.

In other words, I don't agree that this boolean should be stored in the plugin type, since it is business logic if a plugin should be used to automatically detect the environment spec from a provided file. The deciding code would use the same functionality as we'd implement in the EnvironmentSpecBase, rather than what is in the plugin manager, which doesn't care about the implementation details of environment specs at all.

I'd suggest we either move this into the main EnvironmentSpecBase and do something like a detection_supported(), an instance method that can reuse information from the instance:

Suggested change
if hook.enable_autodetection:
log.debug("EnvironmentSpec hook: checking %s", hook.name)
if hook.environment_spec(source).can_handle():
log.debug(
"EnvironmentSpec hook: %s can be %s",
source,
hook.name,
)
found.append(hook)
else:
log.debug(
"EnvironmentSpec hook: %s can NOT be handled by %s",
source,
hook.name,
)
env_spec = hook.environment_spec(source)
if env_spec.detection_supported():
log.debug("EnvironmentSpec hook: checking %s", hook.name)
if env_spec.can_handle():
log.debug(
"EnvironmentSpec hook: %s can be %s",
source,
hook.name,
)
found.append(hook)
else:
log.debug(
"EnvironmentSpec hook: %s can NOT be handled by %s",
source,
hook.name,
)

or we add another plugin type like class OptionalEnvironmentSpec(EnvironmentSpecBase): ..., just for the purpose of providing an optional environment spec, plugin authors would be able to able to switch the behavior by using this different type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets go for adding a detection_supported function opposed to a new type. Pushed up the change.

@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch from 4740867 to 22b4bd5 Compare June 11, 2025 17:18
@soapy1 soapy1 marked this pull request as ready for review June 11, 2025 17:22
@soapy1 soapy1 requested a review from a team as a code owner June 11, 2025 17:22
@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch from 7f8ea6a to e8e7b45 Compare June 12, 2025 16:59
soapy1 added 3 commits June 12, 2025 10:36
Annotating the whole class makes passing refrences to the
class awkward, since it causes 'BinstarSpec' to be a function
opposed to a class. This causes things like CondaEnvironmentSpecifier
to have the wrong type for it's environment_spec field.
These test cases are already covered in the test_env_specs module.
@soapy1
Copy link
Contributor Author

soapy1 commented Jun 12, 2025

pre-commit.ci autofix

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>
@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch from f50b2d6 to 41ae0c1 Compare June 13, 2025 16:22
@soapy1
Copy link
Contributor Author

soapy1 commented Jun 13, 2025

pre-commit.ci autofix

pre-commit-ci bot and others added 2 commits June 13, 2025 16:23
This is just a setting that a plugin author can set as they see
fit. It should not have any extra logic associated with it.
@soapy1 soapy1 requested review from jaimergp, kenodegard and jezdez June 16, 2025 16:16
Copy link
Contributor

@jaimergp jaimergp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, but I think we are missing --environment-spec somewhere? Or is this supposed to come in a follow up PR?

@@ -36,6 +35,7 @@ class BinstarSpec(EnvironmentSpecBase):

msg = None

@deprecated("24.7", "25.9")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to call this one out! I left a note in the commit message:

Annotating the whole class makes passing refrences to the
class awkward, since it causes 'BinstarSpec' to be a function
opposed to a class. This causes things like CondaEnvironmentSpecifier
to have the wrong type for it's environment_spec field.

So, before this change something like:

for hook in hooks:
        if not hook.environment_spec.detection_supported:
            . . .

would error on the binstar plugin because the environment_spec type is a function instead of a class because it has been decorated.

Moving the deprecation decorator to the __init__ method still produces the result of logging the deprecation whenever the binstar class is instantiated, but avoids the above mentioned awkwardness.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'm not sure that's the intended design. cc @kenodegard. We might need an issue to track this if it's not there yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's an odd one, I wouldn't block on it TBH, given that the deprecation is ending soon

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kenodegard Do you have a suggestion how to handle this?

@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch 2 times, most recently from 70e895c to d4e2ef1 Compare June 18, 2025 15:46
Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>
@soapy1 soapy1 force-pushed the enable-autodetection-env-spec branch from d4e2ef1 to 18a4bff Compare June 18, 2025 15:48
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Approved in 🔎 Review Jun 19, 2025
@jaimergp
Copy link
Contributor

@jezdez, given your previous comments in this PR, can you take another look?

@jaimergp jaimergp merged commit b07e66f into conda:main Jun 23, 2025
75 checks passed
@github-project-automation github-project-automation bot moved this from ✅ Approved to 🏁 Done in 🔎 Review Jun 23, 2025
@soapy1 soapy1 deleted the enable-autodetection-env-spec branch June 23, 2025 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed [bot] added once the contributor has signed the CLA
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants