Skip to content

Conversation

isuruf
Copy link
Contributor

@isuruf isuruf commented May 10, 2020

And fix virtual packages when using CONDA_SUBDIR commands.

See https://github.com/archspec/archspec for the naming scheme.

@isuruf isuruf requested a review from a team as a code owner May 10, 2020 18:26
@cla-bot cla-bot bot added the cla-signed [bot] added once the contributor has signed the CLA label May 10, 2020
@h-vetinari
Copy link

Awesome! Will be useful for enabling things like numpy/numpy#13516, I'm guessing. Even though it'd be another dimension on the build-matrix, there's probably 2-3 variants that'd be worth building for...?

@isuruf
Copy link
Contributor Author

isuruf commented May 11, 2020

numpy/numpy#13516 is a different thing which has runtime cpu detection within numpy itself.

@h-vetinari
Copy link

h-vetinari commented May 11, 2020

Are you 100% sure? It seems heavily dependent on many things at build time:

New Build Arguments

  • --cpu-baseline minimal set of required optimizations, default value is min which provides the minimum CPU features that can safely run on a wide range of users platforms.
  • --cpu-dispatch dispatched set of additional optimizations, default value is max -xop -fma4 which enables all CPU features, except for AMD legacy features.

The new arguments can be reached through build, build_clib, build_ext. [...]

New auto-generated C header (core/src/common/_cpu_dispatch.h)

The new header contains all the definitions and headers of instruction-sets for the CPU baseline and dispatch-able features that have enabled through command arguments --cpu-baseline and --cpu-dispatch.

NPY_HAVE_ is the suffix of CPU features definitions, e.g. NPY_HAVE_SSE2, NPY_HAVE_VSX, NPY_HAVE_NEON.

The default minimum does not include AVX on x86, for example.

But numpy PR aside, IIUC, this should enable to have different compiler builds per (group of) cpu features, right?

@isuruf
Copy link
Contributor Author

isuruf commented May 11, 2020

But numpy PR aside, IIUC, this should enable to have different compiler builds per (group of) cpu features, right?

Yes

@isuruf
Copy link
Contributor Author

isuruf commented May 17, 2020

@jjhelmus, @mingwandroid, @katietz, any thoughts on this?

@isuruf
Copy link
Contributor Author

isuruf commented Jun 19, 2020

ping on this

@isuruf
Copy link
Contributor Author

isuruf commented Jul 2, 2020

ping on this again

@ericpre
Copy link
Contributor

ericpre commented Jul 10, 2020

xref #10057 regarding fixing virtual packages just in case this is relevant.

Copy link
Contributor

@mcg1969 mcg1969 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We finally got CI unstuck so hopefully we can review this soon; I'm afraid I don't know enough to comment intelligently, other than to ask if perhaps a test or two is in order?

@chenghlee
Copy link
Contributor

@isuruf: can you please provide some context for this and (possibly) PR#9461? Having conda provide information about the CPU it's running on seems handy, but I'm curious if you had specific use cases in mind.

Also, with this PR, would you want to make conda explicitly dependent on the archspec package? It seems a little odd to me to have the virtual package build string, and thus what conda info reports, dependent on an optional package.

@isuruf
Copy link
Contributor Author

isuruf commented Jul 31, 2020

@isuruf: can you please provide some context for this and (possibly) PR#9461? Having conda provide information about the CPU it's running on seems handy, but I'm curious if you had specific use cases in mind.

Sure. There are some packages that don't support runtime detection of CPUs and they need to be built for different architectures. With the CPU type available in the solver, we can build different variants of the same package and make sure the correct package is installed.

Also, with this PR, would you want to make conda explicitly dependent on the archspec package? It seems a little odd to me to have the virtual package build string, and thus what conda info reports, dependent on an optional package.

I'm fine with any option.

@leofang
Copy link

leofang commented Aug 5, 2020

@isuruf Could you provide an example here as for how to use this feature, perhaps from recipe maintainers' perspective? I am lost by looking at the changes...

cc: @beckermr (Maybe you knew this already, @isuruf pointed out to me that this PR could help build packages for avoiding avx512.)

@chenghlee chenghlee added this to the 4.9.0 milestone Aug 5, 2020
@isuruf
Copy link
Contributor Author

isuruf commented Aug 5, 2020

@leofang, it's up to the downstream packages on how to use this. I was thinking of adding a cpu_feature meta package to conda-forge so that people can say - cpu_feature * *_haswell in host in conda-build.

@beckermr
Copy link
Contributor

beckermr commented Aug 5, 2020

cc @manodeep on using this in the corrfunc feedstock

@github-actions
Copy link

Hi there, thank you for your contribution to Conda!

This pull request has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue or pull request if needed.

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Aug 20, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants