virtual glibc package #9358

scopatz · 2019-10-21T19:44:39Z

Similar to #9349, implements #9348, and addresses some of the issues in conda-forge/staged-recipes#9902.

CC @mingwandroid @jakirkham @msarahan @jjhelmus @isuruf

conda/core/index.py

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

conda/core/index.py

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

docs/source/user-guide/tasks/manage-virtual.rst

xhochy · 2019-10-21T20:38:39Z

tests/core/test_index.py

@@ -54,6 +54,16 @@ def test_supplement_index_with_system():
    assert cuda_pkg.package_type == PackageType.VIRTUAL_SYSTEM


+def test_supplement_index_with_system_glibc():


This test will fail on non-glibc systems

@scopatz, the tests fail because context.libc_family_version is not overriden by the env variable CONDA_OVERRIDE_GLIBC

xhochy · 2019-10-21T20:39:26Z

conda/core/index.py

@@ -157,6 +158,11 @@ def _supplement_index_with_system(index):
        rec = _make_virtual_package('__cuda', cuda_version)
        index[rec] = rec

+    libc_family, libc_version = context.libc_family_version


I'm wondering a bit on how this will behave on systems where we have musl + glibc-wrapper, e.g. the miniconda3:4.7.10-alpine docker image.

Looking at https://github.com/conda/conda/blob/4.7.12/conda/common/_os/linux.py#L20-L48 , I would assume it gets the glibc configuration (under the premise that you run the Anaconda or conda-forge conda and python packages -- if you would dare to pip install conda into musl-built CPython installation things might turn out differently ;D).

Yes, that image gives you glibc, 2.28

Can you point me to where the tests actually fail? All of the Circle tests I saw seem to fail long before this

I was referring to https://ci.appveyor.com/project/ContinuumAnalyticsFOSS/conda/builds/28297712/job/e0in8bb058fta1ep#L904

docs/source/user-guide/tasks/manage-virtual.rst

Co-Authored-By: Uwe L. Korn <xhochy@users.noreply.github.com>

docs/source/user-guide/tasks/manage-virtual.rst

conda/core/index.py

mbargull · 2019-10-22T15:49:28Z

docs/source/user-guide/tasks/manage-virtual.rst

 * ``CONDA_OVERRIDE_CUDA`` - CUDA version number or set to ``""`` for no CUDA
  detected.
+* ``CONDA_OVERRIDE_GLIBC`` - CUDA version number or set to ``""`` for no GLIBC.


I wasn't aware of #8267 -- IMO we shouldn't have that CONDA_OVERRIDE_CUDA environment variable handling, but we should use conda's builtin configuration capabilities that allow overriding config parameters via environment variables.
Meaning, I would propose a virtual_packages (/system_packages/virtual_system_packages/virtual_package_overrides/something) configuration option akin to pinned_packages/disabled_packages.
(Not saying @scopatz has to implement that in this PR, but it would be very advantageous to have a more general solution in place, rather than introducing separate env vars for every "virtual package".)

Yeah, my goal here is to do the minimum possible for this capability. I just need bluetooth capability, and am already pretty deep in the weeds here 😉

tests/core/test_index.py

mbargull · 2019-10-22T15:59:56Z

conda/core/index.py

@@ -157,6 +158,11 @@ def _supplement_index_with_system(index):
        rec = _make_virtual_package('__cuda', cuda_version)
        index[rec] = rec

+    libc_family, libc_version = context.libc_family_version


Looking at https://github.com/conda/conda/blob/4.7.12/conda/common/_os/linux.py#L20-L48 , I would assume it gets the glibc configuration (under the premise that you run the Anaconda or conda-forge conda and python packages -- if you would dare to pip install conda into musl-built CPython installation things might turn out differently ;D).

Co-Authored-By: jakirkham <jakirkham@gmail.com>

scopatz · 2019-10-22T17:36:56Z

OK I think that this is ready for review again. Also note that CI is still broken in ways unrelated to this PR

conda/core/index.py

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

scopatz · 2019-10-22T17:48:29Z

Also, everyone, please feel free to push to this branch directly.

conda/core/index.py

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

scopatz · 2019-10-22T19:40:51Z

Ok I have the tests running now and the added test passes. CI failures are unrelated.

jjhelmus · 2019-10-22T20:06:36Z

LGTM, thanks for the addition @scopatz

jjhelmus · 2019-10-22T20:08:07Z

FYI, a conda 4.8.0 release is planned for later this week / earlier next week. This will make a nice addition.

scopatz · 2019-10-22T20:11:47Z

Excellent thanks! glad I could get this in under the wire 😉

msarahan · 2019-10-22T20:21:33Z

Thanks for taking point on this @scopatz. We have needed this for a while. It will dramatically improve compatibility messaging on Linux. Thanks for the thorough review, @xhochy @isuruf @mbargull @jakirkham.

github-actions · 2021-08-23T06:02:12Z

Hi there, thank you for your contribution to Conda!

This pull request has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue or pull request if needed.

virtual glibc package

56aa5d5

scopatz requested a review from a team as a code owner October 21, 2019 19:44

cla-bot bot added the cla-signed [bot] added once the contributor has signed the CLA label Oct 21, 2019

This was referenced Oct 21, 2019

systemd conda-forge/staged-recipes#9902

Closed

Add system C library virtual package #9348

Closed

isuruf reviewed Oct 21, 2019

View reviewed changes

conda/core/index.py Outdated Show resolved Hide resolved

isuruf reviewed Oct 21, 2019

View reviewed changes

conda/core/index.py Outdated Show resolved Hide resolved

scopatz and others added 2 commits October 21, 2019 16:19

Apply suggestions from code review

68c3852

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

glibc

3efcb25

isuruf reviewed Oct 21, 2019

View reviewed changes

conda/core/index.py Outdated Show resolved Hide resolved

Update conda/core/index.py

22f0aa1

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

kalefranz reviewed Oct 21, 2019

View reviewed changes

docs/source/user-guide/tasks/manage-virtual.rst Outdated Show resolved Hide resolved

xhochy reviewed Oct 21, 2019

View reviewed changes

docs/source/user-guide/tasks/manage-virtual.rst Outdated Show resolved Hide resolved

xhochy reviewed Oct 21, 2019

View reviewed changes

Update docs/source/user-guide/tasks/manage-virtual.rst

36f36ce

Co-Authored-By: Uwe L. Korn <xhochy@users.noreply.github.com>

jakirkham reviewed Oct 22, 2019

View reviewed changes

docs/source/user-guide/tasks/manage-virtual.rst Outdated Show resolved Hide resolved

mbargull requested changes Oct 22, 2019

View reviewed changes

scopatz and others added 2 commits October 22, 2019 13:18

Update docs/source/user-guide/tasks/manage-virtual.rst

671d230

Co-Authored-By: jakirkham <jakirkham@gmail.com>

updates to glibc

b4c42e4

isuruf reviewed Oct 22, 2019

View reviewed changes

conda/core/index.py Show resolved Hide resolved

Update conda/core/index.py

e72c0a8

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

isuruf reviewed Oct 22, 2019

View reviewed changes

conda/core/index.py Outdated Show resolved Hide resolved

scopatz and others added 4 commits October 22, 2019 14:04

Update conda/core/index.py

2485753

Co-Authored-By: Isuru Fernando <isuruf@gmail.com>

move imports

cc63e21

fix libc version

656420c

linux only test

9341508

jjhelmus approved these changes Oct 22, 2019

View reviewed changes

jjhelmus merged commit f39a6fd into conda:master Oct 22, 2019

scopatz deleted the virt-libc branch October 22, 2019 20:11

isuruf mentioned this pull request Oct 22, 2019

ENH add osx version virtual package #9349

Merged

h-vetinari mentioned this pull request Nov 21, 2019

clarify metadata patching for old python-dependent packages msarahan/pypy-python-structure#3

Open

jakirkham mentioned this pull request Aug 5, 2020

Plugin mechanism for virtual packages #10131

Closed

github-actions bot added the locked [bot] locked due to inactivity label Aug 23, 2021

github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2021

		@@ -54,6 +54,16 @@ def test_supplement_index_with_system():
		assert cuda_pkg.package_type == PackageType.VIRTUAL_SYSTEM


		def test_supplement_index_with_system_glibc():

virtual glibc package #9358

virtual glibc package #9358

Uh oh!

Conversation

scopatz commented Oct 21, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scopatz commented Oct 22, 2019

Uh oh!

Uh oh!

scopatz commented Oct 22, 2019

Uh oh!

Uh oh!

scopatz commented Oct 22, 2019

Uh oh!

jjhelmus commented Oct 22, 2019

Uh oh!

jjhelmus commented Oct 22, 2019

Uh oh!

scopatz commented Oct 22, 2019

Uh oh!

msarahan commented Oct 22, 2019

Uh oh!

github-actions bot commented Aug 23, 2021

Uh oh!

Uh oh!