-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Legacy Python removal #11364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Legacy Python removal #11364
Conversation
pre-commit.ci autofix |
131b1d6
to
e5cfc47
Compare
e1d87d5
to
1538bb8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jezdez All passing! Given that this PR is over a month old I suggest we merge this in and continue additional work in another legacy removal PR (e.g. removing odict
/OrderDict
).
@kenodegard You're totally correctly, let's start here and iterate over it. |
I'm under the impression that |
… the CI (#4028) After some debugging, I think I have found the potential root cause of the flaky missing dependencies on MacOS. It is in the conda clone step in which only conda packages are copied over but not pip (untracked) files. For example, in this failure example https://github.com/pytorch/pytorch/actions/runs/4692698401/jobs/8318918238 had: ``` Source: /Users/ec2-user/runner/_work/_temp/conda-python-3.9.12 Destination: /Users/ec2-user/runner/_work/_temp/conda_environment_4692698401 Packages: 58 Files: 0 <--- Nothing here ``` And it used `/Users/ec2-user/runner/_work/_temp/miniconda/bin/conda` as reported by `which conda`. This conda is correct but old 4.12.0 (https://github.com/pytorch/test-infra/blob/main/.github/actions/setup-miniconda/action.yml#L15). On the other hand, when working correctly, these files should have been copied over to the cloned env: ``` Source: /Users/ec2-user/runner/_work/_temp/conda-python-3.9.12 Destination: /Users/ec2-user/runner/_work/_temp/conda_environment_4683905314 Packages: 58 Files: 13010 <-- These files are copied over ``` This conda here was `/opt/homebrew/Caskroom/miniconda/base/bin/conda` on M1 and `/usr/local/bin/conda` on x86_64. Both are newer (23.1.0). It seems that **we unknowingly use the newer system-wide conda on MacOS**, which is not part of the CI. So the attempt fix here is to: * Upgrade MacOS conda to [23.1.0](https://repo.anaconda.com/miniconda/Miniconda3-py39_23.1.0-1-MacOSX-arm64.sh). My theory is that the conda clone issue was fixed by this conda/conda#11364 * Always use `/Users/ec2-user/runner/_work/_temp/miniconda/bin/conda` as this is supposed to be the one controlled by the CI * Check if the number of untracked files (pip dependencies) is 0. If it is, reinstall pip requirements directly on the cloned conda environment ### Testing https://github.com/pytorch/pytorch/actions/runs/4695636286/jobs/8338451917 * Use `/Users/ec2-user/runner/_work/_temp/miniconda/bin/conda` * Use `conda 23.1.0` * Files are copied correctly
This starts a longer journey in removing Python 2 related code. I've tried to separate the code into individual commits to it's easier to review.