CI: add a GPU CI job #22275
CI: add a GPU CI job #22275
Conversation
Closes scipy issue 21740.
|
Note that the job itself won't run on this PR before merging. Here's a CI log that's less than an hour old. |
How about merge and iterate then? Unless you want to fix regression before merging |
I do want the job to pass before merging, otherwise it will make CI on all PRs red. Help is very welcome - most issues are recent regressions for PRs that weren't tested on GPU. I won't be able to do more before Friday at least. It may be a matter of some skips in |
|
#22279 fixes all torch and jax.numpy failures I see locally. |
|
All tests pass after gh-22279, thanks again @ev-br and @lucascolley. Test suite runtime is reasonable at the moment: If we enable much more functionality on GPU then we may end up splitting it, but for now it's easier to have it in a single job. Still optimizing cache strategies a bit, then this should be ready. |
|
Caching performance after the changes I'm about to push, that's about as good as it's going to get: |
|
Okay, ready for final review. |
|
See https://github.com/scipy/scipy/actions/workflows/gpu-ci.yml for more logs if you want to see what the changes do. |
draft for now because not all tests pass yet. these are recent regressions in
main, and I'd prefer for those to be fixed in another PR firstccachebuilds are very fast and the setup time is dominated by the ~3 GB worth of packages that are needed for GPU installs of PyTorch, JAX and CuPy in a single conda environment.pixi.tomlinside.github/workflows/for now, because I don't (yet) want to cross the bridge of enabling Pixi for all SciPy contributors (there's too much logistics to work out still there for how to do lock file updates).cd .github/workflowsand then running the samepixi run test-cuda ...command as the failing CI job step.Closes gh-21740