Skip to content

Fix flakey integration tests #2420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 20, 2025
Merged

Fix flakey integration tests #2420

merged 2 commits into from
Jun 20, 2025

Conversation

michaeldwan
Copy link
Member

@michaeldwan michaeldwan commented Jun 19, 2025

We have a few integration tests that fail randomly due to a race, eg:

FAILED test_integration/test_build.py::test_fast_build - IsADirectoryError: [Errno 21] Is a directory: '/home/runner/work/cog/cog/test-integration/test_integration/fixtures/fast-build/weights.h5'

The issue is that several integration tests alter the same fixture directory during a test. If the tests run concurrently, it's likely the weights.h5 file will be removed by the first before the second is done with it.

This PR adds a fixture helper that'll give each test a unique copy of a fixture that gets cleaned up automatically. I only updated the flakey tests, but this is probably good to add to the rest since things like .cog temp dirs often linger as well.

@michaeldwan michaeldwan requested review from 8W9aG and a team June 20, 2025 15:39
@michaeldwan michaeldwan merged commit 414766d into main Jun 20, 2025
26 checks passed
@michaeldwan michaeldwan deleted the md/fix-flaky-test-weights branch June 20, 2025 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants