refactor: update to coverage 7.9.2 + cleanup #9256
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(there's only one subclass)
* Extract coverage tools to ietf.utils.coverage * Revert to starting checker in settings_test Does not exactly match previous coverage reports. Need to investigate.
Has not been populated in quite some time
Setting up to test this will be complex and we'll notice other test failures/coverage weirdness if this does not behave.
Merged
Way too meta
rjsparks
approved these changes
Aug 4, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upgrades to the current version of coverage 7.9.2. Refactors the code organization to separate the parts that wrangle coverage from the rest of the test_runner. Removes some unused / unneeded bits of code.
A major difference in the new (5.0+) coverage is that its internal data are stored as an sqlite database instead of json. As far as I can see, this is not significant for our purposes. We work with what coverage calls a "report" rather than with its own internal data. This PR refactors the old code with the fairly small set of changes needed to adapt to some reorganization in the coverage project and produces the same JSON coverage data that datatracker has been using in recent years.
Excluding lines / blocks from coverage checking is now done with the
# pragma: no covermagic comment. The old method, using a context manager, was nice but is not easily translated into what the modern coverage package makes possible. A context manager to disable template and url coverage is still used, now calleddisable_coverage(). This needs to be used together with the pragma to turn off all types of coverage checks.I've added tests to ensure that the new
CoverageManagerclass orchestrates things correctly and, as a safeguard, turns into a no-op if it is somehow called in production mode. I did not add tests for the stats reporting module because errors in it are likely to cause test failures, coverage drops, or other problems that we'll notice through other means. Testing it will require pretty involved mocking that didn't seem worth it.Comparing the coverage stats generated by this refactor against the 12.43.1 release coverage finds very good but not exact agreement. There are many trivial format changes ("0" and "1" become "0.0" and "1.0"; rounding of floats is different in the 10^-9 decimal place or so), a few real but insignificant changes that I believe have to do with which lines coverage treats as coverable, and some changes to do with minor configuration changes between dev and GHA testing. None of these affects more than a few lines in any file and the overwhelming majority of files have exactly the same coverage stats.
Regarding speed, according to
timethe full test suite takes a total of 14:53.50 (871 seconds reported by the test runner) on my laptop. The current main branch using py39 and coverage 4.5.4 takes a total of 14:26.69 (855 seconds reported by the test runner). That's a negligible difference.