Skip to content

Reduce randomness in doctests #35522

@tobiasdiez

Description

@tobiasdiez

Is there an existing issue for this?

  • I have searched the existing issues for a bug report that matches the one I want to file, without success.

Problem Description

Currently a few doctests use random input (e.g. random_element()), which leads to unpredictable doctest failures (since depending on the input the tests might fail) and unreliable code coverage metrics (since depending on the input different code paths are executed).

Proposed Solution

Replace random input by deterministic/static input. If a test requires many different inputs to be convincing, it can be refactored using pytest parameterized tests.

Alternatives Considered

Accepts random test failures and unreliable coverage metrics.

Additional Information

A good starting point to see which tests are particularly affected is to go to https://app.codecov.io/gh/sagemath/sage/pulls, pick a PR at random, and look at the "Indirect changes" tab. These are changes in the coverage metrics that (most likely) are not coming from changes in the PR. For example, comparing https://app.codecov.io/gh/sagemath/sage/pull/35521/indirect-changes and https://app.codecov.io/gh/sagemath/sage/pull/35507/indirect-changes indicates that the tests in the following files are prone to lead different coverage:

and a few more. This can then be compared to the codecov report for the develop branch. For example, this line https://app.codecov.io/gh/sagemath/sage/blob/develop/src/sage/modular/modsym/boundary.py#L1385 is not covered in the develop branch but covered in the the PRs.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions