-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Closed
Closed
Copy link
Labels
T: bugSomething isn't workingSomething isn't working
Description
Describe the bug
An extremely strange bug concerning development:
If the number of testcases collected by pytest exceeds 367, and pytest is run with the argument --numprocesses
with a value of 4
, 6
, or 8
, the test cases test_get_sources_with_stdin_filename_and_exclude
and test_get_sources_with_stdin_filename_and_extend_exclude
fail.
To Reproduce
- Clone this repo, as of writing the most recent commit on main is 1abcffc.
- Follow the guide to install the dependencies.
- Run
tox -e py -- --numprocesses 4
and verify that all tests pass. - Note the number of collected test items, currently it is 355.
- Raise the number to 368, for example by creating files in
tests/data/cases
, by adding parameters to a parameterized test or by appending the below to a test file, e.gtests/test_black.py
def test_nothing_1():
pass
def test_nothing_2():
pass
def test_nothing_3():
pass
def test_nothing_4():
pass
def test_nothing_5():
pass
def test_nothing_6():
pass
def test_nothing_7():
pass
def test_nothing_8():
pass
def test_nothing_9():
pass
def test_nothing_10():
pass
def test_nothing_11():
pass
def test_nothing_12():
pass
def test_nothing_13():
pass
- Run
tox -e py -- --numprocesses 4
and expect the following failures:
FAILED tests/test_black.py::TestFileCollection::test_get_sources_with_stdin_filename_and_exclude - AssertionError: assert [] == [PosixPath('_...xclude/a.py')]
FAILED tests/test_black.py::TestFileCollection::test_get_sources_with_stdin_filename_and_extend_exclude - AssertionError: assert [] == [PosixPath('_...xclude/a.py')]
- Remove one of the test cases.
- Run
tox -e py -- --numprocesses 4
and verify that all tests pass again.
Expected behavior
Well, not that.
Environment
$ black --version
black, 24.2.1.dev11+g1abcffc.d20240314 (compiled: no)
Python (CPython) 3.11.8
Pytest environment output:
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
cachedir: .tox/py/.pytest_cache
rootdir: /home/samson/PycharmProjects/black
configfile: pyproject.toml
plugins: xdist-3.5.0, cov-4.1.0
OS: EndeavourOS Linux x86_64
Kernel: 6.7.9-arch1-1
Additional context
I've reproduced this on one other machine.
That's not a large sample size, but the consistency of exactly those two tests failing for the same --numprocesses
values and the same number of tests makes me think it's a real issue.
Metadata
Metadata
Assignees
Labels
T: bugSomething isn't workingSomething isn't working