Cosmetic issue while naming a test using pytest 6.0.1 compared to previous versions #7686
Description
We just hit another issue with pytest 6.0.1. Although, this seems like minor issue from the outside didn't make sense at all until I started to dig out what was happening.
Some background of what we do, but pretty much every test we create needs to be parametrized with some permutations in order to fully support certain constrains that need to be tested. In order to accomplish this, we need to find all permutations possible and that way we parametrized accordingly.
Knowing this, using pytest 6.0.1, the name of test case that has an empty parametrization generates an name that didn't make sense NotSetType.token which I assume is some sort of a variable coming somewhere in the pytest code? 🤔
Anyways, before the name of the test case used to match whatever the name of the parametrization was.
These are the steps to repro the issue
conftest.py
def pytest_generate_tests(metafunc):
permutations = []
ids = []
metafunc.parametrize("configuration", permutations, ids=ids)
test_configuration.py
def test_param(configuration):
pass
On pytest 6.0.1:
$ pytest -v test_configuration.py
============================================================================================= test session starts ============================================================================================== platform win32 -- Python 3.7.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- c:\depot\bitbucket\test\configuration\venv\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\depot\bitbucket\test\configuration
collected 1 item
test_configuration.py::test_param[NotSetType.token] SKIPPED [100%]
============================================================================================== 1 skipped in 0.02s ==============================================================================================
On pytest 5.4.1 and before we used to get:
$ pytest -v test_configuration.py
============================================================================================= test session starts ============================================================================================== platform win32 -- Python 3.7.2, pytest-5.4.1, py-1.9.0, pluggy-0.13.1 -- c:\depot\bitbucket\test\configuration\venv\scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\depot\bitbucket\test\configuration
collected 1 item
test_configuration.py::test_param[configuration0] SKIPPED [100%]
============================================================================================== 1 skipped in 0.02s ==============================================================================================
Note down, it used to be configuration0 instead of NotSetType.token. Configuration0 used to make sense, but now I am kind of clueless into why NotSetType.token is obtained.
In case there is a question into why permutations and ids are emtpy, I just want to emphasize about what I said in the beginning, we dynamically generate paramtrization in order to run test under certain constraints, and empty permutation/ids basically means that test cannot be run under those constraints. Hence, test is not supported, so it is skipped.
Not sure if this was intended and it is minor, but I thought it was still worthwhile to let you know.
Thanks