Version: pytest 5.4.3, current master

pytest caches the evaluation of the string in e.g. @pytest.mark.skipif("sys.platform == 'win32'"). The caching key is only the string itself (see cached_eval in _pytest/mark/evaluate.py). However, the evaluation also depends on the item's globals, so the caching can lead to incorrect results. Example:

# test_module_1.py
import pytest

skip = True

@pytest.mark.skipif("skip")
def test_should_skip():
    assert False

# test_module_2.py
import pytest

skip = False

@pytest.mark.skipif("skip")
def test_should_not_skip():
    assert False

Running pytest test_module_1.py test_module_2.py.

Expected: test_should_skip is skipped, test_should_not_skip is not skipped.

Actual: both are skipped.

I think the most appropriate fix is to simply remove the caching, which I don't think is necessary really, and inline cached_eval into MarkEvaluator._istrue.