Update linters and python version for linting in CI #15200

hamdanal · 2023-05-06T15:54:21Z

I updated the python version used for linting to 3.10. This gives us some time before we have to update again.

The only downside I see is if someone is using a 3.7 python environment locally and they run linters in that environment or if they installed tox with python 3.7 and they run tox run -e lint. A note can be added in the contributing guide to tell users what python versions are supported for dev tools. Another solution would be to delay merging this PR until python 3.7 support is dropped altogether but in this case the pre-commit-ci updates and the #15197 situation would happen again.

(cc @AlexWaygood)

AlexWaygood

I think we should add target-version = ['py37'] to this section of pyproject.toml:

mypy/pyproject.toml

Lines 20 to 28 in b69060a

    
           [tool.black] 
        
           line-length = 99 
        
           target-version = ["py37", "py38", "py39", "py310", "py311"] 
        
           skip-magic-trailing-comma = true 
        
           force-exclude = ''' 
        
           ^/mypy/typeshed| 
        
           ^/mypyc/test-data| 
        
           ^/test-data 
        
           '''

And py_version = 37 to this section:

mypy/pyproject.toml

Lines 30 to 40 in b69060a

    
           [tool.isort] 
        
           profile = "black" 
        
           line_length = 99 
        
           combine_as_imports = true 
        
           skip_gitignore = true 
        
           extra_standard_library = ["typing_extensions"] 
        
           skip_glob = [ 
        
               "mypy/typeshed/*", 
        
               "mypyc/test-data/*", 
        
               "test-data/*", 
        
           ]

If we're going to start running our linters on a newer version of Python than the oldest version we support

AlexWaygood · 2023-05-06T16:04:43Z

You could use environment markers to ensure the py38+ requirements don't get installed on Python 3.7 in CI, like we do over at typeshed: https://github.com/python/typeshed/blob/853d01d55ef5b5c7f0733e62c2e214be1d472657/requirements-tests.txt#L3-L7

… `pyproject.toml`. Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

hamdanal · 2023-05-07T17:37:46Z

I think we should add target-version = ['py37'] to this section of pyproject.toml:

This is already added here

mypy/pyproject.toml

Line 22 in b69060a

target-version = ["py37", "py38", "py39", "py310", "py311"]

And py_version = 37 to this section:

Done, thanks.

I also set skip_install = true in the tox configuration for the lint environment as installing the project itself is not needed to run any of the linters in this environment. It made the lint using tox twice as fast on my machine (33 seconds to 15 seconds, clearing the cache before the run)

AlexWaygood · 2023-05-07T17:39:49Z

I think we should add target-version = ['py37'] to this section of pyproject.toml:

This is already added here

mypy/pyproject.toml

Line 22 in b69060a

target-version = ["py37", "py38", "py39", "py310", "py311"]

🤦‍♂️ Not sure how I missed that!

AlexWaygood · 2023-05-07T17:43:31Z

Possibly we should add a sentence here to say that you'll need Python 3.8+ to install the test requirements: https://github.com/python/mypy/blob/master/CONTRIBUTING.md#4-install-the-test-requirements-and-the-project

CONTRIBUTING.md

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

AlexWaygood · 2023-05-07T18:22:37Z

.github/workflows/test.yml

@@ -97,7 +97,7 @@ jobs:
          # but it's useful to run them with tox too,
          # to ensure the tox env works as expected
        - name: Formatting with Black + isort and code style with flake8
-          python: '3.7'
+          python: '3.10'


Any reason not to use 3.11 here?

Suggested change

python: '3.10'

python: '3.11'

It felt slightly safer to use 3.10 😆. I still feel that 3.11 is new :P.

AlexWaygood

LGTM

github-actions · 2023-05-07T18:38:32Z

Diff from mypy_primer, showing the effect of this PR on open source code:

pandas-stubs (https://github.com/pandas-dev/pandas-stubs) got 1.14x slower (54.3s -> 62.2s)

scikit-learn (https://github.com/scikit-learn/scikit-learn) got 1.14x faster (55.8s -> 49.0s)

hauntsaninja

Thanks! Nice win on tox lint

Update linters

7b7ca52

hamdanal marked this pull request as draft May 6, 2023 15:55

AlexWaygood reviewed May 6, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

hamdanal and others added 3 commits May 7, 2023 00:28

Add environment markers to test-requirements.txt and min version to…

7816f9e

… `pyproject.toml`. Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

Do not install self in tox lint environment

c11abad

Add a note to the dev guide

c44608a

hamdanal marked this pull request as ready for review May 7, 2023 17:38

This comment has been minimized.

Sign in to view

AlexWaygood reviewed May 7, 2023

View reviewed changes

CONTRIBUTING.md Outdated Show resolved Hide resolved

Update CONTRIBUTING.md

0298ca3

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

AlexWaygood reviewed May 7, 2023

View reviewed changes

AlexWaygood approved these changes May 7, 2023

View reviewed changes

hauntsaninja approved these changes May 7, 2023

View reviewed changes

hauntsaninja merged commit 6283516 into python:master May 7, 2023

hamdanal deleted the update-linters branch May 8, 2023 05:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Update linters and python version for linting in CI #15200

Update linters and python version for linting in CI #15200

Uh oh!

hamdanal commented May 6, 2023

Uh oh!

AlexWaygood left a comment

Uh oh!

AlexWaygood commented May 6, 2023

Uh oh!

This comment has been minimized.

hamdanal commented May 7, 2023

Uh oh!

AlexWaygood commented May 7, 2023

Uh oh!

AlexWaygood commented May 7, 2023

Uh oh!

This comment has been minimized.

Uh oh!

AlexWaygood May 7, 2023

Uh oh!

hamdanal May 7, 2023

Uh oh!

AlexWaygood left a comment

Uh oh!

github-actions bot commented May 7, 2023

Uh oh!

hauntsaninja left a comment

Uh oh!

Uh oh!

	[tool.black]
	line-length = 99
	target-version = ["py37", "py38", "py39", "py310", "py311"]
	skip-magic-trailing-comma = true
	force-exclude = '''
	^/mypy/typeshed\|
	^/mypyc/test-data\|
	^/test-data
	'''

	[tool.isort]
	profile = "black"
	line_length = 99
	combine_as_imports = true
	skip_gitignore = true
	extra_standard_library = ["typing_extensions"]
	skip_glob = [
	"mypy/typeshed/*",
	"mypyc/test-data/*",
	"test-data/*",
	]

Uh oh!

Update linters and python version for linting in CI #15200

Update linters and python version for linting in CI #15200

Uh oh!

Conversation

hamdanal commented May 6, 2023

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

AlexWaygood commented May 6, 2023

Uh oh!

This comment has been minimized.

hamdanal commented May 7, 2023

Uh oh!

AlexWaygood commented May 7, 2023

Uh oh!

AlexWaygood commented May 7, 2023

Uh oh!

This comment has been minimized.

Uh oh!

AlexWaygood May 7, 2023

Choose a reason for hiding this comment

Uh oh!

hamdanal May 7, 2023

Choose a reason for hiding this comment

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 7, 2023

Uh oh!

hauntsaninja left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!