pep518 support #14

msftcangoblowm · 2025-01-22T08:10:14Z

closes #12

refactor: pep518 support

Adds dependency flake8-pyproject to requirements-dev.in

The intent is minimal changes, but some fixes were necessary

tests: pytest requires tests folder to have an empty __init__.py
fix(MANIFEST.in): ensure all support files in tarball
tests(conftest.py): pytest plugin name not logassert.pytest_plugin

These fixes are nitpicky

ci(tests): kwargs before positional args
tests: when called by unittest unittest modules require process shield

Recommendations not in the commit

flat layout should be src layout. The flat layout is confusing flake8 causing it to inspect all packages in venv not just logassert. F821 is ignored until this issue is resolved.

To convert to a src-layout:

mkdir -p src/logassert
mv logassert src/logassert

In pyproject.toml, uncomment tool.setuptools.packages.find.where

Requests

apply patches in chronological order to avoid existing PRs needing to be rebased
any changes, just ask. Will make them

- refactor: pep518 support - ci(tests): kwargs before positional args - fix(MANIFEST.in): ensure all support files in tarball - tests: pytest requires tests folder to have an empty __init__.py - tests(conftest.py): pytest plugin name not logassert.pytest_plugin - tests: when called by unittest unittest modules require process shield

facundobatista

Hello! Thanks for these changes! Please check some details and questions.

Regarding flake8, it was working before so we don't really need to ignore anything.

Thanks again!

tests/test_classic.py

.github/workflows/tests.yml

MANIFEST.in

facundobatista · 2025-01-22T12:56:08Z

requirements-dev.in

@@ -1,3 +1,4 @@
 flake8
+flake8-pyproject


Can we put the development dependencies in pyproject.toml?

There are two routes lets discuss both and their real ramifications:

Dynamic optional-dependencies

[project] dynamic = ["optional-dependencies"] [tool.setuptools.dynamic] optional-dependencies.dev = { file = ['requirements-dev.in'] }

Static optional-dependencies

[project.optional-dependencies] dev = [ "flake8", "flake8-pyproject", "pytest", "structlog", ]

The disadvantages of static dependencies:

managing requirements

Packages intended for managing requirements files are no longer an option.
Coerced towards a preference to eventually accept either poetry or uv.

There are packages specifically for managing requirements files rather than
venvs.

target venv?

No mention of target virtual environment. Naively assumes there will ever
only be one.

lock files

Hardening security aside, more important to be able to reproduce dev env, by specifying all dependencies' version.

Would take minimal effort to lock down the dev environment and periodically update it

Ur informed choice

Which shall it be: dynamic or static?

If dynamic, lock dev env or keep it unlocked?

If dynamic, create a requirements/ folder, move requirements files over , remove prepended requirements- from file names

Some ideas, with simplicity in mind:

I will not switch to using uv or poetry; let's keep using pip (and will not use a lock file)

project.optional-dependencies is not a real choice, those are "if they are only needed for a specific feature of your package"; this is not about optional features, but development dependencies

I will not add a requirements directory

With that in mind, is there a way to put the development dependencies in pyproject.toml and make pip install them from there?

There are two types of extras (for maintainers and for end users), but no way to distinguish the two. Both fall under optional dependencies.

pip will work just fine with unlocked static optional dependencies.

facundobatista · 2025-01-22T12:58:20Z

pyproject.toml

+"Source code" = 'https://github.com/facundobatista/logassert'
+
+[tool.setuptools.packages.find]
+# where = ["src"]


Why this commented line?

Locally, flake8 is having a hard time with the flat package layout. Would be resolved by moving to a src package layout.

logassert -> src/logassert

The current situation, flat package layout, locally, flake8 looks thru virtualenv env/ at every single package. Many vendored packages are using six for py27 compatibility. flake8 generates lots of F821 undefined name messages.

Since this action might be surprising, wanted to discuss it and form consensus on the move over to the normal src package layout.

The line is to make the transition from flat package to src package layout smooth.

The transition process

Rename/move logassert/ folder as src/logassert then uncomment the line.

I will not move from logassert to src; please focus the PR in using pyproject.toml which is what I agreed and not changing the project's structure.

Regarding flake8, it is currently working ok in master. So if after your changes it's not working anymore, it may be one of your changes that need to be revisited...

In master (within gh workflow) works fine; there is no virtual environment folder.

Removed the comment line

Specifying the folders ensures flake8 does not go thru the venv folder.

Executes exponentially quicker.

Was hitting the CPU hard and taking a long time to complete.

flake8 --count --select=E9,F63,F7,F82 --show-source --statistics logassert/ tests/ flake8 --count --exit-zero --max-complexity=10 --max-line-length=99 --statistics logassert/ tests/

msftcangoblowm · 2025-01-23T05:46:26Z

Tests are failing cuz the logs plugin is not being discovered.

The plugin (logassert) and plugin dotted path (only one plugin allowed) is specified in pyproject.toml. The normal use scenario is installing logassert

Will add this step:

    - name: Install for plugin discovery
      run: |
        python -m pip install --disable-pip-version-check --no-deps -q .

The local path hack has already been removed:

tests/conftest.py

pytest_plugins = ["logassert.pytest.plugin"]

What it looks like now

pytest_plugins = [
    "logassert",
    "pytester",
]

- ci: install project before running pytest - tests(test_classic): remove process shield

- ci: install package and dependencies before flake8

- ci: remove --no-deps flag

facundobatista

Hey! This is much better shaped!! Cool.

Please check a comment regarding content in MANIFEST.in... btw, when is this file used?

Thanks!!

facundobatista · 2025-01-24T23:03:58Z

MANIFEST.in

+include test
+
+# docs (build converts symlink to copy of README.md)
+graft docs


Why all this new stuff?

.github, .mkdocs, .readthedocs.yml are very project specific

developmnet requirements and tests should not be distributed

docs is just a link to the README...

From distribution will remove all project, development, and tests

Also will remove unnecessary explicit includes (README.md and LICENSE are automagically included)

Do not understand why docs/ even exists. Placeholder? Removed that as well. Normally docs is where mkdocs or Sphinx files/folders would go.

MANIFEST.in... btw, when is this file used?

Running python -m build will create dist/ and places tarball and whl in there.

Been opening the tarball in xarchiver (a GUI that opens compressed files) to see what source files are being distributed and then adjusting MANIFEST.in

If setuptools-scm were used, would automatically include package files. MANIFEST.in would only be needed to exclude files and folders.

- chore: from distribution exclude project, development, and tests

facundobatista

Awesome, thanks for all this work. I'll merge and adjust any further detail if needed. Thanks again!

facundobatista requested changes Jan 22, 2025

View reviewed changes

msftcangoblowm added 3 commits January 24, 2025 04:58

dev static dependencies into pyproject.toml

1551fe9

- ci: install project before running pytest - tests(test_classic): remove process shield

install package and dependencies before flake8

f55e07b

- ci: install package and dependencies before flake8

remove --no-deps flag

d1ea309

- ci: remove --no-deps flag

facundobatista requested changes Jan 24, 2025

View reviewed changes

adjust MANIFEST.in

31530ac

- chore: from distribution exclude project, development, and tests

facundobatista approved these changes Jan 29, 2025

View reviewed changes

facundobatista merged commit 77c91e0 into facundobatista:master Jan 29, 2025
4 checks passed

pep518 support #14

pep518 support #14

Uh oh!

Conversation

msftcangoblowm commented Jan 22, 2025

Recommendations not in the commit

Requests

Uh oh!

facundobatista left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msftcangoblowm Jan 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Dynamic optional-dependencies

Static optional-dependencies

Ur informed choice

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msftcangoblowm Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msftcangoblowm Jan 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msftcangoblowm Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msftcangoblowm commented Jan 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facundobatista left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facundobatista left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

msftcangoblowm Jan 23, 2025 •

edited

Loading

msftcangoblowm Jan 24, 2025 •

edited

Loading

msftcangoblowm Jan 23, 2025 •

edited

Loading

msftcangoblowm Jan 24, 2025 •

edited

Loading

msftcangoblowm commented Jan 23, 2025 •

edited

Loading