Skip to content

[2.11] Bump setuptools to avoid pysolr build errors with setuptools-scm #8699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 26, 2025

Conversation

amercader
Copy link
Member

The CI tests for dev-v2.10 and dev-v2.11 broke a few days ago with the same error

Collecting pysolr==3.9.0 (from -r requirements.txt (line 102))
  Downloading pysolr-3.9.0.tar.gz (55 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /tmp/pip-install-iyrzysc_/pysolr_1ac89b9533694450b0e9e43d23a237e7/.eggs/setuptools_scm-8.2.0-py3.9.egg/setuptools_scm/_integration/setuptools.py:31: RuntimeWarning:
      ERROR: setuptools==58.1.0 is used in combination with setuptools-scm>=8.x

      Your build configuration is incomplete and previously worked by accident!
      setuptools-scm requires setuptools>=61

      Suggested workaround if applicable:
       - migrating from the deprecated setup_requires mechanism to pep517/518 and using a pyproject.toml to declare build dependencies which are reliably pre-installed before running the build tools

Full error: https://gist.github.com/amercader/b047186b9ab6cab0e8e47cd87b1aeebe

Summary of a long and convoluted investigation:

  • Both CKAN 2.10 and 2.11 use pysolr==3.9.0 and are tested on the python:3.9-bullseye image
  • This combination no longer works because pysolr uses setuptools-scm and the latest version (8.2) does not work with the setuptools version the image has (58.*)
  • All higher version Python images work, because of higher setuptools versions
  • CKAN master works because pysolr 3.10.0 does not have this issue (I guess they changed the way build requirements are defined).
  • So we either:
    1. Upgrade pysolr on dev-v2.10 and dev-v2.11
    2. Upgrade setuptools in our particular test environment

I went with option 2 because it's the more straightforward and less likely to break things on existing sites but happy to reconsider.

This PR targets 2.11, on 2.10 we still have the requirement-setuptools.txt file so I bumped the requirement there (0bf1c7d).

Refs:

pypa/setuptools-scm#1112
pypa/setuptools-scm#1111

Error:

https://gist.github.com/amercader/b047186b9ab6cab0e8e47cd87b1aeebe

Collecting pysolr==3.9.0 (from -r requirements.txt (line 102))
  Downloading pysolr-3.9.0.tar.gz (55 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /tmp/pip-install-iyrzysc_/pysolr_1ac89b9533694450b0e9e43d23a237e7/.eggs/setuptools_scm-8.2.0-py3.9.egg/setuptools_scm/_integration/setuptools.py:31: RuntimeWarning:
      ERROR: setuptools==58.1.0 is used in combination with setuptools-scm>=8.x

      Your build configuration is incomplete and previously worked by accident!
      setuptools-scm requires setuptools>=61

      Suggested workaround if applicable:
       - migrating from the deprecated setup_requires mechanism to pep517/518
         and using a pyproject.toml to declare build dependencies
         which are reliably pre-installed before running the build tools

Refs:

pypa/setuptools-scm#1112
pypa/setuptools-scm#1111
@amercader
Copy link
Member Author

Second commit should be cherry-picked to master as well ffcc081

@amercader
Copy link
Member Author

@EricSoroos can you review this one to fix the tests in dev-v2.11 (e.g #8726) cheers

@EricSoroos
Copy link
Contributor

Other option would be to pin setuptools-scm<8 in requirements.txt -- pysolr has it as an unpinned dependency, so we're just going to get the current version.

Updating setuptools in the test infrastructure is good in general, and should be safe enough.

@amercader amercader merged commit a7e35fd into dev-v2.11 Mar 26, 2025
5 of 6 checks passed
@amercader amercader deleted the pysolr-build-errors branch March 26, 2025 14:52
amercader added a commit that referenced this pull request Apr 10, 2025
Use apt-get to avoid warning, install latest setuptools as per
#8699
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants