Skip to content

Direct references should use direct_url hashes as key for sync #1884

@siddharthab

Description

@siddharthab

pip-sync currently assumes that the URL has an egg fragment with a version specifier [1], but that behavior was never in the specification, and so can not be relied upon. Moreover, pip-compile actually actively removes egg fragments to create a direct reference [2]. Direct references rely on hashes rather than versions as the specifier and there is no way to specify a version for them anyway. We should be using direct_url.hash attribute of the installed dist [3] and comparing that with the hash fragment of the direct reference in requirements.txt. The current distribution type that is being used in sync.diff is a narrower type [4] that does not have this information, but a small refactor can make the information available.

[1]

For URL requirements, only provide a useful key if the url includes

[2]
def _build_direct_reference_best_efforts(ireq: InstallRequirement) -> str:

[3] https://github.com/pypa/pip/blob/8a1eea4aaedb1fb1c6b4c652cd0c43502f05ff37/src/pip/_internal/models/direct_url.py#L105
[4]
return [cast(Distribution, dist)._dist for dist in dists]

Environment Versions

  1. OS Type: Debian 12
  2. Python version: $ python -V: Python 3.11.4
  3. pip version: $ pip --version: pip 23.1.2 ...
  4. pip-tools version: $ pip-compile --version: pip-compile, version 6.13.0

Steps to replicate

  1. Create a requirements.txt file:
urllib3 @ https://github.com/urllib3/urllib3/archive/refs/tags/2.0.3.zip#sha1=bc7f779fbdba2befc592eb35a6353cff0109d622
  1. In a clean virtual environment, pip-sync.
  2. Check if the installed dist and the specified requirement match: pip-sync -n.

Expected result

Expected that the rerun of pip-sync will not try to re-install the req.

Actual result

pip-sync re-installs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is not working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions