What's the problem this feature will solve?

I have a single requirements.in file, I would like to generate a single requirements.txt file, that would work with multiple python versions.
Afaik, currently, the only way to solve this issue is to run pip-compile N times - once per python version.
This however leaves you with N different requirements.txt files with mutually unsynchronized versions.

Disclamer:

I am aware of the limitations of pip/PyPI dependency resolution (#635, #639).
I am not asking pip-tools to guess the dependencies for different python versions.

I am willing to run pip-compile for each python version that I want to support.
What I want is a way to aggregate the results of these multiple runs of pip-compile.

Example problem

Let's say, that my requirements.in contains just required-package (this is a hypothetical package, that doesn't actually exist).

required-package has the following versions in different versions of python:

• python3.6 has required-package 1.0.0 which requires packages a and b
• python3.7 has required-package 1.0.0 which requires packages a and c
• python3.8 has required-package 1.0.0 which requires just the package c
python3.8 also has a newer version - required-package 2.0.0 which is only available in python >= 3.8 and requires just the package d
  For simplicity, let's assume that all other packages don't have any further dependencies and have only one version - 1.0.0.

If I understand it correctly, currently the only thing I can do is create 3 requirements.txt, one for each python version:

# python3.6 -m piptools compile requirements.in --no-header --no-annotate --output-file=requirements36.txt
required-package==1.0.0
a==1.0.0
b==1.0.0

# python3.7 -m piptools compile requirements.in --no-header --no-annotate --output-file=requirements37.txt
required-package==1.0.0
a==1.0.0
c==1.0.0

# python3.8 -m piptools compile requirements.in --no-header --no-annotate --output-file=requirements38.txt
required-package==2.0.0
d==1.0.0

There are 2 big problems with the above result:

1. 3.6 and 3.7 require different packages b and c
2. 3.8 has a different version of required-package

Desired result

Ideally, I would like to have a common.requirements.txt along the lines of

required-package==1.0.0
a==1.0.0
b==1.0.0; python_version < "3.7"
c==1.0.0; python_version >= "3.7"

More realistically, I would be OK with something like

required-package==1.0.0
a==1.0.0
b==1.0.0; python_version == "3.6"
c==1.0.0; python_version == "3.7" or python_version == "3.8"

or

required-package==1.0.0
a==1.0.0
b==1.0.0; python_version not in "3.7,3.8"
c==1.0.0; python_version in "3.7,3.8"

or in the worst case scenario

required-package==1.0.0; python_version in "3.6,3.7,3.8"
a==1.0.0; python_version in "3.6,3.7,3.8"
b==1.0.0; python_version in "3.6"
c==1.0.0; python_version in "3.7,3.8"

Solution

Maybe, I am missing something and there is an obvious solution to this problem, but I wasn't able to get this kind of result with the current pip-tools functionality. Please, correct me if I'm wrong.

I propose adding a new --add-environment-markers option.
The behaviour of --add-environment-markers is as follows:

1. Remember the old contents of output-file
2. Compile the requirements like you normally would
3. After the compilation is finished, compare the old and the new requirements
   • instead of deleting a requirement from old requirement.txt
     add ; python_version not in "${current_python_version}" to it
   • when adding a new requirement,
     add ; python_version in "${current_python_version}" to it

This option can then be used something like this:

python3.6 -m piptools compile requirements.in --output-file=requirements.txt
python3.7 -m piptools compile requirements.in --output-file=requirements.txt --add-environment-markers
python3.8 -m piptools compile requirements.in --output-file=requirements.txt --add-environment-markers

The first line runs pip-compile in python3.6 and generates (default behaviour)

required-package==1.0.0
a==1.0.0
b==1.0.0

The second line runs pip-compile in python3.7. b==1.0.0 would have been deleted and c==1.0.0 would have been added. Due to the new --add-environment-markers flag, it adds environment markers instead:

required-package==1.0.0
a==1.0.0
b==1.0.0; python_version not in "3.7"
c==1.0.0; python_version in "3.7"

The third line runs pip-compile in python3.8, going through the same motions as the previous command, generates (assuming the in/not in operators are merged)

required-package==1.0.0
a==1.0.0
b==1.0.0; python_version not in "3.7,3.8"
c==1.0.0; python_version in "3.7,3.8"

the required-package is not updated to 2.0.0 (this is the current behaviour anyway).

If you run all 3 commands with --add-environment-markers instead of just the last 2, you will get the following result:

required-package==1.0.0; python_version in "3.6,3.7,3.8"
a==1.0.0; python_version in "3.6,3.7,3.8"
b==1.0.0; python_version in "3.6" and python_version not in "3.7,3.8"
c==1.0.0; python_version in "3.7,3.8"

(ouput-file is empty before the first command, so every requirement is "added")

You could also potentially add parameters to --add-environment-markers, to specify, which environment markers should be added in step 3.
Like --add-environment-markers=python_version and --add-environment-markers=python_version,os_name.