Allowing the user to install color support through an install extra (e.g. pip install pytest[color]) would be nicer than requiring that a random package be placed in the long list of requirements, I think.

It should be quite straightforward to implement this (see https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies).