As a quick fix to #4001 / prometheus/client_python#317, we pinned our prometheus client to < 0.4.0.

This introduces problems:

• Sooner or later we're going to want to pick up some bugfix or feature from more recent prometheus_clients.
• Arch Linux ships with prometheus_client 0.4, so people on Arch have trouble building synapse (test_metrics fails on both python 2.7 and 3.7 #4167). This problem is likely to affect more platforms in the future.
• It's actually possible for people to suffer prometheus_client 0.4.0 breaks lots of our metrics #4001 in reverse:
  • They start with synapse 0.33.5 or earlier, which doesn't include the pin, so they get prometheus-client 0.4.0, with the *_total metrics.
  • Then they upgrade synapse, which includes the pin, so prometheus-client gets downgraded.

I'm still surprised at the prometheus_client devs making the choice to rename these metrics, but I think we're going to have to live with it, so could do with a migration plan.

When we have renamed metrics in the past, we have supported both old and new names for a few releases. That gives people a chance to update their dashboards and alerts. I believe we should do something similar here.

It may turn out that supporting both names is more easily done with prometheus_client 0.3 than it is with 0.4, so we should do this work before we are forced to update to 0.4 in a hurry which would break everyone's metrics.