We are currently using a gauge incorrectly to report hedged roundtrips on both external endpoints and backend requests. This has worked in the past but recently we have seen this report erroneous results when rate()ing in prometheus. It's possible it never worked correctly and we have only recently noticed. Regardless, it's bad practice and we should fix it.

External Hedged Requests Gauge

Backend Hedged Requests Gauge

Shared code in use for both Gauges

Let's replace the two linked gauges above with counters.