Cause and effect:
During the investigation of grafana/tempo#2743 I was able to track down the problem to specific commits in mino-go #1626 and #1682.
Those two together causes us not being able to authenticate, and subsequent Tempo error.

If that is expected behavior (as it seems according to this Cortex commit cortexproject/cortex#4897) the issue lies in the usage of the library.