Proposal

Probably very surprising for most, there is no requirement that the storage returns matching time series for a given selector. This apparently unlocks certain more or less valid use cases, mostly around remote-read. There is a lengthy discussion in #8053. However, this behavior also prevents a lot of optimizations, and there are implications for the planned migration to UTF-8 capable names. (See the relevant design doc for starters, but there are implications about rewriting the results that aren't yet reflected in the design doc.)

My proposal here is to revisit the assumption that there are valid and useful use cases for selectors returning a result that doesn't match the selector. Almost four years later, we should have a pretty good picture if that is still and actually the case. The Prometheus 3 release is a good time to act on the result:

• If those use cases have turned out to not be very useful after all, we can just mandate that selectors only return matching time series and unlock optimizations etc.
• If those use cases have some use, but are only relevant for a very small fraction of users, we might be able to contain them behind flags or something so that we can still optimize things for the majority of users.
• "Worst case" would be that we just have to live with the possibility that selectors return anything.

I will also submit this to the dev summit because I think it warrants a broader discussion.

Paging @ywwg @GiedriusS @fpetkovski @charleskorn @bwplotka as this might be relevant for you.