query-tee: optionally consider equivalent errors the same when comparing responses #9143
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jhesketh
reviewed
Aug 30, 2024
| // Prometheus' engine: | ||
| regexp.MustCompile(`invalid parameter "query": invalid expression type "range vector" for range query, must be Scalar or instant Vector`), | ||
| // MQE: | ||
| regexp.MustCompile(`invalid parameter "query": query expression produces a range vector, but expression for range queries must produce an instant vector or scalar`), |
There was a problem hiding this comment.
Should MQE be changed to be the same for this one? (and later promql can be fixed to be more descriptive)
There was a problem hiding this comment.
We've intentionally decided to provide clearer error messages where we can in MQE, even if that means diverging from Prometheus' engine.
There was a problem hiding this comment.
Yep, but I'm wondering if we should be the same to ease testing, and open PRs upstream to fix up wording. Nonetheless, not a blocker here :-)
jhesketh
reviewed
Sep 3, 2024
Done in 56f9674. |
jhesketh
approved these changes
Sep 5, 2024
This was referenced Sep 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does
This PR extends query-tee's response comparison to accept equivalent but non-identical error messages as the same.
For example, Prometheus' engine and MQE return slightly different error messages in some circumstances, and Prometheus' engine is non-deterministic when selecting example series in some error messages. Both of these situations can cause query-tee to consider a response as different, but we usually don't care as long as the errors are equivalent.
This new behaviour is enabled by default but can be disabled with
-proxy.require-exact-error-match=true.Which issue(s) this PR fixes or relates to
(none)
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX].about-versioning.mdupdated with experimental features.