-
Notifications
You must be signed in to change notification settings - Fork 9.8k
textparse: Implement CreatedTimestamp()
in openmetricsparse.go
#14356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
textparse: Implement CreatedTimestamp()
in openmetricsparse.go
#14356
Conversation
Getting a blocker at: https://github.com/prometheus/prometheus/pull/14356/files#diff-0ce7ee179a5b61f32f6202777dc48dd5fdc71bf2b3c4779f78c5e7699384c09eR241 At this line here, even if metric names are theoretically the same, it will still result in an error for test cases like:
While the metric is |
CreatedTimestamp()
in openmetricsparse.go
I know we covered some options of advancing the parser but is this correct? ct := int64(newParser.val)
+_, err := p.Next()
+if err != nil {
+ return nil
+}
return &ct |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like we're pretty close :)
Some test cases for histograms/summaries would be nice as well!
@bwplotka @TheSpiritXIII made some changes, Added some tests and they all pass 😳 |
Also, I feel I need some more test cases and I need to confirm some stuff. Eg:
The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good progress! My comments are mostly about readability/simplicity, we're almost there :)
model/textparse/openmetricsparse.go
Outdated
for { | ||
switch t, _ := newParser.Next(); t { | ||
case EntrySeries: | ||
// TODO: potentially broken? Missing type? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment still relevant?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, might still be relevant
We're not sure if TYPE text is optional yet
if it is how do we handle it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Great work. Tried to help and proposed improvements, feel free to challenge!
model/textparse/openmetricsparse.go
Outdated
// lines ending with _created are skipped by default as they are | ||
// parsed by the CreatedTimestamp method. The skipCT flag is set to | ||
// false when the CreatedTimestamp method is called. | ||
skipCT bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is nice, but it might be too hidden logic from the parser caller perspective. Perhaps less surprising option is to offer a choice on parser constructor for this? Another reason is flexibility, perhaps some parser users would like to ALWAYS skip CT series even without calling CreatedTimestamp? Or opposite, they want to preserve those even when calling CreatedTimestamp.
If it's simple to implement for us, perhaps it makes sense more to use constructor for that decision and make it orthogonal to CreatedTimestamp method use.
NOTE: Likely in Prometheus we will set this to True when CT is enabled, but in future there might option to always set it to true (although one could use relabelling for this)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be addressed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About this, essentially whenever we call an OM parser we want skipCT to always be true. The only case the parser will not skip CT is when it is a deepCopy
In that sense I'm not too sure if it makes sense (I could be wrong) to give to option to skipCT via a constructor. This is moreso just a way to track which parser is a deepCopy and which ins't especially when we're calling parser.Next()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm failing to understand why this isn't needed as well 😬. From our discussions, I understood that we don't want to add _created lines as a separate new metric. This means that the original parser needs to skip _created lines when calling Next()
, but the ephemeral parser in CreatedTimestamp()
needs to return it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update to this can be found in this comment: #14356 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm failing to understand why this isn't needed as well 😬 From our discussions, I understood that we don't want to add _created lines as a separate new metric.
Not sure what @ArthurSens mean by "this" now. I only refer to the logic how this field is set. There is quite large and complex explanation in commentry how this is set. I feel it would be easier if we kill this and simply allow constructor to set it. In fact textparse.WithSkipCT
was already implemented for other reasons, so we have it. My proposal is to:
- keep some default (skip = false?)
- ensure that option as we have now e.g
textparse.WithOMParserCTSeriesSkipped()
- don't change this variable anywhere else (ofc pass true to deep copied for CT logic)
I think this is already implemented by @Maniktherana by a quick look, just I added suggestions below for default and option name 🤗 Thanks!
cdd49c9
to
dae377b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Epic thanks!
Some comments are still not addressed, checked all of them seems those are still relevant. Otherwise LGTM!
model/textparse/openmetricsparse.go
Outdated
// lines ending with _created are skipped by default as they are | ||
// parsed by the CreatedTimestamp method. The skipCT flag is set to | ||
// false when the CreatedTimestamp method is called. | ||
skipCT bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be addressed?
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
- implement changes from pair programming session - use newParse.val() - advance parser p if ct is found Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com>
@bwplotka applied all the suggestions |
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small nit to go, I feel bad for slow review cycles, so happy to help by proposing PR to this PR or something to unblock you. Added suggestion I think we should make, but, as always, feel free to reject (:
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, thanks! Some comments, but good direction!
Signed-off-by: bwplotka <bwplotka@gmail.com>
Pair programming with Manik, Arthur and Daniel.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good, I'm fine with merging as it is but let's keep the momentum and continue the work on small issues in the following weeks :)
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
} | ||
|
||
// TypeRequiresCT returns true if the metric type requires a _created timestamp. | ||
func TypeRequiresCT(t model.MetricType) bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why public?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
its being used in promparse_test
so I thought it would be public in this case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's merge. One nit, but can be fixed later. Thanks!
if contentType == "application/openmetrics-text" { | ||
p = textparse.NewOpenMetricsParser(in, symbolTable) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? This appears to be what the code above already does.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will be removed in my PR here
But this looks like its a relic from an earlier implementation in this PR that I forgot to remove: #14356 (comment)
@bwplotka @ArthurSens @TheSpiritXIII