Skip to content

Conversation

Maniktherana
Copy link
Contributor

@Maniktherana Maniktherana commented Jun 27, 2024

@Maniktherana
Copy link
Contributor Author

Getting a blocker at: https://github.com/prometheus/prometheus/pull/14356/files#diff-0ce7ee179a5b61f32f6202777dc48dd5fdc71bf2b3c4779f78c5e7699384c09eR241

At this line here, even if metric names are theoretically the same, it will still result in an error for test cases like:

# TYPE foo counter
foo_total 123
foo_created 1000

While the metric is foo in both cases how do we confidently get the metric name without the suffix?
it can get even more weird for cases like foo_created{some_label="some_label_value"}

@Maniktherana Maniktherana changed the title feat: initial implement of createedTimestamp() with tests Implement CreatedTimestamp() in openmetricsparse.go Jun 27, 2024
@Maniktherana
Copy link
Contributor Author

Maniktherana commented Jul 4, 2024

I know we covered some options of advancing the parser but is this correct?
I'm able to pass all tests currently (going to add some more soon)

ct := int64(newParser.val)
+_, err := p.Next()
+if err != nil {
+    return nil
+}
return &ct

Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we're pretty close :)

Some test cases for histograms/summaries would be nice as well!

@Maniktherana
Copy link
Contributor Author

@bwplotka @TheSpiritXIII made some changes, Added some tests and they all pass 😳

@Maniktherana
Copy link
Contributor Author

Also, I feel I need some more test cases and I need to confirm some stuff.

Eg:

# TYPE bar summary
bar_count 17.0
bar_sum 324789.3
bar{quantile="0.95"} 123.7
bar{quantile="0.99"} 150.0
bar_created 1520430000

The CreatedTimestamp function only works if the _created line is at the very end of a summary and histogram. Is this how it works or can _created be anywhere within a labelset?

Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good progress! My comments are mostly about readability/simplicity, we're almost there :)

for {
switch t, _ := newParser.Next(); t {
case EntrySeries:
// TODO: potentially broken? Missing type?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment still relevant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, might still be relevant
We're not sure if TYPE text is optional yet
if it is how do we handle it

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Great work. Tried to help and proposed improvements, feel free to challenge!

// lines ending with _created are skipped by default as they are
// parsed by the CreatedTimestamp method. The skipCT flag is set to
// false when the CreatedTimestamp method is called.
skipCT bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice, but it might be too hidden logic from the parser caller perspective. Perhaps less surprising option is to offer a choice on parser constructor for this? Another reason is flexibility, perhaps some parser users would like to ALWAYS skip CT series even without calling CreatedTimestamp? Or opposite, they want to preserve those even when calling CreatedTimestamp.

If it's simple to implement for us, perhaps it makes sense more to use constructor for that decision and make it orthogonal to CreatedTimestamp method use.

NOTE: Likely in Prometheus we will set this to True when CT is enabled, but in future there might option to always set it to true (although one could use relabelling for this)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be addressed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About this, essentially whenever we call an OM parser we want skipCT to always be true. The only case the parser will not skip CT is when it is a deepCopy

In that sense I'm not too sure if it makes sense (I could be wrong) to give to option to skipCT via a constructor. This is moreso just a way to track which parser is a deepCopy and which ins't especially when we're calling parser.Next()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm failing to understand why this isn't needed as well 😬. From our discussions, I understood that we don't want to add _created lines as a separate new metric. This means that the original parser needs to skip _created lines when calling Next(), but the ephemeral parser in CreatedTimestamp() needs to return it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update to this can be found in this comment: #14356 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm failing to understand why this isn't needed as well 😬 From our discussions, I understood that we don't want to add _created lines as a separate new metric.

Not sure what @ArthurSens mean by "this" now. I only refer to the logic how this field is set. There is quite large and complex explanation in commentry how this is set. I feel it would be easier if we kill this and simply allow constructor to set it. In fact textparse.WithSkipCT was already implemented for other reasons, so we have it. My proposal is to:

  • keep some default (skip = false?)
  • ensure that option as we have now e.g textparse.WithOMParserCTSeriesSkipped()
  • don't change this variable anywhere else (ofc pass true to deep copied for CT logic)

I think this is already implemented by @Maniktherana by a quick look, just I added suggestions below for default and option name 🤗 Thanks!

@Maniktherana Maniktherana marked this pull request as ready for review July 10, 2024 14:02
@Maniktherana Maniktherana force-pushed the createdTimestamp-parser branch from cdd49c9 to dae377b Compare July 10, 2024 16:10
bwplotka
bwplotka previously approved these changes Jul 10, 2024
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epic thanks!

Some comments are still not addressed, checked all of them seems those are still relevant. Otherwise LGTM!

// lines ending with _created are skipped by default as they are
// parsed by the CreatedTimestamp method. The skipCT flag is set to
// false when the CreatedTimestamp method is called.
skipCT bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be addressed?

Maniktherana and others added 15 commits July 12, 2024 00:48
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
@Maniktherana
Copy link
Contributor Author

@bwplotka applied all the suggestions

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit to go, I feel bad for slow review cycles, so happy to help by proposing PR to this PR or something to unblock you. Added suggestion I think we should make, but, as always, feel free to reject (:

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! Some comments, but good direction!

Signed-off-by: bwplotka <bwplotka@gmail.com>
@bwplotka
Copy link
Member

bwplotka commented Aug 7, 2024

Maniktherana#2

Pair programming with Manik, Arthur and Daniel.
ArthurSens
ArthurSens previously approved these changes Aug 7, 2024
Copy link
Member

@ArthurSens ArthurSens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I'm fine with merging as it is but let's keep the momentum and continue the work on small issues in the following weeks :)

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
}

// TypeRequiresCT returns true if the metric type requires a _created timestamp.
func TypeRequiresCT(t model.MetricType) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why public?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its being used in promparse_test so I thought it would be public in this case

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge. One nit, but can be fixed later. Thanks!

Comment on lines +71 to +73
if contentType == "application/openmetrics-text" {
p = textparse.NewOpenMetricsParser(in, symbolTable)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? This appears to be what the code above already does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be removed in my PR here

But this looks like its a relic from an earlier implementation in this PR that I forgot to remove: #14356 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants