Skip to content

Race condition with Exemplar in Counter #1145

@fstab

Description

@fstab

I found an inconsistent concurrency behavior in client_java, and I think that client_golang has the same issue. Here's the current implementation of the counter's Write() function:

func (c *counter) Write(out *dto.Metric) error {
	val := c.get()

	var exemplar *dto.Exemplar
	if e := c.exemplar.Load(); e != nil {
		exemplar = e.(*dto.Exemplar)
	}

	return populateMetric(CounterValue, val, c.labelPairs, exemplar, out)
}

Counter implements selfCollector. If I understand correctly, this means that the counter continues to be updated while Write() is being called.

So if an update happens after reading val but before reading exemplar, you might see an examplar for an observation that's not represented in val.

I guess to fix this it would be sufficient to read exemplar first and then val, because then the worst case you get an exemplar for a previous observation, but you won't get an exemplar for a not-yet-existing observation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions