Skip to content

Native histograms: refactor chunk/sample handling for generic copy on write #16581

@krajorama

Description

@krajorama

From the Native histograms spec:

As usual, PromQL implementations MAY apply any optimizations they see fit as long as the behavior stays the same. Decoding native histograms can be quite expensive with the potentially many buckets. Similarly, deep-copying a histogram sample within the PromQL engine is much more expensive than copying a simple float sample. This creates a huge potential for optimization compared to a naive approach of always decoding everything and always copying everything.

Prometheus currently tries to avoid needless copies (TODO: but a proper CoW like approach still has to be implemented, as it would be much cleaner and less bug prone) and skips decoding of the buckets for special cases where only the sum and count of observations is required.

Implement a generic high performance framework for handling copy-on-write and pooling of native histogram samples.

Related PR: #13679 where I tried to handle a sub-case, but it turned out to be way too complicated.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions