Skip to content

nhcb: implement chunk encoding of custom buckets #13487

@krajorama

Description

@krajorama

Proposal

Depends on #13486 .

Histogram and FloatHistogram chunk encodings first store the layout see func writeHistogramChunkLayout(b *bstream, schema int32, zeroThreshold float64, positiveSpans, negativeSpans []histogram.Span). And they store the bucket counters after the layout.

In case the schema is equal to CustomBucketsSchema defined in #13486 , store the size of customBounds as varint then store the customBounds data after the chunk layout using the usual xorWrite/xorRead methods. No need to store +Inf in the raw data, simply restore upon read as implicit.

The Appender implementation should remember the interned bucket definitions. Custom bucket histogram is only appendable if the schema is equal to CustomBucketsSchema and the customBounds are equal. Otherwise return new chunk. Never return recoded, at least not in the first iteration.

The Appender should set unknown reset hint if schema changes and a new chunk is started due to schema change or a change in custom buckets.

Update the units tests.

Note: there might be a way to optimize the storage of customBounds later: if customBounds can be scaled to be whole numbers , we might convert to int64 and store a more efficient delta encoding. Not sure if it's worth making the encoding future proof for this possibility? We could probably encode the information by setting the store size of customBounds huge/negative. Probably a good idea to mask out the top 32 bits of the length and reserve for later use.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions