-
Notifications
You must be signed in to change notification settings - Fork 9.8k
Description
Proposal
Depends on #13486 .
Histogram and FloatHistogram chunk encodings first store the layout see func writeHistogramChunkLayout(b *bstream, schema int32, zeroThreshold float64, positiveSpans, negativeSpans []histogram.Span)
. And they store the bucket counters after the layout.
In case the schema is equal to CustomBucketsSchema
defined in #13486 , store the size of customBounds
as varint then store the customBounds
data after the chunk layout using the usual xorWrite/xorRead methods. No need to store +Inf in the raw data, simply restore upon read as implicit.
The Appender implementation should remember the interned bucket definitions. Custom bucket histogram is only appendable if the schema is equal to CustomBucketsSchema and the customBounds are equal. Otherwise return new chunk. Never return recoded
, at least not in the first iteration.
The Appender should set unknown reset hint if schema changes and a new chunk is started due to schema change or a change in custom buckets.
Update the units tests.
Note: there might be a way to optimize the storage of customBounds later: if customBounds can be scaled to be whole numbers , we might convert to int64 and store a more efficient delta encoding. Not sure if it's worth making the encoding future proof for this possibility? We could probably encode the information by setting the store size of customBounds huge/negative. Probably a good idea to mask out the top 32 bits of the length and reserve for later use.