-
Notifications
You must be signed in to change notification settings - Fork 9.8k
Description
Proposal
#10421 added a new method ShardedPostings
in IndexReader
to filter a specific shard using provided shardIndex
and shardCount
parameter.
The implementation in #10421 uses hashmod to check if a series belong to the given shard. However, there are different ways to do that and hashmod is only one way. It would be good to use a configurable function to customize the sharding logic. This might not be used in Prometheus, but it would be useful for downstream projects.
There are two main implementations of ShardedPostings
method:
index.Reader
https://github.com/prometheus/prometheus/blob/main/tsdb/index/index.go#L1774headIndexReader
https://github.com/prometheus/prometheus/blob/main/tsdb/head_read.go
I am still unsure where I can add the logic to allow passing a custom function from downstream projects, especially for index.Reader
. Maybe I have to change ShardedPostings
method signature to
ShardedPostings(p index.Postings, shardIndex, shardCount uint64, shardFunc func(hash, shardIndex, shardCount uint64) bool) index.Postings
Any suggestion would be appreciated!