This issue serves as a public place to track and discuss the progress of remote write v1.1 2.0, which is described in this spec document, and was first worked on here.

Still in progress via #13052 and a number of other PRs.

UPDATE Dec 12, 2023: we're changing this to be 2.0, we don't intend for the protobuf format in the new version to be backwards compatible

Issues (as up-to-date as possible)

In-progress

TODO

• gather results for compression algorithms and maybe allow for their usage remote write 2.0 - implement additional compressions #13366
• write a blog post Remote Write 2.0: Write a blog post #14254
• add official benchmarks Remote Write 2.0: Benchmarks vs 1.0 and OTLP #14253
• official benchmarking framework remote write 2.0 - benchmarking framework; integrated with with prombench #13995
• ensure tests are up to date remote write 2.0 - ensure unit tests are all properly updated #13479
• ensure we can still build write requests of the proper batch size remote write 2.0 - ensure the metadata is not counted against max samples in a write request #13480
• reduce code duplication introduced via usage of new types remote write 2.0 - refactor append functions, build write request, etc. to reduce code duplication #13481
• ensure metadata cache is cleaned up remote write 2.0 - ensure metadata cache in queue manager is cleaned up #13482
• make sure we're writing metadata to the WAL in agent mode remote write 2.0 - test for agent mode with metadata-wal-records and type-and-unit features #13483
• consider 2.1 structured Receiver response for number of rejected samples/exemplars remote_write client metrics: a partially failed batch is counted as all samples failed #14323
• remote write 2.0 - update test for old samples filtering for 2.0 remote write 2.0 - update test for old samples filtering for 2.0 #14415
• remote write 2.0 - update TestSampleDelivery to check for metadata in 2.0 proto remote write 2.0 - update TestSampleDelivery to check for metadata in 2.0 proto #14414
• remote write 2.0 - DRY the queue manager code remote write 2.0 - DRY the queue manager code #14409
• remote write 2.0 - update write handler benchmarks for 2.0 format remote write 2.0 - update write handler benchmarks for 2.0 format #14408
• remote write 2.0 - decide whether addition of metadata should count towards max samples in write request remote write 2.0 - decide whether addition of metadata should count towards max samples in write request #14407

Complete

• compare original v1.1 format to proposed minimized format
• update spec to describe minimized format if we decide to use it over original reduced format
• content negotiation Remote write 2.0 - content negotiation  #13365
• finish off metadata changes from Paschalis remote write 2.0 - metadata #13367
• implement string interning of exemplars labels (completed in remote write 2.0: new proto format with string interning  #13052)
• support created timestamp remote write 2.0: Add support for created timestamp propagation [spec] #13368
• research histogram atomicity for remote write remote write 2.0 - explore histogram atomicity #13410
• custom histogram layouts support prw2.0: Added support for "custom" layouts for native histogram proto #13558
• formal proposal remote write 2.0 - formal proposal #13887