Right now several PRs that are trying to add support for standard storage backends to v2 are reusing the same configuration for storage as we had in v1. Now is a good time to rethink that configuration, in particular:

• better align with standard components of OTEL Collector. For example, the gRPC storage ([jaeger-v2] Add support for GRPC storarge #5228) is conceptually very similar to OTLP Exporter - they both send data over gRPC to a remote service. Our configuration is pretty slim, while OTLP Exporter's configuration has a lot more features, there is no reason why we cannot support them (would be even better if we could simply reuse OTEL's configuration utilities for gRPC)
• our storage configs have been accumulating over time, we can see if there are better logical groupings. For example, ES config is a flat list of everything, we can group many flags into sub-types so that the YAML config looks a lot more structured than using long field names like priority_dependencies_template
• ensure that we have sensible defaults. For example, in [jaeger-v2] add cassandra e2e integration tests #5398 basic options had to be added to Cassandra configuration, even though in v1 many of them are already defaulted to these values, so we're clearly missing defaulting mechanism. It's somewhat complicated by the fact that the storage extension cannot actually define the defaults in CreateDefaultConfig because at that point we don't known which storage will be used.

Scope

• Cassandra storage config -- [jaeger-v2] Align Cassandra Storage Config With OTEL #5928
• Elasticsearch / OpenSearch storage config -- [jaeger-v2] Refactor ElasticSearch/OpenSearch Storage Configurations To Align With OTEL #6059
• Badger storage config -- [jaeger-v2] Align Badger Storage Config With OTEL #5926
• Memory storage config -- [jaeger-v2] Align Memory Storage Config With OTEL #5923
• Kafka config -- [jaeger-v2]: Fix Kafka E2E Test To Use Default Environment Variables #6027
• grpc storage (e.g. issue [grpc storage]: Propagate tenant to grpc backend #6030) -- [jaeger-v2] Consolidate v1 and v2 Configs For GRPC Storage #6041
• Adaptive sampling processor / extension config -- [jaeger-v2] Refactor Adaptive Sampling Processor Configurations #6021
• Query service config -- [jaeger-v2] Align Query Service Config With OTEL #5996
• Collector-specific config -- [jaeger-v2] Map Collector Specific Flags To Corresponding OTEL Configurations #6040
• Utilize env var defaults available since OTEL v0.110 e.g.

  jaeger/cmd/jaeger/internal/integration/kafka_test.go

  Line 24 in c655e56

  // we can change the config to use topic: "${KAFKA_TOPIC:-jaeger-spans}"

Tasks / outcomes

For each of the above components, we need to ensure

• All config fields are tagged with mapstructure such that they can be addressable from YAML via "good" names
• Configs implement Validate() -- example
• Configs via YAML support the same defaults as in jaeger-v1
• Configs reuse standard elements of OTEL configs whenever possible, e.g. TLS, clientcfg, etc.
• Configuration looks "logical" and properly grouped, not flattened with long weird names (good example [Jaeger-V2] Add configurable index data layout and rollover frequency #5790)
• We create a migration guide document with a table that lists v1 CLI flags and the corresponding v2 config fields (example for Kafka)

How to work on this meta-issue

• pick a component / area
• create a dedicated ticket for it
• copy tasks / outcomes from above
• use the ticket for design discussions, updates, links to PRs