should we add any other primary keys here?

trace_id cannot be primary key, since we have many spans with the same ID.

In Cassandra we used (trace_id, span_id, span_hash) as PK, since we never actually retrieve by PK, only by partial PK scan like (trace_id, *). Adding a hash allows storing partial spans (even though we don't fully support that).

@yurishkuro Primary keys in ClickHouse work differently and are not required to be unique per row. The primary key instead acts like an index to reduce the amount of data that is scanned (more info at https://clickhouse.com/docs/migrations/postgresql/data-modeling-techniques#how-are-clickhouse-primary-keys-different).

This is the guidance that ClickHouse gives on choosing an effective primary key (ref: https://clickhouse.com/docs/best-practices/choosing-a-primary-key)

When selecting an ordering key, prioritize columns frequently used in query filters (i.e. the WHERE clause), especially those that exclude large numbers of rows.
Columns highly correlated with other data in the table are also beneficial, as contiguous storage improves compression ratios and memory efficiency during GROUP BY and ORDER BY operations.

not following, what does TO services mean? Can't materializes view exist by itself, derived from spans table?

@yurishkuro Materialized views require a target table, which in this case is services. It is derived from the spans table and stored in the services table. See the following from ClickHouse documentation (ref: https://clickhouse.com/docs/best-practices/use-materialized-views)

Incremental materialized views are updated in real-time. As new data is inserted into the source table, ClickHouse automatically applies the materialized view's query to the new data block and writes the results to a separate target table. Over time, ClickHouse merges these partial results to produce a complete, up-to-date view.

I can't say anything about this approach without us doing stress testing, to see how it performs under heavy insert load. We can use OCI hardware for testing.

without attributes, which are the core data element, it's hard to evaluate how good this schema is.

is this the tracestate header? can we add comments? why is it so high at the top?

I was trying to keep the ordering consistent with https://github.com/open-telemetry/opentelemetry-collector/blob/main/pdata/internal/data/protogen/trace/v1/trace.pb.go#L398

should add the link as comment

The check for empty parent span ID currently only verifies string length. Consider also checking if it's a zero span ID (all zeros), as a span with a zero parent ID should be treated as having no parent. This would make the validation more robust against different representations of "no parent".

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.