Note: This issue doesn't yet prescribe how to make this solution happen - creating it as a placeholder to link to from other issues/RFCs and for tracking backlog. Please ignore this issue for now.

Is your feature request related to a problem?

Currently, you can only scale self-hosted clickhouse instances vertically. On cloud we've mitigated this by setting up sharding.

Constraints

• Handle CLICKHOUSE_REPLICATION environment variable correctly when creating the schema
• Support sharding in helm chart configuration
• Figure out a resharding solution - not trivial due to clickhouse not providing great tooling for it.

Note that even production clickhouse setup has flaws:

• Sharding key is not used during ingestion
• There are tables that have incorrect schemas: Cloud clickhouse schema is incorrectly sharded for events_dead_letter_queue and plugin_log_entries #7571

Additional context

Solving this might depend on:

• Store person_id and person_properties on events #8132
• Async migrations being production-ready

Related from clickhouse side: ClickHouse/ClickHouse#20219

Related: PostHog/product-internal#240

Thank you for your feature request – we love each and every one!