Details:
Currently, Contour attaches a compression filter to envoy httpConnectionManager by default, as a result compression is enabled everywhere. While this feature is beneficial in certain scenarios, we have observed that the associated CPU cost can be prohibitively high compared to network costs. Specifically, we've found that CPU is more expensive than network bandwidth in our use case.

Rationale:
Compression is particularly useful for last-mile traffic. However, when CDNs front the ingress layer, which has dedicated bandwidth to origin servers, the network impact is negligible. In such cases, the return on investment for CPU cycles used in compression is significantly reduced. Therefore, the cost savings of disabling compression outweigh the benefits when not dealing with last-mile traffic or ingress layers meant for internal traffic.

Proposal:
We propose adding a control-plane configuration option, such as --disable-compression=true, to allow users to skip compression from the default filters. This change would provide significant cost savings in scenarios where compression is not beneficial.

Reduces CPU usage and allows for more efficient use of resources by running lean Envoy instances. We believe it will add significant value for users who, like us, need to optimize for cost and resource efficiency.

Observations from Our Cluster Testing:

• Enabling compression increased CPU utilization by nearly three times.
  • Without compression, the same workload utilized approximately 0.6 CPU cores.
  • With compression enabled, CPU utilization spiked to 2 cores.
  • The payload per request during our test was 310KB (uncompressed) and 85KB (compressed).
    Given these results, we would prefer to run lean instances of Envoy and scale out as needed.