Filling issue based on triage discussion today, cc @gammazero if below prior art context is useful

Currently, we have no "requests in flight" cap in boxo/gateway and require reverse proxy in front of go binary to enforce the limit.

Proposed improvement

There should be a built-in mechanism to limit the number of ongoing HTTP gateway requests in flight, and return HTTP 503 Service Unavailable with Retry-After when limit is reached.

Make it a config option, boxo/gateway should ship with some implicit default, protecting people who expose it blindly to the internet

Prior art

• At public gateway,
  • the current ceiling exist at nginx and for our rainbow prod seems to be really high: worker_connections is set to 10240 (per each instance). For reference, nginx default is 512, raising to 1024 seems to be a sensible place to start as default, protecting from traffic spikes.
  • there is a timeout of 30 seconds (nginx's proxy_read_timeout) – we dont have built-in config yet, but its tracked in gateway: ability to set response write timeout #679