feat: retrieval-timeout & max-concurrent-requests #285
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
Include fix from ipfs/boxo@e109608
|
I'll be fixing conformance failures in ipfs/boxo#994 – they are not related to feature added here, but existing bugs that now are surfaced by time-bound execution.
|
applies fix from ipfs/boxo@a51f1fc
4 tasks
gammazero
approved these changes
Aug 13, 2025
hsanjuan
approved these changes
Aug 13, 2025
docs/environment-variables.md
Outdated
|
|
||
| Maximum number of concurrent HTTP requests that the gateway will process. | ||
|
|
||
| This setting provides rate limiting to protect the gateway from resource exhaustion during high load scenarios. When the limit is reached, new requests will receive a `429 Too Many Requests` response with a `Retry-After` header indicating when the client should retry. |
There was a problem hiding this comment.
And when should that retry be? Is it hardcoded or dynamic or...?
There was a problem hiding this comment.
@hsanjuan for now Retry-After header is hardcoded to 60 seconds. Pushed 789b053 to document that.
Making this dynamic would make ipfs/boxo#994 even more complex and wanted to avoid that for now.
We can make it configurable in the future PRs but my experience has been clients either detect header and have own smart backoff (using 60s value as initial hint only), or just ignore the header so for MVP static 1m should be ok.
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
This was referenced Aug 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds built-in limiters for IPFS Gateway requests:
retrieval-timeout(default: 30s) andmax-concurrent-requests(default: 1024).Gateway.RetrievalTimeout|MaxConcurrentRequestskubo#10905On
ipfs.iocc @gammazero @hsanjuan @ns4plabs for visibility as this PR moves some of Nginx responsibilities to Rainbow.
High load infrastructure like
ipfs.iowill need to ensure Nginx limits are bit higher than Rainbow, namely:proxy_read_timeout(nginx) >RAINBOW_RETRIEVAL_TIMEOUThere (right now both 30s, we need to set proxy +5s)worker_connections(nginx) >RAINBOW_MAX_CONCURRENT_REQUESTS(1024 needs to be raised, fysa ipfs.io uses value 10x bigger)We can disable limits by setting both env variables to
0