Description

When trying to join a large room, like #_oftc_#oftc:matrix.org or #freenode_#gentoo:matrix.org, the response for send_join can become very large. Responses in synapse are usually limited to 100MB (https://github.com/matrix-org/synapse/blob/develop/synapse/http/matrixfederationclient.py#L91). This makes the join for rooms like oftc fail, since the room state there exceed 100MB. My patched synapse, where I increased the limit, logs the following for example:

2021-05-26 13:31:54,183 - synapse.http.matrixfederationclient - 276 - INFO - POST-62806 - {PUT-O-29440} [matrix.org] Completed request: 200 OK in 61.61 secs, got 106452938 bytes - PUT matrix://matrix.org/_matrix/federation/v2/send_join/%21ZjZUpZpLKxVfAEWyjp%3Amatrix.org/%2416220358520AGPsK%3Aneko.dev

Sometimes you don't even get that far, as the remote server can timeout while sending the response, probably because the state is too large.

(Not that this is partially because send_join returns the full room state without pagination.)

Steps to reproduce

• try to join #_oftc_#oftc:matrix.org (on a beefy homeserver, if you can)
• Watch it fail because of response size limits

Version information

• Homeserver: neko.dev

If not matrix.org:

• Version: 1.35_rc1 (and a few others)

• Install method: custom ebuild

• Platform: Gentoo/Linux, pretty beefy hardware (64GB RAM, 16 cores, Ryzen 3000, NVMe RAID)