On March 31st (roughly) there was an uptick of "message stuck as latest message" reports to the riot clients (riot-web's upstream issue is element-hq/element-web#10032 for this and other cases).

The cause looks to be because Synapse is not sending events down /sync sometimes for events we've sent. There are also some reports of events just not being received on the other end which could be related. Most reporters were on matrix.org, and the reports seemed to stop once matrix.org rolled back.

Reproduction steps appear to be time-dependent, and possibly only useful for matrix.org-sized homeservers:

1. Send an event
2. Start a /sync request
3. Send another event
4. Receive event ID for first event (/send/m.room.message completes)
5. Receive event down /sync
6. Send yet another event
7. Start a /sync request
8. Receive event ID for second and third events
9. Receive the third event down /sync, but not the second (sometimes the other way around)
10. Never receive the second event through /sync, making it stuck on Riot

Most easily reproduced during high traffic volumes on matrix.org, though one case was also reproduced at ~22:00 UTC yesterday (but never again that night).

Theory at least from my side is that the request hits a busy synchrotron which advances the stream token past the second event to the third event, invoking amnesia.

@erikjohnston's conclusions appear to be that synapse might be sending the event down /sync ahead of the /send/m.room.message request completing?