-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Prevent large ephemeral event traffic to appservices upon startup #10836
Description
@Half-Shot mentioned that upon start up, Synapse will send all missed ephemeral events to application services that have opted into receiving them (see MSC2409). There are some performance issues present with the feature's implementation in a couple of different circumstances.
Some background: a "stream token" is an incrementing integer. Every occurrence (a new typing event, a new read receipt, etc.) gets one of these IDs, and they're (usually rigidly) ordered in this way. Thus we can simply keep track of the last successfully sent stream token per app service to know what the AS needs to still receive.
- When the bridge is first added to the homeserver: it is assumed to have a last-successful stream token of 0:
synapse/synapse/storage/databases/main/appservice.py
Lines 387 to 412 in bdfde6d
async def get_type_stream_id_for_appservice( self, service: ApplicationService, type: str ) -> int: if type not in ("read_receipt", "presence"): raise ValueError( "Expected type to be a valid application stream id type, got %s" % (type,) ) def get_type_stream_id_for_appservice_txn(txn): stream_id_type = "%s_stream_id" % type txn.execute( # We do NOT want to escape `stream_id_type`. "SELECT %s FROM application_services_state WHERE as_id=?" % stream_id_type, (service.id,), ) last_stream_id = txn.fetchone() if last_stream_id is None or last_stream_id[0] is None: # no row exists return 0 else: return int(last_stream_id[0]) return await self.db_pool.runInteraction( "get_type_stream_id_for_appservice", get_type_stream_id_for_appservice_txn )
When it comes time to send out the ephemeral events the appservice has missed, Synapse will send all read receipts and presence updates that it knows about and that are relevant to the namespaces the appservice has registered:
synapse/synapse/handlers/appservice.py
Lines 266 to 273 in a8a27b2
from_key = await self.store.get_type_stream_id_for_appservice( | |
service, "read_receipt" | |
) | |
receipts_source = self.event_sources.sources["receipt"] | |
receipts, _ = await receipts_source.get_new_events_as( | |
service=service, from_key=from_key | |
) | |
return receipts |
This historical data isn't very useful to the appservice as it has just been registered.
- When a bridge has been offline for a while and comes back: it will receive all missed, relevant EDUs since it went offline. In particular, historical EDUs are typically less useful than the current state of things.
I'd be tempted to implement a time bound to the EDUs we send to appservices, such that very old EDUs are no longer considered.
Fixing both of these would help reduce the large amount of EDUs that can be sent to and overload both the appservice and Synapse itself when it tries to pull all of that information out of the database.
Finally, note that the above problems only seem to apply to the implementation of read receipts and presence events. Historical typing events are flat out not sent to appservices:
synapse/synapse/handlers/appservice.py
Line 235 in a8a27b2
# We don't persist the token for typing_key for performance reasons |