Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Error doing background update 'chain_cover' on v1.26.0rc1 #9208

@intelfx

Description

@intelfx

Description

After upgrading v1.25.0 synapse installation to v1.26.0rc1 and restarting the instance one more time, it started to continuously report this error:

янв 22 19:05:51 stratofortress.nexus.i.intelfx.name synapse[373164]: synapse.storage.background_updates: [background_updates-0] Starting update batch on background update 'chain_cover'
янв 22 19:05:51 stratofortress.nexus.i.intelfx.name synapse[373164]: synapse.storage.background_updates: [background_updates-0] Error doing update
                                                                        Traceback (most recent call last):
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/background_updates.py", line 116, in run_background_updates
                                                                            result = await self.do_next_background_update(
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/background_updates.py", line 227, in do_next_background_update
                                                                            await self._do_background_update(desired_duration_ms)
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/background_updates.py", line 264, in _do_background_update
                                                                            items_updated = await update_handler(progress, batch_size)
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 748, in _chain_cover_index
                                                                            result = await self.db_pool.runInteraction(
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/database.py", line 656, in runInteraction
                                                                            result = await self.runWithConnection(
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/database.py", line 739, in runWithConnection
                                                                            return await make_deferred_yieldable(
                                                                          File "/usr/lib/python3.9/site-packages/twisted/python/threadpool.py", line 250, in inContext
                                                                            result = inContext.theWork()
                                                                          File "/usr/lib/python3.9/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
                                                                            inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
                                                                          File "/usr/lib/python3.9/site-packages/twisted/python/context.py", line 122, in callWithContext
                                                                            return self.currentContext().callWithContext(ctx, func, *args, **kw)
                                                                          File "/usr/lib/python3.9/site-packages/twisted/python/context.py", line 85, in callWithContext
                                                                            return func(*args,**kw)
                                                                          File "/usr/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
                                                                            compat.reraise(excValue, excTraceback)
                                                                          File "/usr/lib/python3.9/site-packages/twisted/python/compat.py", line 464, in reraise
                                                                            raise exception.with_traceback(traceback)
                                                                          File "/usr/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
                                                                            result = func(conn, *args, **kw)
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/database.py", line 734, in inner_func
                                                                            return func(db_conn, *args, **kwargs)
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/database.py", line 534, in new_transaction
                                                                            r = func(cursor, *args, **kwargs)
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 920, in _calculate_chain_cover_txn
                                                                            PersistEventsStore._add_chain_cover_index(
                                                                          File "/usr/lib/python3.9/site-packages/synapse/storage/databases/main/events.py", line 634, in _add_chain_cover_index
                                                                            existing_chain_id = chain_map[auth_id]
                                                                        KeyError: '$2PBTqUSs6gQtrZ3jZW8xVUSHvDbUYR3TKpFBoDHHwJk'

Steps to reproduce

  • upgrade v1.25.0 to v1.26.0rc1
  • restart synapse, wait until the migration and background updates are completed
  • restart synapse one more time

Version information

  • Homeserver: intelfx.name

If not matrix.org:

  • Version: {"server_version":"1.26.0rc1","python_version":"3.9.1"}

  • Install method: hand packaged

  • Platform: x86_64 Arch Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    S-MinorBlocks non-critical functionality, workarounds exist.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions