This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
Consecutive remote media thumbnail requests with different methods fails #8649
Description
Description
It seems like the method column is missing on the unique constraint on remote_media_cache_thumbnails, causing internal server errors when trying to fetch the same size of the same thumbnail with different methods.
Steps to reproduce
-
Fetch a thumbnail of a remote image using some specific size and method (scale or crop).
-
Fetch a thumbnail of the same image and same size, but specify a different method.
-
Receive M_UNKNOWN, Internal server error
Here's the error from server logs:
2020-10-22 16:04:50,858 - synapse.http.server - 85 - ERROR - GET-1786524 - Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7fc446756320 method='GET'
uri='/_matrix/media/r0/thumbnail/remote-server.tld/XXXXXXX_removed_file_id?width=44&height=44&method=crop' clientproto='HTTP/1.0' site=8008>
Traceback (most recent call last):
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 230, in _async_render_wrapper
callback_return = await self._async_render(request)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 258, in _async_render
callback_return = await raw_callback_return
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 67, in _async_render_GET
request, server_name, media_id, width, height, method, m_type
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 232, in _select_or_generate_remote_thumbnail
desired_type,
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/rest/media/v1/media_repository.py", line 597, in generate_remote_exact_thumbnail
t_len,
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/databases/main/media_repository.py", line 374, in store_remote_media_thumbnail
desc="store_remote_media_thumbnail",
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 727, in simple_insert
await self.runInteraction(desc, self.simple_insert_txn, table, values)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 577, in runInteraction
**kwargs
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 647, in runWithConnection
self._db_pool.runWithConnection(inner_func, *args, **kwargs)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
compat.reraise(excValue, excTraceback)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/python/compat.py", line 464, in reraise
raise exception.with_traceback(traceback)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
result = func(conn, *args, **kw)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 641, in inner_func
return func(conn, *args, **kwargs)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 447, in new_transaction
r = func(cursor, *args, **kwargs)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 748, in simple_insert_txn
txn.execute(sql, vals)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 212, in execute
self._do_execute(self.txn.execute, sql, *args)
File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/database.py", line 238, in _do_execute
return func(sql, *args)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL: Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(remote-server.tld, XXXXXXX_removed_file_id, 44, 44, image/png) already e
xists.
Added comments from tulir:
"someone forgot the thumbnail_method from the unique constraint"
"wait no, there are two unique constraints, one with and one without thumbnail_method? :D"
synapse=# \d remote_media_cache_thumbnails
Table "public.remote_media_cache_thumbnails"
Column | Type | Collation | Nullable | Default
------------------+---------+-----------+----------+---------
media_origin | text | | |
media_id | text | | |
thumbnail_width | integer | | |
thumbnail_height | integer | | |
thumbnail_method | text | | |
thumbnail_type | text | | |
thumbnail_length | integer | | |
filesystem_id | text | | |
Indexes:
"remote_media_cache_thumbnails_media_origin_media_id_thumbna_key" UNIQUE CONSTRAINT, btree (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)
"remote_media_repository_thumbn_media_origin_id_width_height_met" UNIQUE, btree (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type, thumbnail_method)
Version information
-
Homeserver: hacklab.fi
-
Version: 1.21.2