-
-
Notifications
You must be signed in to change notification settings - Fork 498
Closed
Labels
Milestone
Description
Hello,
There is a serious performance issue in current 12.2 branch. Problem is that we have ~2M thumbnails, so if execution goes here (marked line):
# /sorl/thumbnail/base.py:101 (get_thumbnail)
# We have to check exists() because the Storage backend does not
# overwrite in some implementations.
if not thumbnail.exists(): # <---- This is the root of the problem!
try:
source_image = default.engine.get_image(source)
except IOError:
if settings.THUMBNAIL_DUMMY:
return DummyImageFile(geometry_string)
else:
# if S3Storage says file doesn't exist remotely, don't try to
# create it and exit early.
# Will return working empty image type; 404'd image
logger.warn(text_type('Remote file [%s] at [%s] does not exist'),
file_, geometry_string)
return thumbnail
# We might as well set the size since we have the image in memory
image_info = default.engine.get_image_info(source_image)
options['image_info'] = image_info
size = default.engine.get_image_size(source_image)
source.set_size(size)
try:
self._create_thumbnail(source_image, geometry_string, options,
thumbnail)
self._create_alternative_resolutions(source_image, geometry_string,
options, thumbnail.name)
finally:
default.engine.cleanup(source_image)
it is actually asks boto to return LIST of all stored thumbnails (without even using prefix), so appliction hangs with 100% CPU and high memory usage (well, not a surprise actually).
Wouldn't it be better to provide a prefix for lookup (constructed with the same function as used to store thumbnail) ?
In a mean time we've had to revert to 11.12.1b which works better.
This is related to a fix introduced in #92