Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Rate Limiter: RecursionError: maximum recursion depth exceeded #14480

@richvdh

Description

@richvdh

This can't be good (observed in the federation-reader logs on libera.chat):

2022-11-17 10:48:27,448 - twisted - 274 - CRITICAL - sentinel- Unhandled error in Deferred:
2022-11-17 10:48:27,474 - twisted - 274 - CRITICAL - sentinel- 
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 307, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 513, in _async_render
    callback_return = await raw_callback_return
  File "/usr/local/lib/python3.9/site-packages/synapse/federation/transport/server/_base.py", line 342, in new_func
    await d
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 993, in send
    raise result.value
synapse.api.errors.LimitExceededError: 429: Too Many Requests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/synapse/util/ratelimitutils.py", line 249, in ratelimit
    yield ret
  File "/usr/local/lib/python3.9/site-packages/synapse/federation/transport/server/_base.py", line 344, in new_func
    logger.warning(
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1458, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1577, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info, stacklevel)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1519, in findCaller
    f = currentframe()
  File "/usr/local/lib/python3.9/logging/__init__.py", line 160, in <lambda>
    currentframe = lambda: sys._getframe(3)
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 307, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 513, in _async_render
    callback_return = await raw_callback_return
  File "/usr/local/lib/python3.9/site-packages/synapse/federation/transport/server/_base.py", line 349, in new_func
    response = await func(
  File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/synapse/util/ratelimitutils.py", line 251, in ratelimit
    self._on_exit(request_id)
  File "/usr/local/lib/python3.9/site-packages/synapse/util/ratelimitutils.py", line 366, in _on_exit
    logger.debug("Ratelimit(%s) [%s]: Processed req", self.host, id(request_id))
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/synapse/http/site.py", line 309, in processing
    yield
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 229, in wrapped_async_request_handler
    await h(self, request)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 317, in _async_render_wrapper
    self._send_error_response(f, request)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 525, in _send_error_response
    return_json_error(f, request, self.hs.config)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 124, in return_json_error
    logger.error(
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1475, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1577, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info, stacklevel)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1519, in findCaller
    f = currentframe()
  File "/usr/local/lib/python3.9/logging/__init__.py", line 160, in <lambda>
    currentframe = lambda: sys._getframe(3)
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/synapse/http/site.py", line 312, in processing
    logger.exception(
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1481, in exception
    self.error(msg, *args, exc_info=exc_info, **kwargs)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1475, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1577, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info, stacklevel)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1519, in findCaller
    f = currentframe()
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1696, in _inlineCallbacks
    result = context.run(gen.send, result)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 229, in wrapped_async_request_handler
    await h(self, request)
  File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/synapse/http/site.py", line 328, in processing
    self._finished_processing()
  File "/usr/local/lib/python3.9/site-packages/synapse/http/site.py", line 428, in _finished_processing
    usage = self.logcontext.get_resource_usage()
  File "/usr/local/lib/python3.9/site-packages/synapse/logging/context.py", line 481, in get_resource_usage
    res = self._resource_usage.copy()
  File "/usr/local/lib/python3.9/site-packages/synapse/logging/context.py", line 136, in copy
    return ContextResourceUsage(copy_from=self)
RecursionError: maximum recursion depth exceeded

Metadata

Metadata

Assignees

Labels

A-FederationO-OccasionalAffects or can be seen by some users regularly or most users rarelyS-MajorMajor functionality / product severely impaired, no satisfactory workaround.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions