Skip to content

Expand warnings output for ResourceWarning #9644

@fschulze

Description

@fschulze

It is difficult to debug ResourceWarning from the existing output.

In Python3.6 a source field was added to the warnings.WarningMessage objects.

Since pytest now requires Python 3.7 I propose the following addition to the existing warning_record_to_str:

def warning_record_to_str(warning_message: warnings.WarningMessage) -> str:
    """Convert a warnings.WarningMessage to a string."""
    warn_msg = warning_message.message
    msg = warnings.formatwarning(
        str(warn_msg),
        warning_message.category,
        warning_message.filename,
        warning_message.lineno,
        warning_message.line,
    )
    tb = tracemalloc.get_object_traceback(warning_message.source)
    if tb is not None:
        msg = "%sSource of unclosed object:\n%s" % (msg, '\n'.join(tb.format()))
    elif issubclass(warning_message.category, ResourceWarning):
        # tracemalloc is inactive
        # TODO needs to be adjusted, see below in ticket
        msg = "%sGet a traceback for where the unclosed object was allocated by ..." % msg
    return msg

The output looks like this as an example:

  /.../devpi/devel/lib/python3.8/enum.py:315: ResourceWarning: unclosed <socket.socket [closed] fd=53, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
    return cls.__new__(cls, value)
  Source of unclosed object:
    File "/.../devpi/devel/lib/python3.8/site-packages/urllib3/connection.py", line 205
      conn = self._new_conn()
    File "/.../devpi/devel/lib/python3.8/site-packages/urllib3/connection.py", line 174
      conn = connection.create_connection(
    File "/.../devpi/devel/lib/python3.8/site-packages/urllib3/util/connection.py", line 77
      sock = socket.socket(af, socktype, proto)

This requires that tracemalloc is active. So the open question is, when should tracemalloc.start() be called and how to pass the size of the stack trace to it, as the default of 1 is often too small (I needed 20 to find the source of an unclosed requests Response). I propose three possible solutions:

  1. Document running pytest like this: python -X tracemalloc=20 -m pytest ...
  2. Add a command line option to enable tracemalloc with the limit as parameter.
  3. Enable tracing when warnings for ResourceWarning are set to once or always and always use a stack trace limit of 3.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueeasy issue that is friendly to new contributorplugin: warningsrelated to the warnings builtin plugintype: proposalproposal for a new feature, often to gather opinions or design the API around the new feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions