HTTPResponse.shutdown may shut down a connection already released to the pool #3581
Description
Subject
It appears that calling shutdown on a response will terminate that response's connection, even if that connection has already been returned to the pool and some other response is using it.
Environment
OS Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.39
Python 3.12.3
OpenSSL 3.0.13 30 Jan 2024
urllib3 2.3.0
Steps to Reproduce
import urllib3
with urllib3.HTTPSConnectionPool("urllib3.readthedocs.io") as pool:
resp1 = pool.urlopen("GET", "https://urllib3.readthedocs.io/", preload_content=False)
resp1.drain_conn()
resp1.release_conn()
resp2 = pool.urlopen("GET", "https://urllib3.readthedocs.io/", preload_content=False)
resp1.shutdown()
resp2.read()
resp2.close()Expected Behavior
The call to resp1.shutdown should either have no effect, or raise an exception. Personally, I would prefer if it did nothing (there's nothing to shut down, after all), but an exception would be reasonable too.
Actual Behavior
The call to resp2.read raises instead:
Traceback (most recent call last):
File "/usr/lib/python3.12/http/client.py", line 579, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 546, in _read_next_chunk_size
return int(line, 16)
^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 16: b'\x17\x03\x03\x05j$)wm\xfa\x02\x87\x0b1C\x85\x89\x18S\x85S\xb3\x88\xb0\xd2c"\xc9\xbc\xc8\xd4\x9d\x85\xe8\xc4\xe6\xb7\xad7\xb0\x03\xb4k\xe4\x01\xd4\xe8~\xc9\xe5\xef\x81\x1b\xffs$\xfb\xa9\x19S \xd0{\xc
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.12/http/client.py", line 595, in _read_chunked
while (chunk_left := self._get_chunk_left()) is not None:
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 581, in _get_chunk_left
raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 754, in _error_catcher
yield
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 879, in _raw_read
data = self._fp_read(amt, read1=read1) if not fp_closed else b""
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 862, in _fp_read
return self._fp.read(amt) if amt is not None else self._fp.read()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 473, in read
return self._read_chunked(amt)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/http/client.py", line 607, in _read_chunked
raise IncompleteRead(b''.join(value)) from exc
http.client.IncompleteRead: IncompleteRead(0 bytes read)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/urllib3-repro.py", line 12, in <module>
resp2.read()
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 955, in read
data = self._raw_read(amt)
^^^^^^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 878, in _raw_read
with self._error_catcher():
File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/tmp/venv/lib/python3.12/site-packages/urllib3/response.py", line 781, in _error_catcher
raise ProtocolError(f"Connection broken: {e!r}", e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))