Add trailer parsing logic #11269

Dreamsorcerer · 2025-07-04T18:01:22Z

This adds trailer parsing to the parsing logic. First step towards #1652.

It does not yet expose the trailers, so there are no visible user changes, but paves the way for a future PR to add the public API to these.

for more information, see https://pre-commit.ci

aiohttp/http_parser.py

codecov · 2025-07-04T18:06:01Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.84%. Comparing base (c106c5b) to head (0415f00).
Report is 4 commits behind head on master.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11269      +/-   ##
==========================================
- Coverage   98.84%   98.84%   -0.01%     
==========================================
  Files         131      131              
  Lines       43262    43258       -4     
  Branches     2327     2326       -1     
==========================================
- Hits        42761    42757       -4     
  Misses        346      346              
  Partials      155      155

Flag	Coverage Δ
CI-GHA	`98.73% <100.00%> (-0.01%)`	⬇️
OS-Linux	`98.46% <100.00%> (-0.01%)`	⬇️
OS-Windows	`96.79% <98.14%> (-0.01%)`	⬇️
OS-macOS	`97.68% <98.14%> (-0.01%)`	⬇️
Py-3.10.11	`97.33% <98.14%> (-0.01%)`	⬇️
Py-3.10.18	`97.82% <100.00%> (-0.01%)`	⬇️
Py-3.11.13	`98.00% <100.00%> (-0.01%)`	⬇️
Py-3.11.9	`97.52% <98.14%> (-0.01%)`	⬇️
Py-3.12.10	`97.63% <98.14%> (-0.01%)`	⬇️
Py-3.12.11	`98.10% <100.00%> (-0.01%)`	⬇️
Py-3.13.3	`98.36% <100.00%> (-0.01%)`	⬇️
Py-3.9.13	`97.21% <98.14%> (-0.02%)`	⬇️
Py-3.9.23	`97.69% <100.00%> (-0.01%)`	⬇️
Py-pypy7.3.16	`93.83% <98.14%> (-1.14%)`	⬇️
VM-macos	`97.68% <98.14%> (-0.01%)`	⬇️
VM-ubuntu	`98.46% <100.00%> (-0.01%)`	⬇️
VM-windows	`96.79% <98.14%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

… add-trailer-parser

codspeed-hq · 2025-07-04T18:14:46Z

CodSpeed Performance Report

Merging #11269 will not alter performance

_{Comparing add-trailer-parser (0415f00) with master (91108c9)}

Summary

✅ 59 untouched benchmarks

for more information, see https://pre-commit.ci

tests/test_http_parser.py

for more information, see https://pre-commit.ci

… add-trailer-parser

tests/test_http_parser.py

CHANGES/11269.feature.rst

Copilot

Pull Request Overview

This PR adds support for parsing HTTP trailers in the core parser and updates tests to exercise the new logic without yet exposing a public API.

Integrates HeadersParser into HttpPayloadParser and HTTP parsing routines for trailer extraction.
Updates and adds tests for valid and invalid trailer parsing scenarios.
Adjusts multipart header reader initialization and records the feature in CHANGES.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
tests/test_http_parser.py	Added new trailer tests and updated `HttpPayloadParser` instantiations with `HeadersParser`.
aiohttp/multipart.py	Changed `_read_headers` initialization to start with an empty list.
aiohttp/http_parser.py	Integrated trailer parsing logic, mandatory `headers_parser` injection, and adjusted `parse_headers` slicing.
CHANGES/11269.feature.rst	Logged the initial trailer parsing feature.

Comments suppressed due to low confidence (4)

aiohttp/http_parser.py:769

Adding a mandatory headers_parser parameter to HttpPayloadParser is a breaking change. Consider providing a default value (e.g., headers_parser: HeadersParser = HeadersParser()) to maintain backward compatibility.

        headers_parser: HeadersParser,

CHANGES/11269.feature.rst:1

[nitpick] The change log entry is succinct but could be expanded to briefly describe what trailer parsing covers and note that the API is not yet public-facing.

Added initial trailer parsing logic -- by :user:`Dreamsorcerer`.

aiohttp/multipart.py:784

Initializing lines as an empty list may skip the expected placeholder and could lead to out-of-bounds in downstream header parsing. Ensure that the new list state correctly represents the first header line or reintroduce a sentinel if needed.

        lines = []

tests/test_http_parser.py:1640

[nitpick] Tests repeatedly instantiate HttpPayloadParser with a HeadersParser argument; consider refactoring to use a fixture or default constructor value to reduce boilerplate.

        p = HttpPayloadParser(out, headers_parser=HeadersParser())

tests/test_http_parser.py

bdraco

Looks good. I didn't find any logic errors. 👍

patchback · 2025-07-09T18:16:12Z

Backport to 3.12: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 7dd4b55 on top of patchback/backports/3.12/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269

Backporting merged PR #11269 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.12/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269 upstream/3.12

Now, cherry-pick PR Add trailer parsing logic #11269 contents into that branch:
```
$ git cherry-pick -x 7dd4b5535e6bf9c2d2f05fde638517bff065ba74
```
If it'll yell at you with something like fatal: Commit 7dd4b5535e6bf9c2d2f05fde638517bff065ba74 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 7dd4b5535e6bf9c2d2f05fde638517bff065ba74
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Add trailer parsing logic #11269 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.12/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2025-07-09T18:16:22Z

Backport to 3.13: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 7dd4b55 on top of patchback/backports/3.13/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269

Backporting merged PR #11269 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.13/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269 upstream/3.13

Now, cherry-pick PR Add trailer parsing logic #11269 contents into that branch:
```
$ git cherry-pick -x 7dd4b5535e6bf9c2d2f05fde638517bff065ba74
```
If it'll yell at you with something like fatal: Commit 7dd4b5535e6bf9c2d2f05fde638517bff065ba74 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 7dd4b5535e6bf9c2d2f05fde638517bff065ba74
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Add trailer parsing logic #11269 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.13/7dd4b5535e6bf9c2d2f05fde638517bff065ba74/pr-11269

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

(cherry picked from commit 7dd4b55)

Add trailer parsing

83dd211

Dreamsorcerer requested a review from asvetlov as a code owner July 4, 2025 18:01

Dreamsorcerer requested review from bdraco and removed request for asvetlov July 4, 2025 18:01

[pre-commit.ci] auto fixes from pre-commit.com hooks

aa6027c

for more information, see https://pre-commit.ci

github-advanced-security bot found potential problems Jul 4, 2025

View reviewed changes

aiohttp/http_parser.py Dismissed Show dismissed Hide dismissed

aiohttp/http_parser.py Dismissed Show dismissed Hide dismissed

Dreamsorcerer commented Jul 4, 2025

View reviewed changes

aiohttp/http_parser.py Outdated Show resolved Hide resolved

Dreamsorcerer added 2 commits July 4, 2025 19:07

Fix exception type

b21bc45

Merge branch 'add-trailer-parser' of github.com:aio-libs/aiohttp into…

5aa8b90

… add-trailer-parser

Dreamsorcerer added 2 commits July 4, 2025 19:15

Fix raises

e36cff4

Fix exception not raising on feed_data()

5ad2e34

Dreamsorcerer requested a review from webknjaz as a code owner July 4, 2025 19:16

pre-commit-ci bot and others added 3 commits July 4, 2025 19:17

[pre-commit.ci] auto fixes from pre-commit.com hooks

5f03d9d

for more information, see https://pre-commit.ci

Update setup.cfg

9b0163d

Fix test

36ea8b8

github-advanced-security bot found potential problems Jul 4, 2025

View reviewed changes

tests/test_http_parser.py Fixed Show fixed Hide fixed

tests/test_http_parser.py Fixed Show fixed Hide fixed

tests/test_http_parser.py Fixed Show resolved Hide resolved

Dreamsorcerer and others added 4 commits July 6, 2025 12:26

Fix

3d54e7d

[pre-commit.ci] auto fixes from pre-commit.com hooks

3028fdd

for more information, see https://pre-commit.ci

Avoid passing status line to HeadersParser

588bbdc

Merge branch 'add-trailer-parser' of github.com:aio-libs/aiohttp into…

3c97cfd

… add-trailer-parser

Dreamsorcerer commented Jul 6, 2025

View reviewed changes

tests/test_http_parser.py Outdated Show resolved Hide resolved

Dreamsorcerer added 2 commits July 6, 2025 13:15

Update tests/test_http_parser.py

fd12f30

Create 11269.feature.rst

95c3088

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Jul 6, 2025

Dreamsorcerer added backport-3.12 Trigger automatic backporting to the 3.12 release branch by Patchback robot backport-3.13 Trigger automatic backporting to the 3.13 release branch by Patchback robot and removed bot:chronographer:provided There is a change note present in this PR labels Jul 6, 2025

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Jul 6, 2025

Dreamsorcerer changed the title ~~Add trailer parsing~~ Add trailer parsing logic Jul 6, 2025

webknjaz reviewed Jul 8, 2025

View reviewed changes

CHANGES/11269.feature.rst Outdated Show resolved Hide resolved

webknjaz requested a review from Copilot July 8, 2025 19:22

Copilot AI reviewed Jul 8, 2025

View reviewed changes

webknjaz reviewed Jul 8, 2025

View reviewed changes

tests/test_http_parser.py Show resolved Hide resolved

Dreamsorcerer added 2 commits July 9, 2025 18:07

Update 11269.feature.rst

0c3e827

Update 11269.feature.rst

0415f00

bdraco approved these changes Jul 9, 2025

View reviewed changes

Dreamsorcerer merged commit 7dd4b55 into master Jul 9, 2025
40 checks passed

Dreamsorcerer deleted the add-trailer-parser branch July 9, 2025 18:16

Dreamsorcerer added a commit that referenced this pull request Jul 9, 2025

Add trailer parsing logic (#11269)

9faabc9

(cherry picked from commit 7dd4b55)

Dreamsorcerer added a commit that referenced this pull request Jul 9, 2025

Add trailer parsing logic (#11269)

dd9d602

(cherry picked from commit 7dd4b55)

Dreamsorcerer added a commit that referenced this pull request Jul 9, 2025

Add trailer parsing logic (#11269) (#11286)

41115b7

(cherry picked from commit 7dd4b55)

Dreamsorcerer added a commit that referenced this pull request Jul 9, 2025

Add trailer parsing logic (#11269) (#11287)

e8d774f

(cherry picked from commit 7dd4b55)

Dreamsorcerer mentioned this pull request Jul 11, 2025

Trailer headers #1652

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add trailer parsing logic #11269

Add trailer parsing logic #11269

Uh oh!

Dreamsorcerer commented Jul 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jul 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

bdraco left a comment

Uh oh!

Uh oh!

patchback bot commented Jul 9, 2025 •

edited

Loading

Uh oh!

patchback bot commented Jul 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Add trailer parsing logic #11269

Add trailer parsing logic #11269

Uh oh!

Conversation

Dreamsorcerer commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #11269 will not alter performance

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

bdraco left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

patchback bot commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 3.12: 💔 cherry-picking failed — conflicts found

Backporting merged PR #11269 into master

Uh oh!

patchback bot commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 3.13: 💔 cherry-picking failed — conflicts found

Backporting merged PR #11269 into master

Uh oh!

Uh oh!

Dreamsorcerer commented Jul 4, 2025 •

edited

Loading

codecov bot commented Jul 4, 2025 •

edited

Loading

codspeed-hq bot commented Jul 4, 2025 •

edited

Loading

patchback bot commented Jul 9, 2025 •

edited

Loading

patchback bot commented Jul 9, 2025 •

edited

Loading