-
Notifications
You must be signed in to change notification settings - Fork 7.5k
Gzip: compatibility with recent zlib-ng 2.2.x versions. #403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are two changes in zlib-ng we are addressing here.
First, zlib-ng/zlib-ng@a3fb271 which increases pending buffer size by a factor of 5/4 under certain circumstances. I agree with the change to increase pending buffer size and with the relevant part of the commit log.
Second, zlib-ng/zlib-ng@5c7a066 which changes allocation model to make only one big allocation instead of multiple small ones as before. Here, according to my calculations (see https://github.com/zlib-ng/zlib-ng/blob/develop/deflate.c#L202), the overall buffer size should increase. Particularly, 4 * (64 + sizeof(void *))
should be changed to 7 * (64 + sizeof(void *))
:
- 64 per allocation x5
- 64 for total size padding
- 64 for window padding (non-s390)
7 * sizeof(void *)
for thedeflate_allocs
structure
The original 4 * (64 + sizeof(void *))
is no longer needed since ZALLOC
is no longer used.
I did not get the alignment adjustment explanation of why the 4->7 (you mentioned 8) change can be ignored.
Previously that was unconditional increase in memory, now this is roundup (see related macros). Subranges are large enough in size (cannot prove that atm b/c afk), so the roundup should be nop. |
Now with numbers as mentioned. As stated in 8747191, previously used alloc_aligned() wrapper gave unconditional extra memory space allocation, see ZALLOC and alloc_aligned(). In particular:
In both cases, this gives the last sub-allocation padded to 16 bytes alignment, and the Hope this helps. |
It now uses 5/4 times more memory for the pending buffer. Further, a single allocation is now used, which takes additional 56 bytes for deflate_allocs in 64-bit mode aligned to 16, to store sub-allocation pointers, and the total allocation size now padded up to 128 bytes, which takes theoretically 200 additional bytes in total. This fits though into "4 * (64 + sizeof(void*))" additional space for ZALLOC used in zlib-ng 2.1.x versions. The comment was updated to reflect this.
I agree with calculation, they make sense in the worst theoretical case. This changes to:
That was a typo, thx, should be read as "4". Fixed in the commit message. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks ok.
Nginx version 1.27.4 and above is compatible with zlib-ng versions 2.2.x: nginx/nginx#403
It now uses 5/4 times more memory for the pending buffer.
Further, a single allocation is used, with up to 128 additional bytes for alignment. Though, this fits into "8 * (64 + sizeof(void*))" alignment adjustment used for previous zlib-ng versions. The comment was updated to reflect this.
Closes #263.