Releases: zlib-ng/zlib-ng
2.2.5
This is a bugfix release, backporting a few bugfixes and updating the CI.
Changes
Important fixes
- RiscV: chunkset_rvv: fix SIGSEGV in CHUNKCOPY #1889
- MSVC: Disable optimizations for AVX512 GET_CHUNK_MAG causing inflate failure #1884
- Fix building with runtime CPU detection disabled (native) #1931
- Also check for ZMM support when detecting VPCLMULQDQ support #1932
- Revert "Clean up insert_match() in deflate_medium" due to performance regression #1938
Buildsystem
- Pass POSIX_C_SOURCE for std::alligned_alloc try_compile checks #1896
- X86_AVX512VNNI: check for _mm256_dpbusd_epi32 too #1944
- CMake: Fix incorrect declaration of FORCE_SSE2 #1880
- CMake: Fix CXXFLAGS when coverage enabled #1902
- CMake: Remove late enable_language calls #1903
- CMake: [FreeBSD] Define _XOPEN_SOURCE for gtest_zlib #1900
- CMake: Add bindir into zlib.pc.in for compatibility with Cygwin and Msys2 #1920
- Configure: riscv: add bash configure script support for riscv 1904
Tests/Benchmarks
- Test: Fix pointer type mismatch #1897
- Test: Add large 1mb buffer test for crc32 hashing #1913
- Changes to running benchmark during tests #1892
CI
- CI: Restore support macOS prior 10.15 #1878
- CI: fixes for RISC-V #1890
- CI: Preinstall packages needed for testing and benchmark #1894
- CI: Remove deprecated ubuntu-20.04 image from CI #1898
- CI: Replace deprecated windows-2019 with windows-2022 #1923
Misc
- Add .gitignore to allow run tests with zlib-ng/corpora and local dataset from working copy #1930
2.2.4
This is primarily a bugfix release, fixing a few new and old bugs.
Changes
Important fixes
- Fix potential shift overflow problems reported by static checkers #1859
- VS2015: Fix an unfortunate bug #1862
- RVV: Workaround error G6E97C40B #1853
- s390x: Disable CRC32-VX Extensions for some broken Clang versions #1852
Buildsystem
Tests/Benchmarks
- Add uncompress benchmark #1860
CI
2.2.3
New Years release, zlib-ng's 10 years celebration
My first commit in this repo was in October 8th 2014, although I do remember that I started during summer vacation and had made the zlib cleanup more of a mess than I wanted, so I restarted from scratch in October when I had gotten a better overview of the code and what I wanted to do to clean it up. At that point zlib-ng was not very likely to go anywhere, but despite the odds, over time several people found it and opened PRs with their own improvements, a few of those became long-time contributors, and a few years ago zlib-ng finally became more than an experimental fork. Zlib-ng has since gained traction and several distros have started replacing stock zlib with zlib-ng in compat mode.
The past year we have been lucky enough to have received donations so that we were able to invest in a couple Rpi5 systems for testing, and we are looking forward to hopefully being able to acquire more architectures for development and testing, Risc-V would be interesting for example, and we are lacking a dedicated testing machine capable of AVX512.
Release 2.2.3
This time we have two code fixes for potentially unsafe access, although we have not had any bug-reports about these.
It also contains several optimizations. Especially of note, inflate has been optimized on various instruction sets and also the generic C code has seen improvements, and we have improvements for arches where unaligned accesses are not possible (lacking instructions to handle unaligned access) and also improvements on big endian.
Example benchmarks:
x86-64 AVX2: Inflate ~17.8% faster, Deflate unchanged. -4.6KB library size.
Aarch64: Inflate ~2.3% faster, Deflate unchanged. - 5.5KB library size.
We also took some time to do a comprehensive cleanup of the now misleading UNALIGNED_OK option and of all the "unaligned" functions. We have noticed that some distros have been disabling these, fearing they are using potentially unsafe unaligned pointers, but we already fixed that in 2.1.0-beta1. Since then, these "unaligned" settings/functions have been referring to using unaligned accesses in safe ways, like utilizing unaligned intrinsics or memcpy to fix alignment for example and selecting what safe method is optimal to the arch. So disabling that instead disabled several safe optimizations.
Because this was obviously misleading certain distros into disabling these optimizations, we have cleaned it up, removed a lot of unnecessary preprocessor checks, and made detection of optimal methods happen during compile instead of configure. As a bonus, this cleaned up a lot of code and also let us not compile in many extra variants of compare256/longest_match, saving about 8-10KB of library size.
- PS: s390x is currently potentially unsafe, CI reports a failure on the MSAN test, this is pending investigation by IBM. See #1845.
Update: This is caused by a bug in Clang versions 18 -> 19.1.2, ref: llvm/llvm-project#109113
Any Zlib-ng version on s390x built with VX-extensions enabled compiled using a buggy Clang version will be unsafe. - PPS: 32bit ARM windows release dlls failed to automatically compile due to Github Actions upgrading their build images, so unfortunately there are no binaries for that currently. This does not affect self-built binaries. See #1839.
Changes
Fixes for potentially unsafe access
Optimizations / Cleanups
- Allow the compiler to inline chunkcopy_safe more readily #1781
- Misc inflate cleanup#1797
- Reorder variables in inflate functions to reduce padding holes #1803
- Improve chunkset_avx2 performance #1778
- Simplify inflate fast by dispatching to chunkmemset for all chunkcopy cases #1802
- Make an AVX512 inflate fast with low cost masked writes #1805
- Enable AVX2 functions to be built with BMI2 instructions #1816
- Improve pipeling for AVX512 chunking #1821
- Risc-V: adler32_rvv: Fix two overflow problems #1826
- Remove UNALIGNED_OK checks #1828 #1834 #1835 #1830
- Use GCC's may_alias attribute for unaligned memory access #1548
Big Endian
- Make big endians first class citizens again #1831
- Fix "RLE" compression with big endian architectures #1832
Buildsys fixes / minor fixes
- Fix build on aarch64 android. #1783
- Allow overridde CMAKE_CXX_* variables and fix overridde CMAKE_C_* #1787
- Use target include instead of raw include #1784
- Replace non-ascii characters to fix MSVC warning #1791
- Force Visual C++ to treat source files as UTF-8. #1789
- Explicitly set CMake policy 0169 to silence warning #1792
- configure: Fix linker flags for Haiku. #1799
- configure: add --mandir to override $mandir on command line. #1800
- Force use of latest Windows SDK with 32-bit ARM support #1811
- Fix casting warning/error in test_compress_bound.cc #1814
- Remove unused HAVE_CHUNKMEMSET_1 define #1815
- Fix native detection of ARM CRC instruction #1818
- Address deprecated cmake version warning. #1812
- Add a fallback to ALIGNED_ macro for other compilers #1820
- added in-tree build artifacts to .gitignore #1823
- Fix typos #1825
CI
2.2.2
This release fixes a corruption bug with the inflateback implementation, this was
detected by Docker using pigz w/zlib-ng to decompress a 25GB image file and failing CRC.
Since this is so far the only known way to trigger the bug, it seems to be hard to hit.
Most of the rest are minor changes to avoid triggering warnings in MSVC or optional
warnings in other compilers, as well as a few minor fixes to the buildsystem and CI.
Changes
Important Fixes
- Don't use chunkunroll for inflateBack #1773
Buildsystem
- Enable warning C4242 and treat warnings as errors for Visual C++. #1768
- Fixed false positive HAVE_ARMV6_INTRIN value on old ARM platforms #1774
CI/Test
- Upgrade MacOS ARM64 UBSAN to use gcc-13. #1763
Misc
- Fix compiler warnings #1762 #1764 #1765
- Fix new Windows SDK build break #1771
- Prepare to make use of unaligned loads on big endian in insert_string #1695
S390x
- IBM zSystems: Hardcode HWCAP_S390_VXRS #1766
RISC-V
- Better run-time detection of RVV vector instruction support #1770
2.1.8
This backport release fixes a corruption bug with the inflateback implementation, this was
detected by Docker using pigz w/zlib-ng to decompress a 25GB image file and failing CRC.
Since this is so far the only known way to trigger the bug, it seems to be hard to hit.
Most of the rest are minor changes to avoid triggering warnings in MSVC or optional
warnings in other compilers, as well as a few minor fixes to the buildsystem and CI.
Other changes
Important Fixes
- Don't use chunkunroll for inflateBack #1773
Buildsystem
- Enable warning C4242 and treat warnings as errors for Visual C++. #1768
- Fixed false positive HAVE_ARMV6_INTRIN value on old ARM platforms #1774
- don't use zlib-ng's -Wl,--version-script in tests #1750
CI/Test
- Upgrade MacOS ARM64 UBSAN to use gcc-13. #1763
Misc
S390x
- IBM zSystems: Hardcode HWCAP_S390_VXRS #1766
RISC-V
- Better run-time detection of RVV vector instruction support #1770
2.2.1
This is the first stable release of the 2.2.x branch
Please read the changelog for the 2.2.0 Release Candidate if you didn't already, especially if your software gives zlib-ng a custom allocator.
No bugreports came in during 2.2.0 RC testing, so the only change in 2.2.1 is a small fix for Configure that was already in the pipe:
- Configure: Don't use zlib-ng's -Wl,--version-script in tests #1750
2.2.0 Release Candidate
This release contains several larger changes and optimizations. On x86-64 for example, this leads to a compression speedup of ~12% on default level.
We also have a major reorganization of memory alloc/free to always happen during init, this allows applications to potentially do the init early and be finished with the malloc system calls before it needs to process latency sensitive compression/decompression. It also ensures that zlib-ng can not fail due to memory pressure after having run the init functions successfully. We also now only do a single memory allocation deflate or inflate, ensuring we do less system calls and the allocated buffers live close together in memory.
Compression or decompression of very small buffers will now also be faster due to spending less time doing malloc/free.
The downside to this is that decompression will now always allocate the maximum required memory (~42KB total on 64-bit platforms), previously it would allocate (and potentially free) memory as needed during decompression.
It also means that applications that replace the alloc/free functions with their own can potentially have some issues (Yes I am looking at you Nginx).
Changes
Buildsystem
- Generate CMake package configuration files #1647
- Relocate CMake target export definitions #1657
- Allow overwrite NATIVEFLAG value. #1662 #1684
- Fix xsave intrinsic test for clang, and gcc 8.2 or newer, and icc #1664
- Disable Intel Compiler diagnostic message 10441 #1666
- Add missing checks for 64bit arm/intel with msvc compiler #1667
- Don't export git/github-related files in tar/zip archives #1688
- Cleanup and update NMake Makefiles #1673
- Add more result variables to the cmake package configuration #1671
- Fix building with NVHPC #1698
- CMake: Replace ; by $ in generator-expression #1707
- Bump max CMake policy version to 3.29.0 #1709
- make darwin cross compilation possible #1714
CI/Test
- Improve code coverage handling #1640 #1642 #1675 #1729
- Add VPCLMULQDQ crc32 tests to Google benchmarks #1651
- Add small compress() benchmark #1721 #1730
- Add back-and-forth inflateCopy() test #1731
- Enable orphaned unit tests for compare256_rle family of functions #1739
- Fix MSAN error in test_dict #1726
- CI workflows
- Add dependabot for github actions #1687
- Upgrade ilammy/msvc-dev-cmd to v1.13.0 #1665
- Upgrade codecov/codecov-action to v4. #1676
- Upgrade github/codeql-action from 2 to 3 #1691
- Upgrade actions/upload-artifact from 3 to 4 #1692
- Upgrade mymindstorm/setup-emsdk to v14. #1677
- Update dependencies for 32-bit MinGW CI run #1711
- Use windows-2019 for build with toolset v141 #1725
- Fix macOS Github Actions #1720
Cleanup
Refactoring and Optimizations
- Move C fallback functions into arch/generic [Part 1] #1630 #1631 #1658 #1668
- Remove unneeded pointer for functable.longest_match in deflate_slow #1633
- Improve x86 intrinsics dependency handling #1643
- Split cpu_features.h by architecture #1644
- Speed up crc32_[v]pclmulqdq on small strings. #1650
- Cleanup of bi_flush() #1660
- Split cpu features and arch functions #1685 #1696
- Inline CHUNKCOPY and CHUNKUNROLL #1669
- Remove update_hash and insert_string implementations from functable #1681
- Disable dynamic function dispatching for native or arch-specific builds #1659 #1701
- Clean up insert_match() in deflate_medium #1682
- Prefer HAVE_ALIGNED_ALLOC when available in zng_alloc #1635
- Rewrite deflate memory allocation #1713 #1736
ARM
- Add test for checking if -march=native needs -mfpu=neon for 32-bit ARM. #1683
- Override Clang x4 NEON intrinsics for Android #1694
- Add AArch64 feature detection support for OpenBSD #1732
- Improved Configure ACLE check #1727
Power
- Fix regression in Power8/9 detection #1649
RVV
- Optimized rvv slide_hash #1704
- arch/riscv/riscv_features.c: fix uclibc build #1700
- Disable CodeCov for RISC-V as the toolchain doesn't support generating code coverage #1679
S390x
- Update s390x dockerfile #1716
- IBM zSystems DFLTCC: Extend sanitizer checks #1717
- IBM zSystems DFLTCC: Inline DLFTCC states into zlib states #1718
- Remove unused function dfltcc_alloc_state #1728
x86
- Fix PCLMULQDQ, AVX512VNNI and VPCLMULQDQ feature tests for Intel LLVM compiler (icx) #1672
- Fix invalid instruction usage in Xeon Phi x200 processors #1723
Misc
2.1.7
Due to the high amount of refactoring changes in develop, I have decided to target those to a new version branch 2.2.x.
There is also a lot of fixes and minor improvements, so those will be backported and released as 2.1.7.
To work around the #1708 issue (incompatibility with applications mis-using the zlib zalloc/zfree API), #1710 is merged instead of backporting the much bigger #1713.
Backported Changes
Buildsystem
- Generate CMake package configuration files #1647
- Relocate CMake target export definitions #1657
- Fix xsave intrinsic test for clang, and gcc 8.2 or newer, and icc #1664
- Disable Intel Compiler diagnostic message 10441 #1666
- Add missing checks for 64bit arm/intel with msvc compiler #1667
- Don't export git/github-related files in tar/zip archives #1688
- Add more result variables to the cmake package configuration #1671
- Fix building with NVHPC #1698
- CMake: Replace ; by $ in generator-expression #1707
- Bump max CMake policy version to 3.29.0 #1709
- make darwin cross compilation possible #1714
CI/Test
- Improve code coverage handling #1640 #1642 #1675 #1729
- Add VPCLMULQDQ crc32 tests to Google benchmarks #1651
- Add small compress() benchmark #1721 #1730
- Add back-and-forth inflateCopy() test #1731
- Enable orphaned unit tests for compare256_rle family of functions #1739
- Fix MSAN error in test_dict #1726
- CI workflows
- Add dependabot for github actions #1687
- Upgrade ilammy/msvc-dev-cmd to v1.13.0 #1665
- Upgrade codecov/codecov-action to v4. #1676
- Upgrade github/codeql-action from 2 to 3 #1691
- Upgrade actions/upload-artifact from 3 to 4 #1692
- Upgrade mymindstorm/setup-emsdk to v14. #1677
- Update dependencies for 32-bit MinGW CI run #1711
- Use windows-2019 for build with toolset v141 #1725
- Fix macOS Github Actions #1720
Cleanup
Refactoring and Optimizations
- Remove unneeded pointer for functable.longest_match in deflate_slow #1633
ARM
- Add test for checking if -march=native needs -mfpu=neon for 32-bit ARM. #1683
- Override Clang x4 NEON intrinsics for Android #1694
- Add AArch64 feature detection support for OpenBSD #1732
Power
- Fix regression in Power8/9 detection #1649
RVV
- arch/riscv/riscv_features.c: fix uclibc build #1700
- Disable CodeCov for RISC-V as the toolchain doesn't support generating code coverage #1679
S390x
x86
- Fix PCLMULQDQ, AVX512VNNI and VPCLMULQDQ feature tests for Intel LLVM compiler (icx) #1672
- Fix invalid instruction usage in Xeon Phi x200 processors #1723
Misc
2.1.6
This is a stable release, with several minor improvements and one corruption fix for inflateCopy().
This release also improves on the functable implementation, and also moves its initialization to happen in deflateInit() and inflateInit(). We also have some optimizations for RVV and ARM.
Notes for packagers:
- FAR macro has been added back to zlib-compat mode in this release, please remember to remove downstream patches that add it.
- Please consider removing CMake
INSTALL_LIB_DIRworkarounds, this should not be needed since v2.0.2 (2021), but packagers seem to keep copying the workaround from each other. Please see cmake/detect-install-dirs.cmake.
Changes
- Fix inflateCopy corruption caused by change in 2.1.4 #1628
- This is a regression caused by a change introduced in 2.1.4
-
Functable
-
API
- Add FAR macro to zlib-compat headers to improve compatibility #1637
-
ARM
-
RVV
- Optimize adler32_fold_copy using RVV #1597
-
x86
- Simplify AVX2 and AVX512 adler32_fold_copy by removing templates #1599
-
Buildsystem
- Don't attempt ARMv6 detection on AARCH64 #1617
- Prevent tests writing into source directory #1604
- CMake: Fix clang-cl warnings #1591
- CMake: Export cmake target #1601 #1611
- CMake: Remove duplicate enable tests option #1610
- CMake: Fix reading version information from zlib.h.in #1614
- CMake: Check whether compiler supports -march=native or -mcpu=native #1618
- CMake: Always run compiler feature tests without LTO #1622
- CMake: Make sure uqsub16 check doesn't get optimized away with LTO #1619
- CMake: Update to GoogleTest 1.12.1 #1623
- CI: Add linter workflow for whitespace errors #1625 #1632
- CI: Cancel outdated running CI jobs for PR or branch #1629
- CI: Added CI instance for WITH_NATIVE_INSTRUCTIONS #1634
- Tests: Fix buffer overflow in compare256_rle benchmark #1612
-
Misc
- Update copyright to sync with zlib 1.3 #1615
2.1.5
This is a hotfix release, fixing an issue where certain applications would fail with a checksum error during inflate (decompression).
A few minor fixes and improvements are also included.