What tools do you use for benchmarking? #326
Description
Today I made some benchmarks using jsnell/zlib-bench, using the following config:
my @versions = (
{ id => 'baseline', repository => 'https://github.com/madler/zlib.git', commit_or_branch => 'cacf7f1' },
{ id => 'cloudflare', repository => 'https://github.com/cloudflare/zlib.git', commit_or_branch => 'e55212b' },
{ id => 'intel', repository => 'https://github.com/jtkukunas/zlib.git', commit_or_branch => '4b0ef4e'},
{ id => 'zlib-ng', repository => 'https://github.com/zlib-ng/zlib-ng.git', commit_or_branch => '3ee7109', CONFIGURE_FLAGS => '--zlib-compat'},
);
None of the results are in favor of zlib-ng, even tho it afaik uses patches from both intel and cf?
Am I missing something here? Why does intel outperform everyone on decompression?
What does cloudflare do at level 9?
results:
baseline cloudflare intel zlib-ng
dec exec time [s]: 1.78 ± 0.00 (100.00%) 1.52 ± 0.00 ( 85.74%) 1.37 ± 0.00 ( 77.07%) 1.56 ± 0.00 ( 87.75%)
dec html time [s]: 1.02 ± 0.00 (100.00%) 0.87 ± 0.00 ( 85.42%) 0.68 ± 0.00 ( 67.01%) 0.90 ± 0.00 ( 88.32%)
dec jpeg time [s]: 0.24 ± 0.00 (100.00%) 0.13 ± 0.00 ( 52.57%) 0.14 ± 0.00 ( 60.33%) 0.24 ± 0.00 (100.84%)
dec pngp time [s]: 1.12 ± 0.00 (100.00%) 0.86 ± 0.00 ( 76.44%) 0.74 ± 0.00 ( 66.07%) 0.89 ± 0.00 ( 79.71%)
compress executable -5 (2 iterations)
exec 0.33 time [s]: 0.47 ± 0.00 (100.00%) 0.25 ± 0.00 ( 53.22%) 0.26 ± 0.00 ( 55.07%) 0.27 ± 0.00 ( 58.73%)
html 0.33 time [s]: 0.28 ± 0.00 (100.00%) 0.16 ± 0.00 ( 55.74%) 0.15 ± 0.00 ( 52.17%) 0.18 ± 0.00 ( 64.24%)
jpeg 1.00 time [s]: 0.20 ± 0.00 (100.00%) 0.14 ± 0.00 ( 71.31%) 0.19 ± 0.00 ( 96.80%) 0.20 ± 0.00 (102.78%)
pngp 0.14 time [s]: 0.35 ± 0.00 (100.00%) 0.16 ± 0.00 ( 46.29%) 0.19 ± 0.00 ( 55.16%) 0.20 ± 0.00 ( 55.36%)
compress executable -9 (2 iterations)
exec 0.33 time [s]: 2.55 ± 0.00 (100.00%) 1.09 ± 0.00 ( 42.72%) 2.00 ± 0.00 ( 78.40%) 1.97 ± 0.00 ( 77.23%)
html 0.33 time [s]: 0.77 ± 0.00 (100.00%) 0.43 ± 0.00 ( 56.55%) 0.68 ± 0.00 ( 88.47%) 0.68 ± 0.00 ( 88.42%)
jpeg 1.00 time [s]: 0.19 ± 0.00 (100.00%) 0.14 ± 0.00 ( 73.97%) 0.14 ± 0.00 ( 73.81%) 0.16 ± 0.00 ( 83.79%)
pngp 0.12 time [s]: 7.02 ± 0.00 (100.00%) 3.85 ± 0.00 ( 54.89%) 5.74 ± 0.00 ( 81.77%) 5.15 ± 0.00 ( 73.38%)