Profile-Guided Optimization (PGO) results #5
Description
Hi!
Recently I did Profile-Guided Optimization (PGO) evaluation on multiple projects (the results are available here). So I decided to test QOAudio benchmarks with PGO.
I tested QOAudio with PGO in two configurations:
- AMD Ryzen 9 5900x, Linux kernel 6.4.7, Fedora 38
- Macbook M1 Pro, Ventura 13.4
PGO optimization is done via cargo-pgo. The profile is collected from benchmarks with cargo pgo instrument bench, PGO optimization is done with cargo pgo optimize bench. I got the following results:
Linux:
PGO compared to Release
Benchmarking iter_sprained_ankle
Benchmarking iter_sprained_ankle: Warming up for 3.0000 s
Benchmarking iter_sprained_ankle: Collecting 100 samples in estimated 5.0646 s (200 iterations)
Benchmarking iter_sprained_ankle: Analyzing
iter_sprained_ankle time: [26.246 ms 26.344 ms 26.438 ms]
change: [-28.923% -28.603% -28.260%] (p = 0.00 < 0.05)
Performance has improved.
Found 19 outliers among 100 measurements (19.00%)
16 (16.00%) low severe
3 (3.00%) high mild
Mac:
PGO compared to Release
Benchmarking iter_sprained_ankle
Benchmarking iter_sprained_ankle: Warming up for 3.0000 s
Benchmarking iter_sprained_ankle: Collecting 100 samples in estimated 8.8845 s (200 iterations)
Benchmarking iter_sprained_ankle: Analyzing
iter_sprained_ankle time: [44.183 ms 44.252 ms 44.331 ms]
change: [+5.6024% +7.5641% +9.0545%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
The results are quite interesting. On Linux, the performance is improved but on Mac performance with PGO is worse. I think this information could be useful for users who are interested in optimizing performance for their programs.
Later I guess to perform LLVM BOLT evaluation too (also could be done with cargo-pgo).