Skip to content

Take semver-compatible updates #2481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 9, 2025
Merged

Take semver-compatible updates #2481

merged 1 commit into from
Jun 9, 2025

Conversation

ctz
Copy link
Member

@ctz ctz commented Jun 6, 2025

General cargo update, but with the impetus of taking aws-lc >=1.52.0 via aws-lc-sys 0.29.0 via aws-lc-rs 1.13.1 as this has improved ML-KEM performance compared to aws-lc 1.51.2.

@rustls-benchmarking Rustls Benchmarking
Copy link

rustls-benchmarking bot commented Jun 6, 2025 •
edited
Loading

Benchmark results

Instruction counts

Significant differences

⚠️ There are significant instruction count differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 78523487 64404619 ✅ -14118868 (-17.98%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 78459308 64356218 ✅ -14103090 (-17.98%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 78727050 64583105 ✅ -14143945 (-17.97%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 78492680 64413375 ✅ -14079305 (-17.94%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 78731296 64646076 ✅ -14085220 (-17.89%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 78421404 64392360 ✅ -14029044 (-17.89%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 78708293 64633464 ✅ -14074829 (-17.88%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 78677049 64640682 ✅ -14036367 (-17.84%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 78724120 64720175 ✅ -14003945 (-17.79%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 78489300 64528752 ✅ -13960548 (-17.79%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 78686419 64702934 ✅ -13983485 (-17.77%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 78509682 64561764 ✅ -13947918 (-17.77%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 3579880 3110300 ✅ -469580 (-13.12%) 1.22%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 3586386 3116778 ✅ -469608 (-13.09%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 55217844 48219378 ✅ -6998466 (-12.67%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 55301465 48295935 ✅ -7005530 (-12.67%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 55227154 48239378 ✅ -6987776 (-12.65%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 55202140 48240645 ✅ -6961495 (-12.61%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 55272805 48346143 ✅ -6926662 (-12.53%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 55291740 48371217 ✅ -6920523 (-12.52%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 56603194 49585851 ✅ -7017343 (-12.40%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 56579238 49590842 ✅ -6988396 (-12.35%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 56585643 49602939 ✅ -6982704 (-12.34%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 56550500 49586430 ✅ -6964070 (-12.31%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 56603374 49656027 ✅ -6947347 (-12.27%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 56548188 49610657 ✅ -6937531 (-12.27%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 2003539 1770336 ✅ -233203 (-11.64%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 2001124 1768251 ✅ -232873 (-11.64%) 1.25%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 4749721 4281523 ✅ -468198 (-9.86%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 4746049 4279079 ✅ -466970 (-9.84%) 0.96%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 2904439 2673286 ✅ -231153 (-7.96%) 0.83%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 2907365 2676171 ✅ -231194 (-7.95%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 9941648 9496201 ✅ -445447 (-4.48%) 0.83%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 9949369 9512930 ✅ -436439 (-4.39%) 0.58%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 11633854 11344903 ✅ -288951 (-2.48%) 1.09%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 11591369 11364341 ✅ -227028 (-1.96%) 1.50%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 10472226 10492825 20599 (0.20%) 1.34%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3303402 3305633 2231 (0.07%) 0.28%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 5008960 5011030 2070 (0.04%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 3867561 3869091 1530 (0.04%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 3877231 3878761 1530 (0.04%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4219146 4220676 1530 (0.04%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 10999802 10996538 -3264 (-0.03%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 11127938 11124674 -3264 (-0.03%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 11133699 11130435 -3264 (-0.03%) 0.20%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 1719908 1720113 205 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46452696 46457821 5125 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46459276 46464401 5125 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46448259 46453381 5122 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 58232579 58237704 5125 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 58257402 58262524 5122 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 58235451 58240570 5119 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46397771 46401603 3832 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 58156698 58160533 3835 (0.01%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3304283 3304377 94 (0.00%) 0.23%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 1296472 1296489 17 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 1295280 1295292 12 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80643595 80644246 651 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80632581 80633216 635 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80637665 80638300 635 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 2246486 2246502 16 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92723319 92723960 641 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92697217 92697857 640 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 2334234 2334250 16 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 2339883 2339899 16 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92699462 92700091 629 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 7220994 7221010 16 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 7222963 7222972 9 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 34736323 34736311 -12 (-0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46189286 46189296 10 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 34738044 34738039 -5 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58123164 58123156 -8 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80543179 80543189 10 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46289085 46289082 -3 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80533868 80533873 5 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58130367 58130370 3 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46294220 46294222 2 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92668613 92668617 4 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46284263 46284262 -1 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58154764 58154763 -1 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92659489 92659490 1 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92691089 92691090 1 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 30838844 30838844 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 32111132 32111132 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 32554627 32554627 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_client 31232787 31232787 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client 30927205 30927205 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 31154191 31154191 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 31157528 31157528 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 32452630 32452630 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 31161516 31161516 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 32452672 32452672 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 32111235 32111235 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 32108080 32108080 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_server 4298146 4298146 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_server 4757454 4757454 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 31228799 31228799 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 32449863 32449863 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 30835915 30835915 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 31989585 31989585 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 31986430 31986430 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 31225441 31225441 0 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80538690 80538690 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 31989482 31989482 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_server 32554540 32554540 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_client 4571268 4571268 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_client 4307830 4307830 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client 30930134 30930134 0 (0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_client 58036065 58036065 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 30934188 30934188 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 30842898 30842898 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 32551818 32551818 0 (0.00%) 0.20%

Wall-time

Significant differences

⚠️ There are significant wall-time differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 10.57 ms 8.58 ms ✅ -1.99 ms (-18.80%) 3.75%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 10.52 ms 8.55 ms ✅ -1.96 ms (-18.68%) 3.40%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 10.75 ms 8.77 ms ✅ -1.99 ms (-18.48%) 3.38%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 10.71 ms 8.74 ms ✅ -1.97 ms (-18.38%) 3.12%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 11.25 ms 9.27 ms ✅ -1.98 ms (-17.64%) 3.44%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 11.19 ms 9.22 ms ✅ -1.97 ms (-17.59%) 3.10%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 11.24 ms 9.27 ms ✅ -1.97 ms (-17.56%) 3.27%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 11.18 ms 9.23 ms ✅ -1.95 ms (-17.47%) 2.88%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 11.45 ms 9.45 ms ✅ -1.99 ms (-17.40%) 3.07%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 11.41 ms 9.44 ms ✅ -1.98 ms (-17.32%) 3.09%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 11.37 ms 9.40 ms ✅ -1.97 ms (-17.32%) 2.52%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 11.39 ms 9.41 ms ✅ -1.97 ms (-17.32%) 2.78%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 647.82 µs 587.64 µs ✅ -60.18 µs (-9.29%) 3.66%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 648.60 µs 590.03 µs ✅ -58.56 µs (-9.03%) 3.68%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.33 ms 1.26 ms ✅ -0.06 ms (-4.80%) 1.52%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.34 ms 1.27 ms ✅ -0.06 ms (-4.62%) 1.67%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.29 ms 1.24 ms ✅ -0.06 ms (-4.43%) 2.34%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.30 ms 1.24 ms ✅ -0.06 ms (-4.38%) 2.39%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.56 ms 3.61 ms ⚠️ 0.05 ms (1.27%) 1.00%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.56 ms 3.60 ms ⚠️ 0.04 ms (1.24%) 1.00%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.02 ms 5.12 ms 0.09 ms (1.88%) 6.13%
transfer_no_resume_ring_1.3_ecdsap256_aes 5.34 ms 5.43 ms 0.08 ms (1.54%) 6.30%
transfer_no_resume_ring_1.3_ecdsap384_aes 8.45 ms 8.57 ms 0.12 ms (1.42%) 3.69%
transfer_no_resume_ring_1.3_rsa_aes 5.84 ms 5.92 ms 0.08 ms (1.40%) 5.71%
transfer_no_resume_ring_1.2_rsa_aes 5.79 ms 5.86 ms 0.07 ms (1.28%) 5.43%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.06 ms 1.07 ms 0.01 ms (1.27%) 1.75%
handshake_no_resume_ring_1.3_ecdsap256_aes 462.23 µs 467.98 µs 5.75 µs (1.24%) 2.66%
handshake_no_resume_ring_1.3_ecdsap256_chacha 459.87 µs 464.99 µs 5.12 µs (1.11%) 3.23%
handshake_tickets_ring_1.3_ecdsap384_chacha 8.63 ms 8.72 ms 0.08 ms (0.98%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 8.67 ms 8.75 ms 0.08 ms (0.91%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_aes 8.60 ms 8.68 ms 0.08 ms (0.90%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_chacha 8.57 ms 8.64 ms 0.08 ms (0.88%) 1.00%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.00 ms 16.13 ms 0.13 ms (0.81%) 1.90%
handshake_tickets_ring_1.3_ecdsap256_aes 5.57 ms 5.61 ms 0.04 ms (0.76%) 1.94%
handshake_session_id_ring_1.3_ecdsap256_aes 5.50 ms 5.54 ms 0.04 ms (0.76%) 1.56%
handshake_tickets_ring_1.3_rsa_aes 6.06 ms 6.10 ms 0.04 ms (0.73%) 1.61%
handshake_tickets_ring_1.3_ecdsap256_chacha 5.54 ms 5.57 ms 0.04 ms (0.68%) 1.91%
transfer_no_resume_ring_1.3_ecdsap256_chacha 12.91 ms 13.00 ms 0.09 ms (0.66%) 2.41%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.64 ms 4.67 ms 0.03 ms (0.64%) 7.11%
handshake_session_id_ring_1.3_ecdsap256_chacha 5.47 ms 5.51 ms 0.03 ms (0.63%) 1.36%
handshake_tickets_ring_1.3_rsa_chacha 6.03 ms 6.06 ms 0.04 ms (0.61%) 1.41%
transfer_no_resume_ring_1.3_rsa_chacha 13.40 ms 13.48 ms 0.08 ms (0.59%) 2.50%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.29 ms 5.32 ms 0.03 ms (0.57%) 6.04%
handshake_session_id_ring_1.3_rsa_chacha 5.96 ms 6.00 ms 0.03 ms (0.56%) 1.75%
handshake_session_id_ring_1.3_rsa_aes 6.01 ms 6.04 ms 0.03 ms (0.54%) 1.45%
handshake_session_id_aws_lc_rs_1.2_rsa_aes 1.62 ms 1.62 ms 0.01 ms (0.52%) 3.28%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.34 ms 5.36 ms 0.02 ms (0.36%) 6.03%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 13.76 ms 13.80 ms 0.03 ms (0.25%) 2.27%
handshake_tickets_ring_1.2_rsa_aes 1.59 ms 1.59 ms -0.00 ms (-0.19%) 1.22%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 1.79 ms 1.78 ms -0.00 ms (-0.15%) 3.33%
handshake_no_resume_ring_1.2_rsa_aes 950.65 µs 951.98 µs 1.33 µs (0.14%) 1.14%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.80 ms 13.82 ms 0.02 ms (0.14%) 2.56%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 13.11 ms 13.13 ms 0.02 ms (0.14%) 2.54%
handshake_session_id_ring_1.2_rsa_aes 1.51 ms 1.51 ms -0.00 ms (-0.05%) 1.09%
handshake_no_resume_ring_1.3_rsa_chacha 954.14 µs 954.61 µs 0.47 µs (0.05%) 1.48%
handshake_no_resume_ring_1.3_rsa_aes 953.98 µs 954.17 µs 0.20 µs (0.02%) 1.46%

Additional information

Historical results

Checkout details:

@codecov Codecov
Copy link

codecov bot commented Jun 6, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.40%. Comparing base (19133f2) to head (c80f164).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #2481   +/-   ##
=======================================
  Coverage   95.40%   95.40%           
=======================================
  Files          97       97           
  Lines       21841    21841           
=======================================
  Hits        20838    20838           
  Misses       1003     1003

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ctz
Copy link
Member Author

ctz commented Jun 6, 2025

Here's what this looks like in our micro-benchmarks:

client/MLKEM768         time:   [21.755 µs 21.834 µs 21.925 µs]
                        thrpt:  [45.611 Kelem/s 45.801 Kelem/s 45.967 Kelem/s]
                 change:
                        time:   [−44.979% −44.041% −43.046%] (p = 0.00 < 0.05)
                        thrpt:  [+75.580% +78.702% +81.749%]
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe
client/X25519MLKEM768   time:   [27.589 µs 27.676 µs 27.774 µs]
                        thrpt:  [36.005 Kelem/s 36.132 Kelem/s 36.246 Kelem/s]
                 change:
                        time:   [−39.666% −38.646% −37.535%] (p = 0.00 < 0.05)
                        thrpt:  [+60.090% +62.988% +65.744%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
client/X25519           time:   [5.6183 µs 5.6355 µs 5.6545 µs]
                        thrpt:  [176.85 Kelem/s 177.45 Kelem/s 177.99 Kelem/s]
                 change:
                        time:   [−1.9556% −0.3234% +1.4194%] (p = 0.76 > 0.05)
                        thrpt:  [−1.3995% +0.3244% +1.9946%]
                        No change in performance detected.
Found 15 outliers among 100 measurements (15.00%)
  11 (11.00%) high mild
  4 (4.00%) high severe
client/X25519MLKEM768+X25519
                        time:   [33.110 µs 33.144 µs 33.191 µs]
                        thrpt:  [30.129 Kelem/s 30.171 Kelem/s 30.202 Kelem/s]
                 change:
                        time:   [−35.602% −34.563% −33.471%] (p = 0.00 < 0.05)
                        thrpt:  [+50.309% +52.818% +55.284%]
                        Performance has improved.
Found 19 outliers among 100 measurements (19.00%)
  2 (2.00%) high mild
  17 (17.00%) high severe

server/MLKEM768         time:   [21.108 µs 21.178 µs 21.259 µs]
                        thrpt:  [47.039 Kelem/s 47.218 Kelem/s 47.375 Kelem/s]
                 change:
                        time:   [−53.855% −53.052% −52.276%] (p = 0.00 < 0.05)
                        thrpt:  [+109.54% +113.00% +116.71%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
server/X25519MLKEM768   time:   [44.824 µs 44.957 µs 45.116 µs]
                        thrpt:  [22.165 Kelem/s 22.244 Kelem/s 22.310 Kelem/s]
                 change:
                        time:   [−35.919% −34.865% −33.697%] (p = 0.00 < 0.05)
                        thrpt:  [+50.822% +53.527% +56.053%]
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
server/X25519           time:   [23.402 µs 23.481 µs 23.573 µs]
                        thrpt:  [42.421 Kelem/s 42.588 Kelem/s 42.732 Kelem/s]
                 change:
                        time:   [−2.3597% −0.6426% +1.1642%] (p = 0.52 > 0.05)
                        thrpt:  [−1.1508% +0.6468% +2.4167%]
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

@ctz ctz added this pull request to the merge queue Jun 9, 2025
Merged via the queue into main with commit 3cdc0b0 Jun 9, 2025
34 checks passed
@ctz ctz deleted the jbp-take-semver-updates-2 branch June 9, 2025 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@djc djc djc approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ctz @djc