Skip to content

Conversation

ctz
Copy link
Member

@ctz ctz commented Aug 14, 2024

from_crate_features() and get_default_or_install_from_crate_features() became uncovered since 3249a5b due to unwanted workspace feature unification.

`from_crate_features()` and `get_default_or_install_from_crate_features()`
became uncovered since 3249a5b due to unwanted workspace feature unification.
Copy link

rustls-benchmarking bot commented Aug 14, 2024

Benchmark results

Instruction counts

Significant differences

There are no significant instruction count differences

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 4022098 3926311 -95787 (-2.38%) 3.63%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 5138900 5085495 -53405 (-1.04%) 3.47%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 8897467 8853246 -44221 (-0.50%) 1.29%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 8890756 8856834 -33922 (-0.38%) 0.74%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 13367868 13418114 50246 (0.38%) 0.59%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 13752099 13789478 37379 (0.27%) 0.95%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 13803189 13774875 -28314 (-0.21%) 0.76%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 34770133 34715524 -54609 (-0.16%) 0.62%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 31097484 31146312 48828 (0.16%) 0.55%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 32935488 32984505 49017 (0.15%) 0.54%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 34749756 34798991 49235 (0.14%) 0.82%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46480665 46429625 -51040 (-0.11%) 0.60%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80564749 80646080 81331 (0.10%) 0.27%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 33040197 33008315 -31882 (-0.10%) 0.76%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 31142541 31172271 29730 (0.10%) 0.46%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 3381535 3383865 2330 (0.07%) 0.24%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 58209510 58248683 39173 (0.07%) 0.22%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 30722053 30741579 19526 (0.06%) 0.43%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46430047 46410188 -19859 (-0.04%) 0.27%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92690820 92720067 29247 (0.03%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 30702698 30708571 5873 (0.02%) 0.45%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 30743108 30737812 -5296 (-0.02%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 2136057 2136424 367 (0.02%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 2234997 2235335 338 (0.02%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 2951406 2951788 382 (0.01%) 0.20%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 2017239 2016986 -253 (-0.01%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 4026651 4026152 -499 (-0.01%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 31166809 31170242 3433 (0.01%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 41881064 41885547 4483 (0.01%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 1919651 1919846 195 (0.01%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_server 4704432 4704906 474 (0.01%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 30717329 30714262 -3067 (-0.01%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 43365538 43369458 3920 (0.01%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3917884 3918235 351 (0.01%) 0.23%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 2227940 2228131 191 (0.01%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_client 4284409 4284776 367 (0.01%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4442037 4442404 367 (0.01%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 2957397 2957616 219 (0.01%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_client 4557303 4557612 309 (0.01%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_server 4266578 4266315 -263 (-0.01%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 42410077 42407780 -2297 (-0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92698795 92703337 4542 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 3385717 3385561 -156 (-0.00%) 0.25%
handshake_session_id_ring_1.3_ecdsap256_aes_client 41958853 41960699 1846 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 34785723 34787210 1487 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 33000738 33002125 1387 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 58247050 58244604 -2446 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 42404956 42406573 1617 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 12185828 12186274 446 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 31146163 31145068 -1095 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 1916757 1916692 -65 (-0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 11991695 11992082 387 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3916803 3916677 -126 (-0.00%) 0.45%
handshake_session_id_ring_1.3_ecdsap384_aes_client 41957180 41958428 1248 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 43361095 43362280 1185 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 43956261 43955136 -1125 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 4393491 4393594 103 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92664399 92662289 -2110 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 13744165 13744473 308 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 4390120 4390217 97 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 42358461 42357547 -914 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 43959165 43958356 -809 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 41876977 41876225 -752 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 30758646 30759171 525 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 30734007 30733529 -478 (-0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 41974862 41974226 -636 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 31121442 31121001 -441 (-0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 43458549 43459126 577 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 43365215 43365773 558 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_server 44038528 44039066 538 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 41893628 41894127 499 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 44038832 44039314 482 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 32961163 32960811 -352 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 12179871 12179999 128 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 34786527 34786891 364 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 35476172 35475844 -328 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80508103 80507379 -724 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 34766808 34767111 303 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 31192251 31191980 -271 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58330682 58331133 451 (0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46368524 46368216 -308 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92660025 92660606 581 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 42345615 42345354 -261 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80516083 80515679 -404 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 32960136 32960301 165 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 44035504 44035724 220 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_client 42423204 42423413 209 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 68665842 68666161 319 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92705741 92705350 -391 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46428619 46428790 171 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46427248 46427078 -170 (-0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 2853844 2853834 -10 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 2134023 2134030 7 (0.00%) 0.77%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 13742250 13742294 44 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80606511 80606254 -257 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 43958258 43958388 130 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 43462911 43463039 128 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92664648 92664411 -237 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 33001887 33001805 -82 (-0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_client 58208841 58208984 143 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 35473699 35473786 87 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 43463505 43463414 -91 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46462605 46462510 -95 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 42340377 42340291 -86 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46454427 46454340 -87 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 58247738 58247823 85 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80608002 80607899 -103 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80504207 80504291 84 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58328340 58328395 55 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46451412 46451455 43 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58331780 58331767 -13 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 34767002 34767009 7 (0.00%) 0.20%

Wall-time

Significant differences

There are no significant wall-time differences

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.23 ms 5.26 ms 0.03 ms (0.58%) 2.80%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.51 ms 4.54 ms 0.02 ms (0.54%) 3.93%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 481.79 µs 484.36 µs 2.57 µs (0.53%) 2.81%
transfer_no_resume_ring_1.3_ecdsap256_aes 6.35 ms 6.38 ms 0.03 ms (0.50%) 2.74%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 483.08 µs 485.30 µs 2.22 µs (0.46%) 2.19%
handshake_no_resume_ring_1.3_ecdsap256_aes 508.73 µs 511.04 µs 2.31 µs (0.45%) 1.46%
transfer_no_resume_ring_1.3_rsa_aes 6.84 ms 6.87 ms 0.03 ms (0.44%) 2.63%
handshake_no_resume_ring_1.3_ecdsap256_chacha 506.20 µs 508.12 µs 1.92 µs (0.38%) 1.62%
transfer_no_resume_ring_1.3_ecdsap384_aes 9.44 ms 9.48 ms 0.04 ms (0.38%) 1.91%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.47 ms 5.49 ms 0.02 ms (0.35%) 3.67%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.60 ms 3.61 ms 0.01 ms (0.33%) 1.00%
handshake_session_id_ring_1.2_rsa_aes 1.57 ms 1.58 ms 0.00 ms (0.31%) 1.13%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.20 ms 1.20 ms 0.00 ms (0.26%) 1.45%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.48 ms 5.49 ms 0.01 ms (0.24%) 3.91%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.08 ms 16.12 ms 0.04 ms (0.23%) 1.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.60 ms 3.61 ms 0.01 ms (0.23%) 1.00%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.69 ms 13.73 ms 0.03 ms (0.23%) 1.33%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 12.97 ms 13.00 ms 0.03 ms (0.23%) 1.34%
handshake_no_resume_ring_1.2_rsa_aes 978.52 µs 980.68 µs 2.16 µs (0.22%) 1.01%
handshake_session_id_aws_lc_rs_1.2_rsa_aes 2.11 ms 2.11 ms 0.00 ms (0.22%) 4.56%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 2.31 ms 2.31 ms 0.00 ms (0.22%) 2.26%
transfer_no_resume_ring_1.2_rsa_aes 6.75 ms 6.76 ms 0.01 ms (0.21%) 2.08%
handshake_no_resume_ring_1.3_rsa_aes 1.00 ms 1.00 ms 0.00 ms (0.20%) 1.19%
transfer_no_resume_ring_1.3_ecdsap256_chacha 12.98 ms 13.01 ms 0.03 ms (0.20%) 1.41%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.20 ms 1.20 ms 0.00 ms (0.19%) 1.17%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 13.94 ms 13.96 ms 0.03 ms (0.18%) 1.55%
handshake_tickets_ring_1.3_ecdsap256_chacha 6.76 ms 6.75 ms -0.01 ms (-0.17%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.44 ms 1.44 ms 0.00 ms (0.17%) 6.10%
handshake_session_id_ring_1.3_ecdsap256_chacha 6.71 ms 6.70 ms -0.01 ms (-0.17%) 1.29%
handshake_tickets_ring_1.2_rsa_aes 1.66 ms 1.66 ms 0.00 ms (0.14%) 1.77%
transfer_no_resume_ring_1.3_rsa_chacha 13.49 ms 13.50 ms 0.02 ms (0.14%) 1.29%
handshake_no_resume_ring_1.3_rsa_chacha 1.00 ms 1.00 ms 0.00 ms (0.13%) 1.00%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 6.12 ms 6.11 ms -0.01 ms (-0.13%) 1.34%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 6.10 ms 6.09 ms -0.01 ms (-0.13%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_aes 9.83 ms 9.84 ms 0.01 ms (0.13%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.43 ms 1.43 ms 0.00 ms (0.13%) 6.65%
handshake_tickets_ring_1.3_ecdsap256_aes 6.81 ms 6.80 ms -0.01 ms (-0.11%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_aes 6.75 ms 6.75 ms -0.01 ms (-0.10%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 9.88 ms 9.89 ms 0.01 ms (0.08%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_chacha 9.80 ms 9.80 ms 0.01 ms (0.08%) 1.00%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 6.36 ms 6.35 ms -0.01 ms (-0.08%) 1.00%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 6.48 ms 6.49 ms 0.00 ms (0.08%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_chacha 9.84 ms 9.85 ms 0.01 ms (0.06%) 1.00%
handshake_session_id_ring_1.3_rsa_chacha 7.20 ms 7.20 ms 0.00 ms (0.06%) 1.09%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 6.38 ms 6.38 ms -0.00 ms (-0.06%) 1.00%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.39 ms 1.39 ms 0.00 ms (0.06%) 6.13%
handshake_session_id_ring_1.3_rsa_aes 7.24 ms 7.25 ms 0.00 ms (0.05%) 1.00%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 5.52 ms 5.52 ms -0.00 ms (-0.05%) 1.00%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 5.40 ms 5.40 ms -0.00 ms (-0.04%) 1.42%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 6.23 ms 6.23 ms -0.00 ms (-0.04%) 1.00%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 6.23 ms 6.23 ms 0.00 ms (0.04%) 1.00%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 5.39 ms 5.38 ms -0.00 ms (-0.02%) 1.28%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 5.51 ms 5.51 ms 0.00 ms (0.02%) 1.04%
handshake_tickets_ring_1.3_rsa_chacha 7.25 ms 7.24 ms -0.00 ms (-0.01%) 1.00%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 6.49 ms 6.49 ms -0.00 ms (-0.01%) 1.00%
handshake_tickets_ring_1.3_rsa_aes 7.29 ms 7.29 ms 0.00 ms (0.00%) 1.00%

Additional information

Historical results

Checkout details:

Copy link

codecov bot commented Aug 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.78%. Comparing base (93d9aa6) to head (c474a36).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2083      +/-   ##
==========================================
+ Coverage   94.52%   94.78%   +0.25%     
==========================================
  Files         102      102              
  Lines       23465    23489      +24     
==========================================
+ Hits        22180    22263      +83     
+ Misses       1285     1226      -59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@djc djc added this pull request to the merge queue Aug 19, 2024
Merged via the queue into main with commit f17866c Aug 19, 2024
46 checks passed
@djc djc deleted the jbp-reinstate-provider-selection-tests branch August 19, 2024 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants