Skip to content

👨🏽‍🔬 Improve and standardize nps reporting #1081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Oct 6, 2024
Merged

👨🏽‍🔬 Improve and standardize nps reporting #1081

merged 6 commits into from
Oct 6, 2024

Conversation

eduherminio
Copy link
Member

@eduherminio eduherminio commented Oct 6, 2024 •
edited
Loading

bench command output is significantly altered by this change, reporting higher nodes and nps.
In general, search nps reported are also higher and kinda more aligned with the expected values of an engine with Lynx's strength.

The change is non-functional.

Output comparison of `go depth 25` command

v1.7.0

> dotnet run -c Release "go depth 25"
Lynx 1.7.0 by Eduardo Caceres
info depth 1 seldepth 1 multipv 1 score cp 75 nodes 20 nps 20 time 0 pv g1f3
info depth 2 seldepth 4 multipv 1 score cp 0 nodes 208 nps 208 time 0 pv g1f3 g8f6
info depth 3 seldepth 4 multipv 1 score cp 59 nodes 229 nps 229 time 1 pv d2d4 g8f6 g1f3
info depth 4 seldepth 8 multipv 1 score cp 0 nodes 564 nps 562 time 4 pv e2e3 g8f6 b1c3 e7e6
info depth 5 seldepth 10 multipv 1 score cp 55 nodes 1399 nps 1385 time 10 pv b1c3 d7d5 d2d4 g8f6 e2e3
info depth 6 seldepth 15 multipv 1 score cp 17 nodes 5655 nps 5458 time 36 pv e2e4 e7e5 g1f3 g8f6 d2d4 d7d5 d4e5
info depth 7 seldepth 15 multipv 1 score cp 53 nodes 4459 nps 4211 time 59 pv g1f3 d7d5 e2e3 b8c6 d2d4 e7e6 b1c3
info depth 8 seldepth 15 multipv 1 score cp 0 nodes 10558 nps 9393 time 124 pv b1c3 d7d5 g1f3 g8f6 e2e3 b8c6 d2d4 e7e6
info depth 9 seldepth 21 multipv 1 score cp 47 nodes 26614 nps 21797 time 221 pv e2e4 d7d5 e4e5 c7c5 d2d4 b8c6 c2c3 e7e6 f1b5 c5d4 c3d4
info depth 10 seldepth 22 multipv 1 score cp 19 nodes 54632 nps 42416 time 288 pv g1f3 g8f6 c2c4 c7c6 e2e3 e7e6 f1d3 d7d5 c4d5 e6d5 e1g1
info depth 11 seldepth 24 multipv 1 score cp 49 nodes 60661 nps 44702 time 357 pv e2e4 e7e5 b1c3 b8c6 f1c4 f8c5 g1f3 d7d6 d2d3 a7a6 a2a3
info depth 12 seldepth 24 multipv 1 score cp 42 nodes 64571 nps 45633 time 415 pv e2e4 e7e6 g1f3 d7d5 e4e5 c7c5 d2d4 c5d4 f1b5 c8d7 f3d4 a7a6 b5d3
info depth 13 seldepth 24 multipv 1 score cp 50 nodes 75778 nps 50688 time 495 pv e2e4 e7e6 g1f3 d7d5 e4e5 c7c5 c2c3 b8c6 d2d4 c5d4 c3d4 f8b4 b1c3 a7a6
info depth 14 seldepth 25 multipv 1 score cp 63 nodes 141760 nps 88545 time 601 pv e2e4 e7e6 d2d4 d7d5 e4e5 c7c5 c2c3 b8c6 f1e2 f8e7 g1f3 d8c7 d1c2 a7a6
info depth 15 seldepth 29 multipv 1 score cp 70 nodes 308644 nps 168198 time 835 pv e2e4 e7e6 d2d4 d7d5 e4e5 c7c5 c2c3 b8c6 f1e2 a7a6 g1f3 d8c7 d1c2 c5d4 c3d4
info depth 16 seldepth 29 multipv 1 score cp 58 nodes 432734 nps 202781 time 1134 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 b1c3 f8c5 d2d3 h7h6 a2a3 c5b6 c1e3 d7d6 d3d4 c8e6 c4e6 f7e6 d4e5 c6e5 e3b6 a7b6 f3e5 d6e5
info depth 17 seldepth 29 multipv 1 score cp 66 nodes 313295 nps 132752 time 1360 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 b1c3 f8c5 d2d3 h7h6 a2a3 c5b6 c1e3 d7d6 d3d4 c8g4 d4d5 b6e3 f2e3
info depth 18 seldepth 33 multipv 1 score cp 22 nodes 3962912 nps 757293 time 4233 pv g1f3 g8f6 d2d4 d7d5 c1f4 e7e6 e2e3 c7c5 c2c3 b8c6 b1d2 h7h6 d1c2 c5d4 c3d4 f8d6 f4d6 d8d6
info depth 19 seldepth 36 multipv 1 score cp 26 nodes 2190745 nps 320800 time 5829 pv g1f3 g8f6 d2d4 d7d5 c1f4 c8f5 e2e3 e7e6 f1d3 f6e4 b1d2 b8d7 f3e5 f8d6 e1g1 h7h6 c2c4 e4d2 d1d2 d6e5 d4e5 f5d3 d2d3 d5c4 d3c4 e8g8
info depth 20 seldepth 36 multipv 1 score cp 36 nodes 6640308 nps 576416 time 10520 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8b4 e2e3 h7h6 g5f6 d8f6 g1f3 c7c5 c4d5 e6d5 f1d3 c5c4 d3c2 b8c6 e1g1 b4c3 b2c3 e8g8
info depth 21 seldepth 36 multipv 1 score cp 31 nodes 3875017 nps 275312 time 13075 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8b4 c4d5 e6d5 g1f3 b4c3 b2c3 h7h6 g5f6 d8f6 e2e3 b8c6 d1b3 f6g6 b3d5 e8g8
info depth 22 seldepth 37 multipv 1 score cp 24 nodes 8692113 nps 437956 time 18847 pv g1f3 g8f6 e2e3 e7e6 d2d4 c7c5 c2c4 d7d5 b1c3 b8c6 c4d5 e6d5 f1d3 c5c4 d3c2 f8d6 a2a3 a7a6 h2h3 h7h6 d1e2 c8e6
info depth 23 seldepth 41 multipv 1 score cp 26 nodes 9019876 nps 346838 time 25006 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 d2d3 f8c5 c2c3 c5b6 b2b4 d7d6 a2a4 a7a5 b4b5 c6e7 c1g5 c7c6 g5f6 g7f6 b5c6 d6d5 e4d5 e7d5 c6b7 c8b7 e1g1 e8g8
info depth 24 seldepth 41 multipv 1 score cp 26 nodes 6413828 nps 210538 time 29464 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 d2d3 f8c5 c2c3 c5b6 b2b4 d7d6 a2a4 a7a5 b4b5 c6e7 c1g5 c7c6 g5f6 g7f6 b5c6 d6d5 e4d5
info depth 25 seldepth 41 multipv 1 score cp 32 nodes 19769734 nps 445294 time 43397 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 f8d6 c1e3 h8e8 f2f3 f6d5 e3d4 d5c3 d4c3 f7f6 g2g3 d8e7 a2a3 c8e6 e1c1
info depth 25 seldepth 41 multipv 1 score cp 32 nodes 19769734 nps 445294 time 43397 hashfull 564 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 f8d6 c1e3 h8e8 f2f3 f6d5 e3d4 d5c3 d4c3 f7f6 g2g3 d8e7 a2a3 c8e6 e1c1
bestmove e2e4 ponder e7e5

this branch:

> dotnet run -c Release "go depth 25"
Lynx 1.7.0 by Eduardo Caceres
info depth 1 seldepth 1 multipv 1 score cp 75 nodes 20 nps 266667 time 1 pv g1f3
info depth 2 seldepth 4 multipv 1 score cp 0 nodes 228 nps 374016 time 1 pv g1f3 g8f6
info depth 3 seldepth 4 multipv 1 score cp 59 nodes 457 nps 320432 time 1 pv d2d4 g8f6 g1f3
info depth 4 seldepth 8 multipv 1 score cp 0 nodes 1021 nps 242201 time 4 pv e2e3 g8f6 b1c3 e7e6
info depth 5 seldepth 10 multipv 1 score cp 55 nodes 2420 nps 202974 time 12 pv b1c3 d7d5 d2d4 g8f6 e2e3
info depth 6 seldepth 15 multipv 1 score cp 17 nodes 8075 nps 181107 time 45 pv e2e4 e7e5 g1f3 g8f6 d2d4 d7d5 d4e5
info depth 7 seldepth 15 multipv 1 score cp 53 nodes 12534 nps 174078 time 72 pv g1f3 d7d5 e2e3 b8c6 d2d4 e7e6 b1c3
info depth 8 seldepth 15 multipv 1 score cp 0 nodes 23092 nps 164847 time 140 pv b1c3 d7d5 g1f3 g8f6 e2e3 b8c6 d2d4 e7e6
info depth 9 seldepth 21 multipv 1 score cp 47 nodes 49706 nps 166555 time 298 pv e2e4 d7d5 e4e5 c7c5 d2d4 b8c6 c2c3 e7e6 f1b5 c5d4 c3d4
info depth 10 seldepth 22 multipv 1 score cp 19 nodes 104338 nps 287983 time 362 pv g1f3 g8f6 c2c4 c7c6 e2e3 e7e6 f1d3 d7d5 c4d5 e6d5 e1g1
info depth 11 seldepth 24 multipv 1 score cp 49 nodes 164999 nps 409622 time 403 pv e2e4 e7e5 b1c3 b8c6 f1c4 f8c5 g1f3 d7d6 d2d3 a7a6 a2a3
info depth 12 seldepth 24 multipv 1 score cp 42 nodes 229570 nps 509512 time 451 pv e2e4 e7e6 g1f3 d7d5 e4e5 c7c5 d2d4 c5d4 f1b5 c8d7 f3d4 a7a6 b5d3
info depth 13 seldepth 24 multipv 1 score cp 50 nodes 305348 nps 602386 time 507 pv e2e4 e7e6 g1f3 d7d5 e4e5 c7c5 c2c3 b8c6 d2d4 c5d4 c3d4 f8b4 b1c3 a7a6
info depth 14 seldepth 25 multipv 1 score cp 63 nodes 447108 nps 742927 time 602 pv e2e4 e7e6 d2d4 d7d5 e4e5 c7c5 c2c3 b8c6 f1e2 f8e7 g1f3 d8c7 d1c2 a7a6
info depth 15 seldepth 29 multipv 1 score cp 70 nodes 755752 nps 897728 time 842 pv e2e4 e7e6 d2d4 d7d5 e4e5 c7c5 c2c3 b8c6 f1e2 a7a6 g1f3 d8c7 d1c2 c5d4 c3d4
info depth 16 seldepth 29 multipv 1 score cp 58 nodes 1188486 nps 1023190 time 1162 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 b1c3 f8c5 d2d3 h7h6 a2a3 c5b6 c1e3 d7d6 d3d4 c8e6 c4e6 f7e6 d4e5 c6e5 e3b6 a7b6 f3e5 d6e5
info depth 17 seldepth 29 multipv 1 score cp 66 nodes 1501781 nps 1075567 time 1396 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 b1c3 f8c5 d2d3 h7h6 a2a3 c5b6 c1e3 d7d6 d3d4 c8g4 d4d5 b6e3 f2e3
info depth 18 seldepth 33 multipv 1 score cp 22 nodes 5464693 nps 1242818 time 4397 pv g1f3 g8f6 d2d4 d7d5 c1f4 e7e6 e2e3 c7c5 c2c3 b8c6 b1d2 h7h6 d1c2 c5d4 c3d4 f8d6 f4d6 d8d6
info depth 19 seldepth 36 multipv 1 score cp 26 nodes 7655438 nps 1280950 time 5976 pv g1f3 g8f6 d2d4 d7d5 c1f4 c8f5 e2e3 e7e6 f1d3 f6e4 b1d2 b8d7 f3e5 f8d6 e1g1 h7h6 c2c4 e4d2 d1d2 d6e5 d4e5 f5d3 d2d3 d5c4 d3c4 e8g8
info depth 20 seldepth 36 multipv 1 score cp 36 nodes 14295746 nps 1326672 time 10776 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8b4 e2e3 h7h6 g5f6 d8f6 g1f3 c7c5 c4d5 e6d5 f1d3 c5c4 d3c2 b8c6 e1g1 b4c3 b2c3 e8g8
info depth 21 seldepth 36 multipv 1 score cp 31 nodes 18170763 nps 1334060 time 13621 pv d2d4 d7d5 c2c4 e7e6 b1c3 g8f6 c1g5 f8b4 c4d5 e6d5 g1f3 b4c3 b2c3 h7h6 g5f6 d8f6 e2e3 b8c6 d1b3 f6g6 b3d5 e8g8
info depth 22 seldepth 37 multipv 1 score cp 24 nodes 26862876 nps 1367461 time 19644 pv g1f3 g8f6 e2e3 e7e6 d2d4 c7c5 c2c4 d7d5 b1c3 b8c6 c4d5 e6d5 f1d3 c5c4 d3c2 f8d6 a2a3 a7a6 h2h3 h7h6 d1e2 c8e6
info depth 23 seldepth 41 multipv 1 score cp 26 nodes 35882752 nps 1379216 time 26017 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 d2d3 f8c5 c2c3 c5b6 b2b4 d7d6 a2a4 a7a5 b4b5 c6e7 c1g5 c7c6 g5f6 g7f6 b5c6 d6d5 e4d5 e7d5 c6b7 c8b7 e1g1 e8g8
info depth 24 seldepth 41 multipv 1 score cp 26 nodes 42296580 nps 1383007 time 30583 pv e2e4 e7e5 g1f3 b8c6 f1c4 g8f6 d2d3 f8c5 c2c3 c5b6 b2b4 d7d6 a2a4 a7a5 b4b5 c6e7 c1g5 c7c6 g5f6 g7f6 b5c6 d6d5 e4d5
info depth 25 seldepth 41 multipv 1 score cp 32 nodes 62066314 nps 1387368 time 44737 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 f8d6 c1e3 h8e8 f2f3 f6d5 e3d4 d5c3 d4c3 f7f6 g2g3 d8e7 a2a3 c8e6 e1c1
info depth 25 seldepth 41 multipv 1 score cp 32 nodes 62066314 nps 1387367 time 44737 hashfull 564 pv e2e4 e7e5 g1f3 b8c6 d2d4 e5d4 f3d4 g8f6 d4c6 d7c6 d1d8 e8d8 b1c3 f8d6 c1e3 h8e8 f2f3 f6d5 e3d4 d5c3 d4c3 f7f6 g2g3 d8e7 a2a3 c8e6 e1c1
bestmove e2e4 ponder e7e5

@eduherminio eduherminio changed the title 👨🏽‍🔬 Improve NPS reporting 👨🏽‍🔬 Improve and standardize nps reporting Oct 6, 2024
@eduherminio eduherminio marked this pull request as ready for review October 6, 2024 16:50
@eduherminio eduherminio merged commit f397ef8 into main Oct 6, 2024
27 checks passed
@eduherminio eduherminio deleted the refactor/nodes-count branch October 6, 2024 17:41
@eduherminio
Copy link
Member Author

Providing there are no flagrant bugs that force to to release a patch, this change will be shipped with next 'regular' minor release, whenever that's ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@eduherminio