Skip to content

Conversation

UdjinM6
Copy link

@UdjinM6 UdjinM6 commented Feb 12, 2018

Add DSHA256 and X11 benchmarks. Also refactor names of other algo benchmarks to group them together.

DSHA256 and X11 also have additional tests for data from 32 to 2048 bytes (for comparison, in steps).

Looks like this now:

HASH_DSHA256,224,0.004438042640686,0.006658911705017,0.004743687808514,10224691,15343074,10929264
HASH_DSHA256_0032b,2,0.658596992492676,0.658596992492676,0.658596992492676,1517379473,1517379473,1517379473
HASH_DSHA256_0032b_single,1572864,0.000000618980266,0.000000720610842,0.000000653125123,1426,1660,1504
HASH_DSHA256_0080b_single,1048576,0.000000948086381,0.000001071261067,0.000000992204605,2184,2468,2285
HASH_DSHA256_0128b_single,851968,0.000001185868314,0.000001370561222,0.000001246062031,2732,3157,2870
HASH_DSHA256_0512b_single,327680,0.000002957531251,0.000003447115887,0.000003106717486,6814,7942,7157
HASH_DSHA256_1024b_single,196608,0.000005231195246,0.000005721551133,0.000005449997843,12052,13182,12556
HASH_DSHA256_2048b_single,98304,0.000009891489753,0.000011300900951,0.000010374829192,22789,26034,23903
HASH_RIPEMD160,384,0.002567186951637,0.002986758947372,0.002696075787147,5914708,6881347,6211646
HASH_SHA1,576,0.001737110316753,0.002056375145912,0.001831571261088,4002219,4737889,4219863
HASH_SHA256,224,0.004442453384399,0.005187511444092,0.004694205309664,10235832,11951745,10815254
HASH_SHA256_0032b,4,0.328065514564514,0.331060528755188,0.329563021659851,755848191,762750730,759299460
HASH_SHA512,384,0.002631470561028,0.003067433834076,0.002789856866002,6062805,7067288,6427714
HASH_SipHash_0032b,30,0.033042430877686,0.037223458290100,0.034758965174357,76128290,85760828,80083281
HASH_X11,120,0.008013010025024,0.009173631668091,0.008706633249919,18461995,21135490,20059719
HASH_X11_0032b_single,20480,0.000051029841416,0.000058016157709,0.000053492863663,117569,133667,123245
HASH_X11_0080b_single,18432,0.000050947302952,0.000059785670601,0.000054728304450,117380,137742,126091
HASH_X11_0128b_single,18432,0.000053015653975,0.000060724560171,0.000054999991941,122145,139905,126717
HASH_X11_0512b_single,18432,0.000055402866565,0.000063250074163,0.000058378907852,127646,145725,134502
HASH_X11_1024b_single,16384,0.000059022568166,0.000065412139520,0.000061924380134,135985,150708,142671
HASH_X11_2048b_single,14336,0.000067298766226,0.000086345709860,0.000071261510519,155052,198937,164183

EDIT: closes #1806 as was reminded below :)

…ks to group them together

DSHA256 and X11 also have additional tests for data from 32 to 2048 bytes (for comparison, in steps)
@UdjinM6 UdjinM6 added this to the 12.3 milestone Feb 12, 2018
Copy link

@codablock codablock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK
Also closes: #1806

@UdjinM6 UdjinM6 merged commit 0f4d963 into dashpay:develop Feb 12, 2018
andvgal pushed a commit to energicryptocurrency/gen2-energi that referenced this pull request Jan 6, 2019
…ks to group them together (dashpay#1925)

DSHA256 and X11 also have additional tests for data from 32 to 2048 bytes (for comparison, in steps)
CryptoCentric pushed a commit to absolute-community/absolute that referenced this pull request Feb 28, 2019
…ks to group them together (dashpay#1925)

DSHA256 and X11 also have additional tests for data from 32 to 2048 bytes (for comparison, in steps)
CryptoCentric pushed a commit to absolute-community/absolute that referenced this pull request Mar 2, 2019
…ks to group them together (dashpay#1925)

DSHA256 and X11 also have additional tests for data from 32 to 2048 bytes (for comparison, in steps)
@UdjinM6 UdjinM6 deleted the morecryptobenchmarks branch November 26, 2020 11:38
PastaPastaPasta added a commit that referenced this pull request Jul 17, 2025
, bitcoin#27598, bitcoin#29180, bitcoin#29407, bitcoin#29528, bitcoin#28893, bitcoin#29834, partial bitcoin#26158, bitcoin#25200, bitcoin#24322 (auxiliary backports: part 25)

2bb8e5b chore: resolve nit from review (Kittywhiskers Van Gogh)
61bba13 fix: tweak bls/bls_dkg benchmarks to work faster in `-sanity-check` mode (UdjinM6)
0eff28c merge bitcoin#29834: Change MAC_OSX macro to __APPLE__ in crypto (Kittywhiskers Van Gogh)
76cef17 merge bitcoin#28893: Fix SSE4.1-related issues (Kittywhiskers Van Gogh)
16209a8 merge bitcoin#29528: move sha256_sse4 into libbitcoin_crypto_base (Kittywhiskers Van Gogh)
ec1f492 merge bitcoin#29407: remove confusing and inconsistent disable-asm option (Kittywhiskers Van Gogh)
9440c18 merge bitcoin#29180: remove use of BUILD_BITCOIN_INTERNAL macro in sha256 (Kittywhiskers Van Gogh)
68f6bea fix: place `LIBTEST_*` before `LIBBITCOIN_*` for fuzzing binaries (Kittywhiskers Van Gogh)
a3c11b6 partial bitcoin#24322: Introduce initial `libbitcoinkernel` (Kittywhiskers Van Gogh)
9f981dd merge bitcoin#27598: Add SHA256 implementation specific benchmarks (Kittywhiskers Van Gogh)
306fa9a chore: realign function names in `crypto_hash` bench with upstream (Kittywhiskers Van Gogh)
3bdc15a chore: replace leftover `minEpochIterations` from `crypto_hash` benches (Kittywhiskers Van Gogh)
f300277 chore: drop `minEpochIterations(10)` from `crypto_hash` bench functions (Kittywhiskers Van Gogh)
c8f0d54 chore: move Dash-specific tests downward, sort like upstream (Kittywhiskers Van Gogh)
fda951f merge bitcoin#26481: Suppress output when running with `-sanity-check` option (Kittywhiskers Van Gogh)
a5718a0 partial bitcoin#25200: Fix spelling errors identified by codespell in comments (Kittywhiskers Van Gogh)
58a6248 partial bitcoin#26158: add "priority level" to the benchmark framework (Kittywhiskers Van Gogh)
160b3fc merge bitcoin#25107: Add `--sanity-check` flag, use it in `make check` (Kittywhiskers Van Gogh)
2ad60b3 fix: use atomic in `BLS_Verify_BatchedParallel` to avoid data race (Kittywhiskers Van Gogh)
24ad076 fix: ensure that globals are set and checks present to avoid bench errs (Kittywhiskers Van Gogh)
1c54997 trivial: re-enable benchmarks for `make check` (Kittywhiskers Van Gogh)
2a0df15 merge bitcoin#25058: Move output script RPCs to separate file, rename misc.cpp (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * `make check` upstream includes a dry run of the benchmarking suite, this behavior was introduced in [bitcoin#14092](bitcoin#14092) (a557022) but was disabled for Dash Core as our benchmarking suite is substantially heavier. Unfortunately, as a side effect of that, failures in the benchmark suite go unnoticed by CI (see e507a51 for one such fix).

    With the introduction of [bitcoin#25107](bitcoin#25107) and [bitcoin#26158](bitcoin#26158), the sanity checks require significantly less overhead and should be suitable for reintroduction into the `make check` rotation.

    * The sanity check caught failures in the test suite ([build](https://github.com/dashpay/dash/actions/runs/16291896463/job/46003746200#step:10:206)) due to missing initialization or insufficient checking and they have been remedied in this pull request. An example is available below.

      <details>

      <summary>Debugger stacktrace</summary>

      ```
      #0  StatsdClient::send<unsigned long> (this=0x0, key="CheckBlock_us", value=2847, type="ms", sample_rate=1) at stats/client.cpp:108
      #1  0x0000555555c3603f in StatsdClient::timing (this=0x0, key="CheckBlock_us", ms=2847, sample_rate=1) at stats/client.cpp:100
      #2  0x0000555555ca0b51 in CheckBlock (block=..., state=..., consensusParams=..., fCheckPOW=true, fCheckMerkleRoot=true) at validation.cpp:3932
      #3  0x000055555560cca5 in DeserializeAndCheckBlockTest(ankerl::nanobench::Bench&)::$_0::operator()() const (this=0x7fffffffd120) at bench/checkblock.cpp:47
      #4  ankerl::nanobench::Bench::run<DeserializeAndCheckBlockTest(ankerl::nanobench::Bench&)::$_0>(DeserializeAndCheckBlockTest(ankerl::nanobench::Bench&)::$_0&&) (this=0x7fffffffd430, op=...) at ./bench/nanobench.h:1221
      #5  0x000055555560c74b in DeserializeAndCheckBlockTest (bench=...) at bench/checkblock.cpp:40
      #6  0x00005555555e2f4a in std::function<void (ankerl::nanobench::Bench&)>::operator()(ankerl::nanobench::Bench&) const (this=0x555556942a90, __args=...) at /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:591
      #7  benchmark::BenchRunner::RunAll (args=...) at bench/bench.cpp:120
      #8  0x00005555555de5f4 in main (argc=<optimized out>, argv=<optimized out>) at bench/bench_bitcoin.cpp:134
      ```

      </details>

    * The benchmark `BLS_Verify_BatchedParallel` also has a data race that was spotted courtesy of `make check` ([build](https://github.com/dashpay/dash/actions/runs/16295305205/job/46015360773#step:10:183)) that has been resolved by using an atomic.

  * To allow backporting [bitcoin#27598](bitcoin#27598), the naming scheme changes in `src/bench/crypto_hash.cpp` introduced by [dash#1925](#1925) have been effectively reverted and Dash-specific benchmarks have been moved downward to mitigate merge conflict issues.
    * Similarly, changes to minimum iteration and batching by input size were made to realign with upstream.

  * The order of libraries for the linker had to be rearranged as the linker reads dependencies from left to right (i.e. top to bottom), which makes them ordering sensitive. Without this, the linker complains visibly (see below).

      <details>

      <summary>Linker errors:</summary>

      ```
      $ make -j10
      [...]
        AR       libbitcoin_node.a
        CXXLD    dashd
        CXXLD    test/test_dash
        CXXLD    qt/dash-qt
        CXXLD    test/fuzz/fuzz
        CXXLD    qt/test/test_dash-qt
        CXXLD    bench/bench_dash
      /usr/bin/ld: libtest_util.a(libtest_util_a-setup_common.o): in function `DashChainstateSetup(ChainstateManager&, node::NodeContext&, bool, bool, Consensus::Params const&)':
      /src/dash/dash/src/test/util/setup_common.cpp:128: undefined reference to `node::DashChainstateSetup(ChainstateManager&, CGovernanceManager&, CMasternodeMetaMan&, CMasternodeSync&, CSporkManager&, std::unique_ptr<CActiveMasternodeManager, std::default_delete<CActiveMasternodeManager> >&, std::unique_ptr<CChainstateHelper, std::default_delete<CChainstateHelper> >&, std::unique_ptr<CCreditPoolManager, std::default_delete<CCreditPoolManager> >&, std::unique_ptr<CDeterministicMNManager, std::default_delete<CDeterministicMNManager> >&, std::unique_ptr<CEvoDB, std::default_delete<CEvoDB> >&, std::unique_ptr<CMNHFManager, std::default_delete<CMNHFManager> >&, std::unique_ptr<LLMQContext, std::default_delete<LLMQContext> >&, CTxMemPool*, bool, bool, Consensus::Params const&)'
      /usr/bin/ld: libtest_util.a(libtest_util_a-setup_common.o): in function `DashChainstateSetupClose(node::NodeContext&)':
      /src/dash/dash/src/test/util/setup_common.cpp:136: undefined reference to `node::DashChainstateSetupClose(std::unique_ptr<CChainstateHelper, std::default_delete<CChainstateHelper> >&, std::unique_ptr<CCreditPoolManager, std::default_delete<CCreditPoolManager> >&, std::unique_ptr<CDeterministicMNManager, std::default_delete<CDeterministicMNManager> >&, std::unique_ptr<CMNHFManager, std::default_delete<CMNHFManager> >&, std::unique_ptr<LLMQContext, std::default_delete<LLMQContext> >&, CTxMemPool*)'
      /usr/bin/ld: libtest_util.a(libtest_util_a-setup_common.o): in function `BasicTestingSetup::BasicTestingSetup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&)':
      [...]
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      make[2]: *** [Makefile:7723: test/fuzz/fuzz] Error 1
      make[2]: *** Waiting for unfinished jobs....
      make[2]: Leaving directory '/src/dash/dash/src'
      make[1]: *** [Makefile:20430: all-recursive] Error 1
      make[1]: Leaving directory '/src/dash/dash/src'
      make: *** [Makefile:796: all-recursive] Error 1
      ```

     </details>

  ## Breaking Changes

  None expected.

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 2bb8e5b

Tree-SHA512: 1771f6af9d7a4da9f5417dc60fa0a69f7f1adb1c366ab458d9c2d572c2924ac03fd2c4b9e00cda8a63022cbc193b6b262349c5eab79654a1d0006e5e65327340
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants