Skip to content

Conversation

hebasto
Copy link
Member

@hebasto hebasto commented Aug 13, 2020

Split out from #18710.

Some results (borrowed from #18710):
89121718-a3329800-d4c1-11ea-8bd1-66da20619696

@hebasto
Copy link
Member Author

hebasto commented Aug 13, 2020

cc @martinus @JeremyRubin

@JeremyRubin
Copy link
Contributor

Hmmmm...

I'd maybe rather just abort the test and return early if only 1 thread gets made because we shouldn't ever be running with the checkqueue on a single core machine...

@DrahtBot DrahtBot added the Tests label Aug 13, 2020
@hebasto
Copy link
Member Author

hebasto commented Aug 13, 2020

@JeremyRubin

Hmmmm...

I'd maybe rather just abort the test and return early if only 1 thread gets made because we shouldn't ever be running with the checkqueue on a single core machine...

We do not know for sure who many CPU cores/threads users dedicate to Bitcoin Core :)

With this PR benchmark results are quite consistent:

$ taskset --cpu-list 0-7 src/bench/bench_bitcoin -filter=CCheckQueueSpeedPrevectorJob
|              ns/job |               job/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|              177.62 |        5,629,902.65 |    1.9% |      0.01 | `CCheckQueueSpeedPrevectorJob`
$ taskset --cpu-list 0-3 src/bench/bench_bitcoin -filter=CCheckQueueSpeedPrevectorJob
|              ns/job |               job/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|              176.05 |        5,680,327.83 |    3.3% |      0.01 | `CCheckQueueSpeedPrevectorJob`
$ taskset --cpu-list 0-1 src/bench/bench_bitcoin -filter=CCheckQueueSpeedPrevectorJob
|              ns/job |               job/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|               55.45 |       18,034,367.56 |    2.6% |      0.00 | `CCheckQueueSpeedPrevectorJob`
$ taskset --cpu-list 0 src/bench/bench_bitcoin -filter=CCheckQueueSpeedPrevectorJob
|              ns/job |               job/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|               54.73 |       18,271,118.07 |    4.0% |      0.00 | `CCheckQueueSpeedPrevectorJob`

And early return do not disable the test, right?

@laanwj
Copy link
Member

laanwj commented Aug 14, 2020

If it's a benchmark that benchmarks thread synchronization/interaction (which is, I think, the only valid reason to run a benchmark multi-threaded) then I agree running it without threads makes little sense.

(And if so you could even see the "two threads on a single-core machine" case as a realistic case, because bitcoind will create threads on such a machine)

Co-authored-by: Martin Ankerl <Martin.Ankerl@gmail.com>
@hebasto
Copy link
Member Author

hebasto commented Aug 14, 2020

Updated d786765 -> a65c1ad (pr19710.01 -> pr19710.02, diff).

Addressed:

Hmmmm...

I'd maybe rather just abort the test and return early if only 1 thread gets made because we shouldn't ever be running with the checkqueue on a single core machine...

If it's a benchmark that benchmarks thread synchronization/interaction (which is, I think, the only valid reason to run a benchmark multi-threaded) then I agree running it without threads makes little sense.

@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 20, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@maflcko
Copy link
Member

maflcko commented Aug 21, 2020

cr ACK a65c1ad

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK a65c1ad

@hebasto
Copy link
Member Author

hebasto commented Aug 25, 2020

Updated a65c1ad -> 4f91643 (pr19710.02 -> pr19710.03, diff).

Addressed @vasild's comments:

Also, maybe don't drop the MIN_CORES constant and use it here instead of the magic number "2"?

Maybe also comment here "why we do - 1".

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

@hebasto
Copy link
Member Author

hebasto commented Aug 25, 2020

Updated 4f91643 -> 5e7acd6 (pr19710.03 -> pr19710.04, diff).

Addressed @promag's comment:

nit, single line and also add comment.

@promag
Copy link
Contributor

promag commented Aug 25, 2020

Code review ACK 5e7acd6.

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 5e7acd6

Thanks for the comments!

@hebasto
Copy link
Member Author

hebasto commented Aug 26, 2020

@MarcoFalke Mind having another look at this PR?

This change decreases the variance of benchmark results.
@hebasto
Copy link
Member Author

hebasto commented Aug 26, 2020

Updated 5e7acd6 -> 3edc4e3 (pr19710.04 -> pr19710.05, diff).

Addressed @promag's comment:

Maybe ditch MIN_CORES and change this to GetNumCores() == 1?

@fjahr
Copy link
Contributor

fjahr commented Aug 30, 2020

Code review ACK 3edc4e3

@maflcko maflcko merged commit afffbb1 into bitcoin:master Aug 31, 2020
@hebasto hebasto deleted the 200813-var branch August 31, 2020 07:59
@promag
Copy link
Contributor

promag commented Aug 31, 2020

ACK

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Aug 31, 2020
…ases the variance of result values

3edc4e3 bench: Prevent thread oversubscription (Hennadii Stepanov)
ce3e6a7 bench: Allow skip benchmark (Hennadii Stepanov)

Pull request description:

  Split out from bitcoin#18710.

  Some results (borrowed from bitcoin#18710):
  ![89121718-a3329800-d4c1-11ea-8bd1-66da20619696](https://user-images.githubusercontent.com/32963518/90146614-ecb89800-dd89-11ea-80fe-bac0e46e735e.png)

ACKs for top commit:
  fjahr:
    Code review ACK 3edc4e3

Tree-SHA512: df7413ec9ea326564a8e8de54752c9d1444ff7de34edb03e1e0c2120fc333e4640767fdbe3e87eab6a7b389a4863c02e22ad2ae0dbf139fad6a9b85e00f563b4
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants