Overview of slow unit tests #10026
Description
At last IRC meeting we've discussed trying to speed up the unit tests, as they've become much too slow.
For this purpose I've run test_bitcoin --log_level=test_suite and picked out the slowest from the list.
The top 20 is (updated 2018-04-14 by @lucash-dev):
| Test | Time (μs) |
|---|---|
test_big_witness_transaction |
39133014 |
test_CheckQueue_Correct_Random |
14808019 |
knapsack_solver_test |
9304171 |
checkinputs_test |
6370348 |
CreateNewBlock_validity |
6105558 |
test_CheckQueue_Memory |
4230330 |
versionbits_test |
3835090 |
coins_cache_simulation_test |
3205869 |
updatecoins_simulation_test |
2499840 |
merkle_test |
2391688 |
rescan |
1900315 |
SelectCoins_test |
1731901 |
passphrase |
1730167 |
ListCoins |
1689793 |
test_CheckQueue_Catches_Failure |
1563869 |
test_CheckQueue_UniqueCheck |
1521864 |
PrevectorTestInt |
1409636 |
test_CheckQueue_Correct_Max |
1401687 |
bnb_search_test |
1147974 |
cuckoocache_generations |
978637 |
I think (open for discussion, of course) it is unreasonable for a unit test case to take more than ~1s (mind you, we have 242 of them, so each test is supposed to be quick) so we should take a look at each of these up to merkle_test.
It could be as simple in some cases as getting rid of (reasonably heavy) BOOST_* tests in inner loops, moving the checks up to a higher level.
Edit: I added the script to create this table to maintainer-tools: https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/unittest-statistics.py