contrib/signet/miner: grind will fail for high difficulty chain #30102
Description
Is there an existing issue for this?
- I have searched the existing issues
Current behaviour
Mining will fail with a Could not satisfy difficulty target.
❯ ~/2-development/bitcoin/bitcoin-core/contrib/signet/miner --cli "/Users/jose.edil/2-development/bitcoin/bitcoin-core/src/bitcoin-cli -datadir=/Users/jose.edil/2-development/bitcoin/signet-mining-experiments/signet-fix-hashing" generate --address tb1qylfujt900rjxzfxjj7sktpu7dpm2n9j60ch7jt --grind-cmd "/Users/jose.edil/2-development/bitcoin/bitcoin-core/src/bitcoin-util grind" --nbits 1d008d28 --ongoing
2024-05-14 16:29:05 INFO Mined block at height 10079; next in -324h56m20s (mine)
2024-05-14 16:31:22 INFO Mined block at height 10080; next in -324h56m7s (mine)
2024-05-14 16:32:34 INFO Mined block at height 10081; next in -324h54m49s (mine)
Could not satisfy difficulty target
Traceback (most recent call last):
File "/Users/jose.edil/2-development/bitcoin/bitcoin-core/contrib/signet/miner", line 545, in <module>
main()
File "/Users/jose.edil/2-development/bitcoin/bitcoin-core/contrib/signet/miner", line 539, in main
return args.fn(args)
^^^^^^^^^^^^^
File "/Users/jose.edil/2-development/bitcoin/bitcoin-core/contrib/signet/miner", line 418, in do_generate
block = finish_block(block, signet_solution, args.grind_cmd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jose.edil/2-development/bitcoin/bitcoin-core/contrib/signet/miner", line 107, in finish_block
newheadhex = subprocess.run(cmd, stdout=subprocess.PIPE, input=b"", check=True).stdout.strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/jose.edil/2-development/bitcoin/bitcoin-core/src/bitcoin-util', 'grind', '0000002063d19ae45a71b26fc04ddeef152919d2ecc22ed936ea7129d3bf291f03000000703be9fb35f03fc17b9b3210ac8364c1b76888d793397b7686128107d21c0b4a39e33166ac77031d00000000']' returned non-zero exit status 1.This is a follow-up of the discussion in #30091: I started a signet with difficulty --nbits 1d008d28 and mined the first block with a date 30 days in the past. That would let the miner run as fast as it can until difficulty is so high it will converge to the 10 minute average time between blocks.
After each 2016 blocks, the difficulty is adjusted as per the network consensus and it gets increasingly harder to find new blocks. After 5 adjustments, the grinder eventually exhausts the nonce search space and fails, making the python script to fail as well.
I inspected the bitcoin-util grinder source, but could not find any problem there. Indeed, it does what one would expect: look for PoW and fail if the nonce search space is exhausted. That's where the stdout message comes.
Expected behaviour
I tracked down the problem to this section of the code.
The finish_block function should be able to catch the exception that can arise from the grinder subprocess and in case of failure try another block header. In case of signet, I believe it is a matter of resigning it as in and looking for PoW again until it finds something.
I started a fix to submit a PR, but wanted to see if someone else could reproduce this behavior in the meantime. It's not clear to me how to make a (unit) test to systematically expose the problem, though.
Steps to reproduce
Follow the outline discussed in #30091 and let the miner run (for a few hours) until the difficulty gets high enough so that one needs a bigger search space to find valid PoW for new blocks.
Relevant log output
Current chain info
❯ ~/2-development/bitcoin/bitcoin-core/src/bitcoin-cli -datadir="/Users/jose.edil/2-development/bitcoin/signet-mining-experiments/signet-fix-hashing" -signet getblockchaininfo
{
"chain": "signet",
"blocks": 10081,
"headers": 10081,
"bestblockhash": "000000031f29bfd32971ea36d92ec2ecd2192915efde4dc06fb2715ae49ad163",
"difficulty": 0.2883904525532027,
"time": 1714545315,
"mediantime": 1714544565,
"verificationprogress": 1,
"initialblockdownload": true,
"chainwork": "000000000000000000000000000000000000000000000000000000c3e464c5fc",
"size_on_disk": 4153462,
"pruned": false,
"warnings": ""
}How did you obtain Bitcoin Core
Compiled from source
What version of Bitcoin Core are you using?
v27.0
Operating system and version
MacOS Sonoma 14.4
Machine specifications
Macbook Pro M3 Pro
18GB Memory