miner: Reorg Testnet4 minimum difficulty blocks #31117
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31117. ReviewsSee the guideline for information on the review process. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
|
CI is broken because BIP94 is also active on regtest and this change breaks some functional tests that are not related to the issue. I don't think I need to work on fixing those tests unless I get an indication that people think this is actually interesting to merge. We are getting some report that the |
f1f9d54 to
7864871
Compare
|
In addition to the immediate minimum difficulty blocks after each full difficulty block, someone appears to have started bloating the UTXO set. They have created about 1.5 M UTXOs today so far: https://mempool.space/testnet4/address/tb1q8z0jcrdmfuhyafz06ukhyvfnx4vhcn5azv6jkr |
Well, the solution to that is quite simple: if non-standard transactions would be normally propagated, then developers could simply publish them, and have them confirmed. Unless you want to get an additional spam protection, by requiring every developer to provide 32 leading zero bits in Proof of Work. But in that case, just doing OP_SIZE on a signature will give you the same outcome.
Note that testnet4 is already traded at 20 satoshis per 1 tBTC4. And testnet5 will also be, if it will be released in a similar way: https://altquick.com/exchange/market/BitcoinTestnet4
When coins get non-zero value, then people are simply trying to get the maximum number of blocks, and sell them. And the original algorithm from Bitcoin Core is far from optimal. If you pick future timestamps, then you can for example instantly mine as many CPU blocks as possible, and then put some ASIC block on top of that. Which means, that staying 2 hours in the future is profitable. And also: mining ASIC blocks 2 hours in the future is also profitable, because then, you block potential CPU-mined blocks from being broadcasted, for the next 20 minutes (meanwhile, you prepare such block, and then release it, when the network time will get there).
It is always the case, if you count CPU-mined blocks into difficulty adjustments. Because if you have 2016 blocks, where 1008 blocks are CPU-mined, and 1008 blocks are ASIC-mined, then in practice, the real difficulty should be calculated after the next 1008 ASIC blocks. But instead, it is re-adjusted now, so the more CPU-mined blocks are there, the less ASIC-mined blocks remains (until falling above 10 minutes per block on average, when the difficulty then starts falling).
If it will be harder to mine testnet4, then the value per coin will increase. Which means more trading. Unless people will stay in testnet3, where the chain is faster, because of blockstorms.
All of them? Including also those with zero fees, or other non-matching mempool policy? |
|
cc @Emzy |
This comment was marked as abuse.
This comment was marked as abuse.
neither this is not a statement with regards to core maintainance though, I'm not opinionated with regards to that just pointing out that there is demand for this kind of network |
What about using "getblocktemplate" on mainnet, and mining blocks, which will quickly become stale? If you want to have worthless coins, then they should be stale, in other cases, they will be traded. Another possibility is to use "Pay to Proof of Work" addresses on signet/regtest. Then, moving a coin will require grinding a DER signature, to get it below N bytes, so it will require some mining power.
Now, testnet4 coins are actively traded, as well as testnet3 coins. Which means, that it is time for testnet5. But: it should have different rules, because if blocks would have any persistence, then they will be traded again. That's why all of them should be stale, so they could be shared between interested parties, while also being worthless at the same time. |
|
In 7864871. If I fire a Valgrind stack trace |
|
A miner (earning coinbase rewards to tb1q2dsc... is currently overwhelmingly dominant on testnet4. Since block 000000000000000167286a4bea3bca60d6c3ab2cbbef79537bdf07b6420a2d2d / 54780 this miner has mined every block with a difficulty = 1.0 |
7864871 to
2e3fcb0
Compare
@garlonicon Can you provide a link? Otherwise I don't think I can follow your ideas how they can be used for a open test network without creating other issues. I am happy to review a proposal if you write it up in more detail. |
Thanks @Davidson-Souza , good find! I pushed an update that prevents the rollback during IBD and that should fix it. |
|
@orangesurf Thanks for the insight, I have seen some your comments on Twitter too, just pretty busy at the moment :) It seems now the latest block with a real difficulty is 55100 so from time to time real miners still get luck but it seems to be much less frequently than in the beginning. This might be due to honest miners pointing their hashrate elsewhere or just shutting them down. At least that's the only explanation I have for seeing less non-difficulty 1 blocks. |
This comment was marked as abuse.
This comment was marked as abuse.
|
I'd like to try this with Stratum v2 which doesn't use the |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
|
⌛ There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
|
https://github.com/average-gary/bitcoin-garrys-mod/tree/testnet4-reorg-sv2 I had Claude implement changes from this PR with https://github.com/Sjors/bitcoin/tree/sv2 branch so I could use Sv2 for mining. Hoping to leave it hashing to see if we can't reduce the difficulty on TN4 to normalized levels. I mined this block with it: https://mempool.space/testnet4/block/0000000000000000b92517e2fe1c4d68d8dcf5ad0068297355d7adfd867fe95c |
Note: This is just a proof of concept put here for discussion/feedback/brainstorming/improvement. I am not sure if we want to merge this into core at all since it can be dealt with externally as well. But this was the simplest way for me to make it work and it's probably also simpler for many miners to run this branch rather than another external software.
Context
Testnet4 fixes block storms but it kept the minimum-difficulty exception in place. The rationale was that this would be the only way developers could still get their non-standard transactions into the blockchain. However, probably due to the public discussion of this feature, this has attracted the attention of people whose motivation might be A) grab as many testnet coins as possible or B) to annoy developers. There are currently one or more actors who just max out the min-difficulty exception as much as possible and as quickly as possible, see https://mempool.space/testnet4. There are always several min-difficulty blocks appearing instantly after a normal difficulty mined block.
There are several implications to this:
Solution
The solution to this problem was already briefly discussed in the original pull request for Testnet4: The min-difficulty blocks can all be easily re-orged by a single block with the actual difficulty. The code here implements this functionality within the
getblocktemplateRPC. All blocks with min-difficulty (including forks) are temporarily marked invalid while the template is created. The blocks are only temporarily marked invalid since another miner might find an actual difficulty block on top of the min-difficulty blocks and we don't want to create a fork in the chain in that case.We do want to keep the transactions from the blocks and include them in the block template. That should work with this solution via the orphan pool but I haven't done extensive testing on this yet. In reality it seems like there are several transactions in these min-difficulty blocks with a minimum block height and these can not be included in the re-org block (except the ones that are in the first one). However, given that these transactions appear in almost every block it might just be self-transfers created by the spammer.
Mining Pool
The follow-up to this would be to run a public (solo) pool with this enabled so that we can get some more hash rate behind this without requiring any effort on the side of those with hash rate and demotivate the spammers.
Acknowledgement
I worked on this at Bitcoin++ in Berlin a week ago together with Emzy and Sjors. Together we were able to demo a pool running a previous version of this PR and connected a miner to it. That pool isn't public yet though.
Demo
Here are some log outputs showing how first the orphan blocks are invalidated and then the actual chain is rolled back. Then, after the template was constructed, the blocks are reconsidered.
Logs