Skip to content

Conversation

starius
Copy link
Contributor

@starius starius commented Feb 1, 2024

Inspired by #27446, this PR implements the proposal detailed in the comment #27446 (comment).

Rationale

Introduce the ability to configure a custom target time between blocks in a custom Bitcoin signet network. This enhancement enables users to create a signet that is more conducive to testing. The change enhances the flexibility of signet, rendering it more versatile for various testing scenarios. For instance, I am currently working on setting up a signet with a 30-second block time. However, this caused numerous difficulty adjustments, resulting in an inconsistent network state. Regtest isn't a viable alternative for me in this context since we prefer defaults to utilize our custom signet when configured, without impeding local regtest development.

Implementation

If the challenge format is OP_RETURN PUSHDATA<params> PUSHDATA<actual challenge>, the actual challenge from the second data push is used as the signet challenge, and the parameters from the first push are used to configure the network. Otherwise the challenge is used as is.

Under the previous rules, such a signet challenge would always evaluate to false, suggesting that it is likely not in use by anyone. Updating bitcoind to a version that includes this change will not cause any disruptions — existing signet challenges will retain their original meaning without alteration.

The only parameter currently available is "target_spacing" (default 600 seconds). To set it, place 0x01<target_spacing as uint64_t, little endian> in the params. Empty params are also valid. If other network parameters are added in the future, they should use 0x02<option 2 value>, 0x03<option 3 value>, etc., following the protobuf style.

Two public functions were added to chainparams.h:

  • ParseWrappedSignetChallenge: Extracts signet params and signet challenge from a wrapped signet challenge.
  • ParseSignetParams: Parses <params> bytes of the first data push.

Function ReadSigNetArgs calls ParseWrappedSignetChallenge and ParseSignetParams to implement the new meaning of signetchallenge.

The description of the flag -signetchallenge was updated to reflect the changes.

A new unit tests file, chainparams_tests.cpp, was added, containing tests for ParseWrappedSignetChallenge and ParseSignetParams.

The test signet_parse_tests from the file validation_tests.cpp was modified to ensure proper understanding of the new logic.

In the functional test feature_signet.py, a test case was added with the value of -signetchallenge set to the wrapped challenge, setting spacing to 30 seconds and having the actual challenge OP_TRUE.

The Signet miner was updated, introducing a new option --target-spacing with a default of 600 seconds. It must be set to the value used by the network.

Example

I tested this PR against Mutinynet, a signet running on a custom fork of Bitcoin Core, implementing 30s target spacing. I successfully synchronized the blockchain using the following config:

signet=1
[signet]
signetchallenge=6a4c09011e000000000000004c25512102f7561d208dd9ae99bf497273e16f389bdbd6c4742ddb8e6b216e64fa2928ad8f51ae
addnode=45.79.52.207:38333
dnsseed=0

The content of this wrapped challenge:

6a OP_RETURN
4c OP_PUSHDATA1
09 (length of signet params = 9)
011e00000000000000 (signet params: 0x01, pow_target_spacing=30)
4c OP_PUSHDATA1
25 (length of challenge = 37)
512102f7561d208dd9ae99bf497273e16f389bdbd6c4742ddb8e6b216e64fa2928ad8f51ae
(original Mutinynet challenge)

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 1, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29365.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK 1440000bytes
Concept ACK benthecarman, TheCharlatan, Eunovo, edilmedeiros, cryptoquick, torkelrogstad, pinheadmz
Stale ACK i-am-yuvi

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17783 (common: Disallow calling IsArgSet() on ALLOW_LIST options by ryanofsky)
  • #17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
  • #17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
  • #17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@benthecarman
Copy link
Contributor

Huge Concept ACK, this is great

@@ -341,7 +341,7 @@ class SigNetParams : public CChainParams {
consensus.CSVHeight = 1;
consensus.SegwitHeight = 1;
consensus.nPowTargetTimespan = 14 * 24 * 60 * 60; // two weeks
consensus.nPowTargetSpacing = 10 * 60;
consensus.nPowTargetSpacing = options.pow_target_spacing;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial worry when seeing this PR was that it would touch mainnet consensus code in order to support these parameters. But clearly it doesn't, which is nice.

return;
}

if (params.size() != 1 + 8) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any specific reason why the params length must be 9?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format of params is extendable in case more fields are added in the future. It is encoded as a concatenation of (field_id, value) tuples, protobuf style. Currently there is only one field defined pow_target_spacing, whose field_id is 0x01 and the lendth of encoding is 8 (int64_t). So valid values of params are:

  • empty string (checked in if block above)
  • 0x01 followed by 8 bytes of pow_target_spacing (9 bytes in total)

If length is not 0 and not 9, the value can not be parsed.

Copy link
Contributor

@Eunovo Eunovo Feb 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@starius Might be useful to leave comment explaining why the params size is limited this way. Anyone looking to add a param in the future will see that they can change it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Eunovo That's a good idea, thanks! I added the text of my comment above to the code and pushed in a separate temporal commit to track changes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect to see a loop here that parses the byte vector:

  1. read byte -- if its unknown (not 01, currently) throw an InitError
  2. for byte == 01 read the next 8 bytes to set the value
  3. repeat for all future extensions

@TheCharlatan
Copy link
Contributor

Concept ACK

@Eunovo
Copy link
Contributor

Eunovo commented Feb 21, 2024

Tested ACK
Tried different signetchallenge inputs and got the expected results.

@starius
Copy link
Contributor Author

starius commented Feb 26, 2024

Rebased to resolve conflict.

Copy link
Contributor

@edilmedeiros edilmedeiros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

I prefer the approach in #27446 but I can't say this would not be the bitcoin style of using the existing script machinery to implement so a logic.

I would love to see an accompanying BIP to upgrade BIP 325, since the signet behavior is standardized there. In practice, this change in core will potentially make this implementation an "undocumented standard".

I personally would prefer the BIP discussion to happen before merging, specially to define which params would be included in the configurable set and see if anyone proposes a better encoding. I'm not suggesting, though, that this doesn't get merged if consensus form around this proposal. I was messing around recently to set a faster signet for instructional purposes, this seem to be the perfect fit.

I'm still to give the code a better look and to test it locally.

@@ -28,7 +28,7 @@ def set_test_params(self):
self.chain = "signet"
self.num_nodes = 6
self.setup_clean_chain = True
shared_args1 = ["-signetchallenge=51"] # OP_TRUE
shared_args1 = ["-signetchallenge=6a4c09011e000000000000004c0151"] # OP_TRUE, target_spacing=30.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel like changing this test is a good idea since the proposed change is supposed to be backward-compatible. Better would be to add an additional functional test just for the purpose of testing the new behavior while keeping the old functional test as a guard rail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the proposal! I reworked the change of feature_signet.py test. Instead of modifying the existing test case, I added a new one with the wrapped signet challenge.

@cryptoquick
Copy link

cryptoquick commented May 28, 2024

Tested ACK
Works great against the Mutinynet node with the provided configuration, and the approach seems appropriate

One thing I noticed is that progress=1.000000 is displayed on every block from the start... This isn't normal, but it might be a quirk of Mutinynet.

@Zk2u
Copy link

Zk2u commented Sep 12, 2024

@starius could you possibly rebase this onto latest?

@starius starius force-pushed the signet-blockitme-in-challenge branch from 64e9332 to f6d419e Compare September 12, 2024 16:16
@starius
Copy link
Contributor Author

starius commented Sep 12, 2024

I rebased the PR against latest master. Manually resolved a small code conflict in src/chainparams.cpp and a large one in contrib/signet/miner. In new OOP version of contrib/signet/miner my change is much smaller :)

Also I noticed that Makefile based testing was replaced with CMake based approach. I added new C++ file with tests (chainparams_tests.cpp) to src/test/CMakeLists.txt instead of src/Makefile.test.include.

@Zk2u
Copy link

Zk2u commented Sep 13, 2024

You're a legend - thank you.

@starius starius force-pushed the signet-blockitme-in-challenge branch 2 times, most recently from 66ec814 to c6bc0bb Compare December 31, 2024 14:03
Copy link
Member

@pinheadmz pinheadmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concept ACK

assert_equal(self.nodes[2].submitblock(block), None)
height += 1
assert_equal(self.nodes[2].getblockcount(), height)
for node_idx in [2, 6]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this? Why not also include nodes 0 & 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verify that nodes accept blocks mined using the default signet challenge. This test includes one default signet node (2) and another node (6) configured with an extended signet challenge that uses the actual script OP_TRUE (accepts all blocks).

I added a comment with this explanation to the code.

@pinheadmz
Copy link
Member

This change is backward-compatible, as per the old rules, such a signet challenge would always evaluate to false, rendering it unused.

I don't completely agree with this, since starting an old version of bitcoind with (for example)
-signetchallenge=6a4c0901ff000000000000004c0151 would create an incompatible node. It would have a different network magic bytes and I think the OP_RETURN challenge would cause it to reject all blocks.

@starius
Copy link
Contributor Author

starius commented Jan 2, 2025

This change is backward-compatible, as per the old rules, such a signet challenge would always evaluate to false, rendering it unused.

I don't completely agree with this, since starting an old version of bitcoind with (for example) -signetchallenge=6a4c0901ff000000000000004c0151 would create an incompatible node. It would have a different network magic bytes and I think the OP_RETURN challenge would cause it to reject all blocks.

The change is backward-compatible in the sense that if bitcoind is updated to a version with this change, nothing will break - no existing signet challenge should have its meaning altered.

You are correct that starting an old node with an extended signet challenge will result in it rejecting all blocks. However, I believe this outcome is actually beneficial: it would alert the node owner to the issue, prompting them to update their software.

I updated that part of the description to make it more clear: "Under the previous rules, such a signet challenge would always evaluate to false, suggesting that it is likely not in use by anyone. Updating bitcoind to a version that includes this change will not cause any disruptions — existing signet challenges will retain their original meaning without alteration.".

It would have a different network magic bytes.

This isn't accurate. The network magic bytes are derived solely from the real challenge (51 (OP_TRUE) in the case of 6a4c0901ff000000000000004c0151). Additionally, getblockchaininfo and getmininginfo both return 51, not the extended challenge, as verified in the feature_signet.py test.

@pinheadmz
Copy link
Member

This isn't accurate.

I meant, if you passed the extended challenge to an old version of bitcoin it would compute different magic bytes than if you passed the same challenge to an upgraded node.

@starius starius force-pushed the signet-blockitme-in-challenge branch from c6bc0bb to 7416b18 Compare January 2, 2025 17:50
@starius starius force-pushed the signet-blockitme-in-challenge branch 2 times, most recently from 1f28c0a to a59e9fe Compare April 19, 2025 23:32
Inspired by bitcoin#27446, this commit
implements the proposal detailed in the comment
bitcoin#27446 (comment).

Rationale.

Introduce the ability to configure a custom target time between blocks in a
custom Bitcoin signet network. This enhancement enables users to create a signet
that is more conducive to testing. The change enhances the flexibility of signet,
rendering it more versatile for various testing scenarios. For instance, I am
currently working on setting up a signet with a 30-second block time. However,
this caused numerous difficulty adjustments, resulting in an inconsistent
network state. Regtest isn't a viable alternative for me in this context since
we prefer defaults to utilize our custom signet when configured, without
impeding local regtest development.

Implementation.

If the challenge format is "OP_RETURN PUSHDATA<params> PUSHDATA<actual
challenge>", the actual challenge from the second data push is used as the
signet challenge, and the parameters from the first push are used to configure
the network. Otherwise the challenge is used as is.

Under the previous rules, such a signet challenge would always evaluate to
false, suggesting that it is likely not in use by anyone. Updating bitcoind to a
version that includes this change will not cause any disruptions - existing
signet challenges will retain their original meaning without alteration.

The only parameter currently available is "target_spacing" (default 600
seconds). To set it, place "0x01<target_spacing as uint64_t, little endian>" in
the params. Empty params are also valid. If other network parameters are added
in the future, they should use "0x02<option 2 value>", "0x03<option 3 value>",
etc., following the protobuf style.

Two public functions were added to chainparams.h:
  - ParseWrappedSignetChallenge: Extracts signet params and signet challenge
    from a wrapped signet challenge.
  - ParseSignetParams: Parses <params> bytes of the first data push.

Function ReadSigNetArgs calls ParseWrappedSignetChallenge and ParseSignetParams
to implement the new meaning of signetchallenge.

The description of the flag -signetchallenge was updated to reflect the changes.

A new unit tests file, chainparams_tests.cpp, was added, containing tests for
ParseWrappedSignetChallenge and ParseSignetParams.

The test signet_parse_tests from the file validation_tests.cpp was modified to
ensure proper understanding of the new logic.

In the functional test feature_signet.py, a test case was added with the value
of -signetchallenge set to the wrapped challenge, setting spacing to 30 seconds
and having the actual challenge OP_TRUE.

The Signet miner was updated, introducing a new option --target-spacing with a
default of 600 seconds. It must be set to the value used by the network.

Example.

I tested this commit against Mutinynet, a signet running on a custom fork of
Bitcoin Core, implementing 30s target spacing. I successfully synchronized the
blockchain using the following config:

signet=1
[signet]
signetchallenge=6a4c09011e000000000000004c25512102f7561d208dd9ae99bf497273e16f389bdbd6c4742ddb8e6b216e64fa2928ad8f51ae
addnode=45.79.52.207:38333
dnsseed=0

The content of this wrapped challenge:

6a OP_RETURN
4c OP_PUSHDATA1
09 (length of signet params = 9)
011e00000000000000 (signet params: 0x01, pow_target_spacing=30)
4c OP_PUSHDATA1
25 (length of challenge = 37)
512102f7561d208dd9ae99bf497273e16f389bdbd6c4742ddb8e6b216e64fa2928ad8f51ae
(original Mutinynet challenge, can be found here:
https://blog.mutinywallet.com/mutinynet/ )
@starius starius force-pushed the signet-blockitme-in-challenge branch from a59e9fe to 41728b5 Compare May 1, 2025 19:44
Copy link

@1440000bytes 1440000bytes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 41728b5

Isn't the commit message too long? You could even have multiple commits in this pull request IMO.

@DrahtBot DrahtBot requested review from yuvicc and pinheadmz May 1, 2025 20:26
@starius
Copy link
Contributor Author

starius commented May 1, 2025

utACK 41728b5

Isn't the commit message too long? You could even have multiple commits in this pull request IMO.

It is possible to split the commit (and the commit message) into multiple parts:

  • add function ParseWrappedSignetChallenge and its unit test (files src/chainparams.{h,cpp} src/test/chainparams_tests.cpp src/test/CMakeLists.txt)
  • add function ParseSignetParams and its unit test (files src/chainparams.{h,cpp} src/test/chainparams_tests.cpp)
  • update handling of option -signetchallenge, its help message and unit test (files src/chainparams.cpp src/kernel/chainparams.{h,cpp} src/chainparamsbase.cpp src/test/validation_tests.cpp)
  • add new test cases to Python test test/functional/feature_signet.py
  • Update signet miner contrib/signet/miner

Do you think, that such commit structure would make sense?

@Sjors
Copy link
Member

Sjors commented May 2, 2025

That sounds reasonable at first glance. Each commit has to pass the tests. Ideally anytime you introduce a new function you would also add at least one test, and/or use that function in existing code (to show that it doesn't break any test).

Copy link
Member

@pinheadmz pinheadmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code reviewed at 41728b5

Built and tested on arm64/macos. Played with the feature locally using 10 and 30 second block times. A few comments below, my main concern being a mismatch between the python miner and the custom consensus rules. Didn't have time to try and mine 2, 016 signet blocks to check that difficulty adjusted as expected, but otherwise everything seems to work as expected.

Consensus code is touched, but only for signet ;-)

@@ -21,7 +21,7 @@ void SetupChainParamsBaseOptions(ArgsManager& argsman)
argsman.AddArg("-testnet4", "Use the testnet4 chain. Equivalent to -chain=testnet4.", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
argsman.AddArg("-vbparams=deployment:start:end[:min_activation_height]", "Use given start/end times and min_activation_height for specified version bits deployment (regtest-only)", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CHAINPARAMS);
argsman.AddArg("-signet", "Use the signet chain. Equivalent to -chain=signet. Note that the network is defined by the -signetchallenge parameter", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
argsman.AddArg("-signetchallenge", "Blocks must satisfy the given script to be considered valid (only for signet networks; defaults to the global default signet test network challenge)", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::CHAINPARAMS);
argsman.AddArg("-signetchallenge", "Blocks must satisfy the given script to be considered valid (only for signet networks; defaults to the global default signet test network challenge); in case -signetchallenge is in the form of 'OP_RETURN PUSHDATA<params> PUSHDATA<actual challenge>', then <actual challenge> is used as a challenge and <params> is used to set parameters of signet; currently the only supported parameter is target spacing, the format of <params> to set it is 01<8 bytes value of target spacing, seconds, little endian>", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::CHAINPARAMS);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently the only supported parameter is target spacing, the format of to set it is 01<8 bytes value of target spacing, seconds, little endian>

I think you can remove this, it could get carried away if more parameters are added in the future. You could just link to verbose documentation in signet/README.md or reccomend using signet/miner, which leads me to a second suggestion:

Add a new command to signet/miner like generate, solvepsbt etc... maybe makechallenge that accepts arguments like --target-spacing and --actual-challenge or even --pubkey and spits back out a hex string to start your blockchain with.

You could also then pass --challenge to the generate command and the mining script will just parse all the consensus parameters automatically from that. I think that will reduce the possibilty of mistakes and make the whole thing easier to put together.

Comment on lines +116 to +120
std::vector<unsigned char> params;
std::vector<unsigned char> challenge;
ParseWrappedSignetChallenge(*val, params, challenge);
ParseSignetParams(params, options);
options.challenge.emplace(challenge);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not pass options and args to one function ParseSignetChallenge() that calls the two functions you have, and fills in the SigNetOptions members where applicable? In other words, why write data to these vectors?

Comment on lines +92 to +93
const uint64_t value = uint64_t(bytes[0]) | uint64_t(bytes[1])<<8 | uint64_t(bytes[2])<<16 | uint64_t(bytes[3])<<24 |
uint64_t(bytes[4])<<32 | uint64_t(bytes[5])<<40 | uint64_t(bytes[6])<<48 | uint64_t(bytes[7])<<56;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we have a serializer helper method for this kind of thing?

return;
}

if (params.size() != 1 + 8) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect to see a loop here that parses the byte vector:

  1. read byte -- if its unknown (not 01, currently) throw an InitError
  2. for byte == 01 read the next 8 bytes to set the value
  3. repeat for all future extensions

@DrahtBot DrahtBot requested a review from pinheadmz May 2, 2025 18:25
@pinheadmz
Copy link
Member

Crashed with long time spacing (are you sure you need 2^64 seconds?)

build/bin/bitcoind -signet -signetchallenge=6a4c09011effff00000000004c0151 -debug -debugexclude=libevent

bitcoin-cli -signet -generate

log:


2025-05-02T19:18:23Z [http] Received a POST request for / from 127.0.0.1:39282
2025-05-02T19:18:23Z [rpc] ThreadRPCServer method=getnewaddress user=__cookie__
2025-05-02T19:18:24Z [http] Received a POST request for / from 127.0.0.1:39284
2025-05-02T19:18:24Z [rpc] ThreadRPCServer method=generatetoaddress user=__cookie__
2025-05-02T19:18:24Z CreateNewBlock(): block weight: 884 txs: 0 fees: 0 sigops 400
Floating point exception

@starius
Copy link
Contributor Author

starius commented May 3, 2025

Crashed with long time spacing (are you sure you need 2^64 seconds?)

build/bin/bitcoind -signet -signetchallenge=6a4c09011effff00000000004c0151 -debug -debugexclude=libevent

bitcoin-cli -signet -generate

log:


2025-05-02T19:18:23Z [http] Received a POST request for / from 127.0.0.1:39282
2025-05-02T19:18:23Z [rpc] ThreadRPCServer method=getnewaddress user=__cookie__
2025-05-02T19:18:24Z [http] Received a POST request for / from 127.0.0.1:39284
2025-05-02T19:18:24Z [rpc] ThreadRPCServer method=generatetoaddress user=__cookie__
2025-05-02T19:18:24Z CreateNewBlock(): block weight: 884 txs: 0 fees: 0 sigops 400
Floating point exception

Great catch!

I think this is related to this place in src/kernel/chainparams.cpp:

consensus.nPowTargetTimespan = 14 * 24 * 60 * 60; // two weeks
consensus.nPowTargetSpacing = options.pow_target_spacing;

There is a code which divides these numbers in src/consensus/params.h:

int64_t DifficultyAdjustmentInterval() const { return nPowTargetTimespan / nPowTargetSpacing; }

nPowTargetTimespan should be calculated as 2016 * options.pow_target_spacing so difficulty adjustment interval scales together with the block interval. I'll apply the patch.

I'll try to reproduce this crash in test/functional/feature_signet.py adding edge cases.

are you sure you need 2^64 seconds?

I don't think we need such high values :-)
Maybe 2 bytes is enough. It corresponds to ~18 hours.

@starius
Copy link
Contributor Author

starius commented May 5, 2025

I think I discovered a bug in existing test test/functional/feature_signet.py.
I found that it generates a block. I added the code to make sure that the block is distributed between the two nodes (node 0 and node 1) using the same signet challenge:

diff --git a/test/functional/feature_signet.py b/test/functional/feature_signet.py
index 02e37f0fdd..86ab3202b8 100755
--- a/test/functional/feature_signet.py
+++ b/test/functional/feature_signet.py
@@ -87,7 +87,10 @@ class SignetBasicTest(BitcoinTestFramework):
         check_getmininginfo(node_idx=3, signet_idx=1)
         check_getmininginfo(node_idx=4, signet_idx=2)
 
-        self.generate(self.nodes[0], 1, sync_fun=self.no_op)
+        while not self.generate(self.nodes[0], 1, sync_fun=lambda: self.sync_blocks(self.nodes[0:2])):
+            pass
+        self.wait_until(lambda: self.nodes[0].getblockcount() == 1)
+        self.wait_until(lambda: self.nodes[1].getblockcount() == 1)
 
         self.log.info("pregenerated signet blocks check")

(The patched branch can be found here. It is the patch above applied to the master branch.)

The loop is needed because one self.generate call sometimes produces no blocks. My theory is that this happens because of maxtries=1000000 passed by the test framework to the node. Sometimes 1000000 tries is not enough to mine a block. That is why I added a loop.

self.wait_until is needed to make sure both nodes got the block. But this code is not even executed if the test fails.

This made the test flaky. Here is one of the fail logs

I see that node 0 successfully generated the block and updated its tip:

node0 2025-05-05T04:55:18.225539Z [scheduler] [validationinterface.cpp:179] [operator()] [validation] UpdatedBlockTip: new block hash=000002d68e882daae449c4eee999af64d893c68e34840737daa32d6d557dc5fb fork block hash=00000008819873e925422c1ff0f99f7cc9bbb232af63a077a480a3633bee1ef6 (in IBD=false) 

but node 1 for some reason got only block header and not the block itself:

node1 2025-05-05T04:55:18.226722Z [msghand] [net_processing.cpp:3437] [LogBlockHeader] [validation] Saw new header hash=000002d68e882daae449c4eee999af64d893c68e34840737daa32d6d557dc5fb height=1 peer=0 

Then self.sync_blocks failed because of a timeout.

How to debug this further? Is this a genuine bug or my change above is wrong?
Any help to sort this out is very appreciated!

@pinheadmz
Copy link
Member

What commit is that failure from?

@starius
Copy link
Contributor Author

starius commented May 5, 2025

@pinheadmz The failure is from commit starius@60a7042
It doesn't include this PR, it just the master (slightly outdated, commit eba5f9c) + the patch I posted above.
The command I'm running is build/test/functional/test_runner.py feature_signet.py --nocleanup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.