Testchains: Introduce custom chain whose constructor... #8994

jtimon · 2016-10-22T02:25:25Z

...reads from runtime params and simplify the creation of partitioned chains by simply generating different gensis block hashes from a given custom name.

It also allows to customize any chain param in these custom chains (but not the other chains).

Dependencies:

Testschains: Many regtests with different genesis and default datadir Testschains: Many regtests with different genesis and default datadir #17037
Tests: Chainparams: Make regtest almost fully customizable Tests: Chainparams: Make regtest almost fully customizable #17032
Tests: Use self.chain instead of 'regtest' in all current tests Tests: Use self.chain instead of 'regtest' in all current tests #16681
Preparations for more testchains Preparations for more testchains #16680
Use a proper factory for creating chainparams Use a proper factory for creating chainparams #8855
~~- [ ] Really don't validate genesis block Really don't validate genesis block #9102~~
Introduce an ArgsManager class encapsulating cs_args, mapArgs and mapMultiArgs Introduce an ArgsManager class encapsulating cs_args, mapArgs and mapMultiArgs #9494
QA: segwit.py: s/find_unspent/find_spendable_utxo/ QA: segwit.py: s/find_unspent/find_spendable_utxo/ #11869
Refactor: One CBaseChainParams should be enough Refactor: One CBaseChainParams should be enough #12128

Other features:

Uses a custom chain for all python tests.
Create new testchains with different genesis hashes at will.
Load chainparams from ~~separated~~ file or command line. (file left for later, see https://github.com/jtimon/bitcoin/tree/b16-new-testnet-file )
New chains are neither orange, blue nor green: they're purple and have your custom chain petname shown in the GUI.
Extra context: some people asked for signed blocks but that's way more disruptive and this is already review-thirsty (see NOMERGE: WIP: Support block signed custom testchains #9177 ).

btcdrak · 2016-10-22T07:08:02Z

Concept ACK

btcdrak · 2016-10-22T07:27:26Z

Some ideas from IRC today: Allow users to specify a .json file with all the parameters, and include an option for federated signing with list of signing keys.

jtimon · 2016-11-03T22:39:08Z

Needed rebase. Also, new configurable parameter was introduced in #9053

jtimon · 2016-11-08T03:50:36Z

Rebased on top of #9102

jtimon · 2016-11-14T20:02:40Z

Since #8855 needed rebase, this one too. Also adapted some more rpc tests to use the custom chain instead of regtest (only 4 missing it seems, but travis should pass).

jtimon · 2016-12-02T09:02:50Z

Needed rebase.

Update:

Now all rpc tests pass for self.chain="regtest"

"Magic" fixes:

pruning.py and mempool_packages.py now pass with both regtest and custom (they were passing travis before because they are in the extended set of tests).

"Magic" fails:

Now p2p-segwit.py only passes with regtest but not with custom like before

This is related to #9102 (see #9102 (comment) ping @laanwj ) in the sense that we cannot test that the genesis block is not validated unless we can run the system for a chain whose genesis block doesn't comply with the rules.
For example, the custom chain (unlike regtest) doesn't comply with pow (although some values for -chainpetname should make it).

@gmaxwell, you suggested that something like #9177 would need a lot of coverage testing and review, and I completely agree. Any suggestion to advance in that front in this PR instead of #9177 (which currently doesn't work and has one unittest that is not independent) would be welcomed.

You have recently made improvements in the rpc test framework, @MarcoFalke , and are familiar with it. If you get bored and can help fix one of the 3 tests that aren't passing for custom or just comment about the python changes in general, that would be great.

REM:

The last 2 commits are not necessary, but anyone feel free to comment on them.
Except for p2p-compactblocks.py, segwit.py (and now p2p-segwit.py too) all rpc tests pass with "custom" too

jtimon · 2016-12-02T22:24:32Z

@TheBlueMatt said in #9243 :

I think for things which are their own classes, they should either have the arguments they care about passed into the constructor, or passed in when they need them (eg the way mempool limiting is done now)

The longer version of CreateChainParams could indeed take all fields in CChainParams and Consensus::Params as arguments, but that would break the encapsulation. Every time a field is added or removed all calls would need to be adapted. Note that ideally, in the future, all tests would rely on CreateChainParams but none of them on the chainparams globals (accessed to via SelectParams and Params() ).

Another simpler option would be to simply let chainparams.cpp depend on globals mapArgs and mapMultiArgs (it depends on util.o anyway).

Yet another option is to undo some of that changes in #9243 if that gets merged first, which would be my personal preference.

If anyone has more options, I'm all ears.

jtimon · 2016-12-03T13:13:51Z

Rebased

jtimon · 2016-12-20T07:08:26Z

Rebased

jtimon · 2019-10-03T03:25:04Z

Rebased on top of separated #17037

1) genesis block hash 2) default datadir 3) chain name (bip70) all directly or indirectly configured with the -chain option ...whose constructor reads params from regular arguments (like regtests)... Qt: Add a default purple and title for unkown chains

…getblockstats)

jtimon · 2019-10-08T20:18:19Z

Rebased on top of rebased and modified #16681 , one less commit in total, but the total diff it's the same.

jtimon · 2019-10-09T13:38:39Z

Since #16681 was closed, closing this too. There's also more discussions there with people saying they don't like the idea of moving the test framework default chain from "regtest" to "regtest2".

…nt tests 1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón) Pull request description: Simply avoiding the hardcoded string in more places for consistency. It can also allow for more easily reusing tests for other chains other than regtest. Separated from #8994 . Continues #16509 . It is still not complete (ie to be complete, we need the -chain parameter in #16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in #8994 ] ). But while being incomplete like #16509 , it's quite simple to review and another step forward IMO. ACKs for top commit: Sjors: re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files. elichai: Code review ACK 1abcecc ryanofsky: Code review ACK 1abcecc. ryanofsky: Code review ACK 1abcecc Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee

…l current tests 1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón) Pull request description: Simply avoiding the hardcoded string in more places for consistency. It can also allow for more easily reusing tests for other chains other than regtest. Separated from bitcoin#8994 . Continues bitcoin#16509 . It is still not complete (ie to be complete, we need the -chain parameter in bitcoin#16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in bitcoin#8994 ] ). But while being incomplete like bitcoin#16509 , it's quite simple to review and another step forward IMO. ACKs for top commit: Sjors: re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files. elichai: Code review ACK 1abcecc ryanofsky: Code review ACK 1abcecc. ryanofsky: Code review ACK 1abcecc Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee

… "regtest" faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke) fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke) 68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley) Pull request description: This is required for various work in progress: * testchains bitcoin#8994 * signet bitcoin#16411 * some of my locally written tests While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes. ACKs for top commit: jonatack: ACK faf3683, ran tests and reviewed the code. Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6

…t' in all current tests 1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón) Pull request description: Simply avoiding the hardcoded string in more places for consistency. It can also allow for more easily reusing tests for other chains other than regtest. Separated from bitcoin#8994 . Continues bitcoin#16509 . It is still not complete (ie to be complete, we need the -chain parameter in bitcoin#16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in bitcoin#8994 ] ). But while being incomplete like bitcoin#16509 , it's quite simple to review and another step forward IMO. ACKs for top commit: Sjors: re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files. elichai: Code review ACK 1abcecc ryanofsky: Code review ACK 1abcecc. ryanofsky: Code review ACK 1abcecc Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee

* Merge bitcoin#16509: test: Adapt test framework for chains other than "regtest" faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke) fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke) 68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley) Pull request description: This is required for various work in progress: * testchains bitcoin#8994 * signet bitcoin#16411 * some of my locally written tests While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes. ACKs for top commit: jonatack: ACK faf3683, ran tests and reviewed the code. Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6 * Add devnet support for tests * test: make sure devnet can connect to each other and start * Partial merge bitcoin#16681: Tests: Use self.chain instead of 'regtest' in almost all current tests, revert one TODO while at it Co-authored-by: MarcoFalke <falke.marco@gmail.com> Co-authored-by: Jorge Timón <jtimon@jtimon.cc>

6fa901f Don't edit Chainparams after initialization (Jorge Timón) 980b38f MOVEONLY: Move versionbits info out of versionbits.o (Jorge Timón) Pull request description: This encapsulates the "-vbparams" option, which is only meant for regtest, directly on CRegTestParams. This is a refactor and doesn't change functionality. Related to bitcoin#8994 Tree-SHA512: 79771d729a63a720e743a9c77d5e2d80369f072d66202a43c1304e83a7d0ef7c6103d4968a03aea9666cc89a7203c618da972124a677b38cfe62ddaeb28f9f5d

* Merge bitcoin#13311: Don't edit Chainparams after initialization 6fa901f Don't edit Chainparams after initialization (Jorge Timón) 980b38f MOVEONLY: Move versionbits info out of versionbits.o (Jorge Timón) Pull request description: This encapsulates the "-vbparams" option, which is only meant for regtest, directly on CRegTestParams. This is a refactor and doesn't change functionality. Related to bitcoin#8994 Tree-SHA512: 79771d729a63a720e743a9c77d5e2d80369f072d66202a43c1304e83a7d0ef7c6103d4968a03aea9666cc89a7203c618da972124a677b38cfe62ddaeb28f9f5d * Resolve Merge with bitcoin#13311 * Incorporated review changes * Apply suggestions from code review * Update src/chainparams.cpp * Update src/chainparams.cpp Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com> Co-authored-by: MarcoFalke <falke.marco@gmail.com> Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com> Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

maflcko added Tests Consensus labels Oct 23, 2016

jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from ba59b47 to 62f928f Compare October 29, 2016 00:51

jtimon force-pushed the 0.13-new-testchain branch from 62f928f to 41207b1 Compare November 3, 2016 22:38

jtimon force-pushed the 0.13-new-testchain branch 2 times, most recently from 977b564 to ba1c6c4 Compare November 8, 2016 03:47

jtimon mentioned this pull request Nov 8, 2016

Really don't validate genesis block #9102

Closed

jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from e528398 to 39ac540 Compare November 14, 2016 19:59

jtimon force-pushed the 0.13-new-testchain branch from 39ac540 to d71e013 Compare November 15, 2016 23:50

jtimon mentioned this pull request Nov 17, 2016

NOMERGE: WIP: Support block signed custom testchains #9177

Closed

7 tasks

jtimon force-pushed the 0.13-new-testchain branch from a2477b9 to 9c76ffb Compare November 18, 2016 04:33

jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from fa6547a to 31b68a9 Compare December 2, 2016 07:49

jtimon mentioned this pull request Dec 2, 2016

Clean up mapArgs and mapMultiArgs Usage #9243

Merged

jtimon force-pushed the 0.13-new-testchain branch from 31b68a9 to 17528b6 Compare December 3, 2016 13:13

jtimon force-pushed the 0.13-new-testchain branch from 17528b6 to 8cb1b85 Compare December 20, 2016 07:05

jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from 59b7195 to 99343d1 Compare October 5, 2019 23:28

jtimon and others added 9 commits October 8, 2019 21:49

Tests: Use self.chain instead of 'regtest' in almost all current tests

83f08b3

9102: Really don't validate genesis block

c366558

Config: Allow any chain/section name

b333989

Chainparams: Test: Make is_test_chain configurable for regtest

6ad087e

Test: select chain using -chain=regtest instead of regtest=1

5326b3f

Test: Custom chain genesis' are deterministic from the name

d42efe1

Chainparams: Make regtest almost fully customizable

635771d

QA: Use resgtest2 chain instead of regtest for rpc tests (except rpc_…

0da5e72

…getblockstats)

jtimon force-pushed the 0.13-new-testchain branch from 99343d1 to 0da5e72 Compare October 8, 2019 20:15

jtimon mentioned this pull request Oct 8, 2019

BIP-325: Signet support #16411

Closed

3 tasks

jtimon closed this Oct 9, 2019

rkarthik2k21 mentioned this pull request Aug 25, 2021

Merge #13311: Don't edit Chainparams after initialization dashpay/dash#4361

Closed

bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Testchains: Introduce custom chain whose constructor... #8994

Testchains: Introduce custom chain whose constructor... #8994

Uh oh!

jtimon commented Oct 22, 2016 •

edited

Loading

Uh oh!

btcdrak commented Oct 22, 2016

Uh oh!

btcdrak commented Oct 22, 2016

Uh oh!

jtimon commented Nov 3, 2016

Uh oh!

jtimon commented Nov 8, 2016

Uh oh!

jtimon commented Nov 14, 2016

Uh oh!

jtimon commented Dec 2, 2016 •

edited

Loading

Uh oh!

jtimon commented Dec 2, 2016

Uh oh!

jtimon commented Dec 3, 2016

Uh oh!

jtimon commented Dec 20, 2016

Uh oh!

jtimon commented Oct 3, 2019

Uh oh!

jtimon commented Oct 8, 2019

Uh oh!

jtimon commented Oct 9, 2019

Uh oh!

Uh oh!

Testchains: Introduce custom chain whose constructor... #8994

Testchains: Introduce custom chain whose constructor... #8994

Uh oh!

Conversation

jtimon commented Oct 22, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

btcdrak commented Oct 22, 2016

Uh oh!

btcdrak commented Oct 22, 2016

Uh oh!

jtimon commented Nov 3, 2016

Uh oh!

jtimon commented Nov 8, 2016

Uh oh!

jtimon commented Nov 14, 2016

Uh oh!

jtimon commented Dec 2, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtimon commented Dec 2, 2016

Uh oh!

jtimon commented Dec 3, 2016

Uh oh!

jtimon commented Dec 20, 2016

Uh oh!

jtimon commented Oct 3, 2019

Uh oh!

jtimon commented Oct 8, 2019

Uh oh!

jtimon commented Oct 9, 2019

Uh oh!

Uh oh!

jtimon commented Oct 22, 2016 •

edited

Loading

jtimon commented Dec 2, 2016 •

edited

Loading