Skip to content

Conversation

jtimon
Copy link
Contributor

@jtimon jtimon commented Sep 14, 2015

This replaces/includes #6625 (previously #6526), #6009, #6051 and #6591 (previously #6024).

EDIT (further explanation demanded):

Although the PRs linked from here can provide a better explanation of the motivation, it is not so convenient for reviewers since they have to go through them individually.

The first 4 commits are taken from #6625, which is a modification of #6526 that doesn't break some other things. The modifications are explained in #6625. Although the #6526 has an extensive description, I will summarize its general purpose here.
The goal is to prepare the consensus code for an eventual block size hardfork and also for previous testing. It should simplify other PRs like #6382 (that introduces std::numeric_limits<uint64_t>::max() testnet chains to test different block sizes), #6451 (BIP102) or any other block-size-related proposal.

The next commit is taken from #6009, which separates GetMedianTimePast() from CBlockIndex and moves it to consensus (it's done in the same commit instead of the next one because it's "cheaper diff-wise"). GetMedianTimePast() is needed for verifying block headers, but chain.o will not.

The next commit is the code movement. All consensus critical functions that are not separated already (like 4 of the 5 functions in pow.o [the other one can be separated out]) in build-separable files are moved. If you think any of the moved functions are not consensus critical or that there's some missing functions, please, say so. One exception is main::ConnectBlock() which is consensus critical but contains other things that are not welcomed in libconsensus. A VerifyBlock library-safe function can be extracted from it later. #6051 was a similar movement.

The last commit is taken from #6591 (previously #6024). This one has 2 purposes:

  1. Remove chainparams.h and timedata.h dependencies (which are not library-friendly) from consensus.o
  2. It's part of a bigger process (see DEPENDENT: Globals: Avoid calling Params() #5970 ) to eventually be able to remove the pCurrentParams global (hidden behind Params(), but still a global). I hope I don't need to explain here why removing globals is a desirable thing...

@jtimon jtimon force-pushed the consensus-moveonly-0.12.99 branch from 07427df to f99a0fd Compare September 14, 2015 19:50
@jtimon
Copy link
Contributor Author

jtimon commented Sep 14, 2015

Fixed @luke-jr 's nits.

@jonasschnelli
Copy link
Contributor

Tested/Reviewed ACK.
A solution for FormatStateMessage() can be found later.

@jtimon
Copy link
Contributor Author

jtimon commented Sep 14, 2015

Some people are wondering why I made a particular change. See my answer in #6009 (comment)

@gmaxwell
Copy link
Contributor

Greg was here.

@morcos
Copy link
Contributor

morcos commented Sep 14, 2015

lightly reviewed ACK. passes pruning.py.

int64_t* pend = &pmedian[MEDIAN_TIME_SPAN];

for (unsigned int i = 0; i < MEDIAN_TIME_SPAN && pindex; i++, pindex = pindex->pprev)
*(--pbegin) = (int64_t)pindex->nTime;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated prior, this has changed, but ACK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, I agree and I'd prefer to see this left as a function. That'll mean addressing it again when the time comes.

jtimon referenced this pull request in jtimon/bitcoin Sep 14, 2015
…dent function

...and CBlockIndex::nMedianTimeSpan into Consensus::Params::nMedianTimeSpan
@jtimon jtimon force-pushed the consensus-moveonly-0.12.99 branch from f99a0fd to 8d60d87 Compare September 14, 2015 22:40
@jtimon
Copy link
Contributor Author

jtimon commented Sep 14, 2015

Solved @dcousens and @sipa 's nits.

@dcousens
Copy link
Contributor

utACK

@btcdrak
Copy link
Contributor

btcdrak commented Sep 14, 2015

Looking good now.

* Context-independent CTransaction validity checks
*/
bool CheckTx(const CTransaction& tx, CValidationState& state, const Params& consensusParams);
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit, the docblocks need a blank link after each declaration (throughout this file).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? The documentation seems enough of a separation to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtimon it helps to distinguish blocks apart from each other, and is the standard adopted by most of the code base as far as I can tell.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from IRC:

< dcousens> jtimon: the doc blocks padded line is a standard used throughout hte codebas
< dcousens> the codebase*
< dcousens> It makes it easier to see blocks
< dcousens> Rather than just 1 contiguous piece of code
< jtimon> dcousens: this "standard" is not documented in the style guide and it's not followed in many places (for example, main.h)
< dcousens> jtimon: I only opened 3-4 files before I said that, maybe my sample size was too small
< dcousens> even in main.h, its pretty consistently there except for the 1 liners
< dcousens> I'd say its 50/50 in that file, with a higher prevalance in other files
< jtimon> we should document this if it's part of the style (unfortunately I don't think clang-format can enforce it)
< dcousens> jtimon: maybe just be consistent with what is already in the file
< dcousens> jtimon: i've lost the PR link, but, I'll leave that up to you
< dcousens> its a nit anyway
< jtimon> not opposing to it, but some people are already verifying the move-only commit and I would prefer not to modify the hash of the commit unless I need to for some other reason
< jtimon> well, there's not much in consensus/consensus.h before this PR...
< dcousens> :thumbs_up:, just mention that, the inconsistencies and that at some point we should unify it
< dcousens> its not relevant to your PR if its inconsistent elsewhere

@theuni
Copy link
Member

theuni commented Sep 14, 2015

ACK, not yet verified completely move-only

@sipa
Copy link
Member

sipa commented Sep 14, 2015

Confirmed move-only 35f8ebd80614dd1617b40ee221070871241ff551, apart from a set -> std::set.

@jgarzik
Copy link
Contributor

jgarzik commented Sep 15, 2015

General comment:

Is there a link to any sort of written or spoken plan for these sort of commits? Was that covered at a conference roundtable?

To someone not watching this specific subset of libconsensus refactoring PRs, the 10,000 foot view is that there is a rather random stream of refactors that proceed in fits and starts without apparent plan or end other than a one sentence "isolate consensus state and code" summary.

@dcousens
Copy link
Contributor

Agreed with @jgarzik, these PRs do appear out of nowhere and often have little to no description. But there does seem to be some immediate consensus as though some discussion has occurred?

Don't get me wrong, having reviewed each of the above PR's, I am in favour of the refactor, but, it appeared to be random each time as to why they keep getting closed/re-opened and even re-authored without much of a comment.

@jgarzik
Copy link
Contributor

jgarzik commented Sep 15, 2015

In general there is the goal to move consensus state and code to a specific, separate lib.

I am hoping that

  • There is some plan
  • We will not see a five year stream of random consensus code movement patches causing lots of downstream developer headaches.

Added - I read every code change in every pull request that comes into github/bitcoin/bitcoin with three exceptions:

  • consensus code movement changes - too big, too chaotic, too frequent, too unfocused
  • some non-code changes (docs)
  • ignore 80% of the Qt changes

@jtimon
Copy link
Contributor Author

jtimon commented Sep 15, 2015

@btcdrak I could have included using namespace std to avoid that little change, but I actually like to sneak at least one little change in move-only commits to make sure somebody actually verifies the move-only (now I know that sipa and you have done so).

@jgarzik @dcousens I'm sorry for not providing a bigger description, but I thought that reading the PRs that this includes/replaces would be enough. I will update the description and answer to the off-topic complains in the mailing list.

@jtimon jtimon force-pushed the consensus-moveonly-0.12.99 branch from 8d60d87 to 321becc Compare September 15, 2015 17:19
@jtimon
Copy link
Contributor Author

jtimon commented Sep 15, 2015

I updated the initial description and the last commit (I forgot to remove #include "timedata.h").

@dcousens
Copy link
Contributor

@jtimon bah, you rebased. You could always squash later if your just fixing small things?
It would just make re-ACKing the PR easier.

@dcousens
Copy link
Contributor

utACK 321becc

@jtimon
Copy link
Contributor Author

jtimon commented Sep 16, 2015

I made it separate, but it was a one line change to the last commit so I just squashed it directly. Previous commits (particularly the move-only one) are unaffected.

@jtimon jtimon force-pushed the consensus-moveonly-0.12.99 branch from 321becc to 23e8e95 Compare September 22, 2015 17:33
@jtimon
Copy link
Contributor Author

jtimon commented Sep 22, 2015

Needed rebase.

@jgarzik
Copy link
Contributor

jgarzik commented Sep 22, 2015

Needs comprehensive plan discussed on bitcoin-dev.

@jtimon
Copy link
Contributor Author

jtimon commented Sep 23, 2015

@jgarzik there have been multiple discussions in the mailing list about this but you have chosen to ignore all of them (probably because you have limited time and this hasn't been a priority for you). In addition to that, I've been persistently asking for review on freenode's #bitcoin-dev People like @laanwj , @theuni , @sipa , @jonasschnelli and @morcos know this very well. If you want to be added to my "to ask for libconsensus-related review" list I am more than happy to do so. You would never miss one of my libconsensus-related PRs again. If not, you can still any concrete questions you may have about them (unfortunately you don't have any concrete questions for this one, despite how confusing and "random" it looks to you).

I won't create yet another ml thread that I myself have a hard time searching for when periodically someone asks about "the big picture" or "a detailed plan" for libconsensus-related work. This time I have created an issue that I can edit on github instead: #6714

If you have any questions about this PR, please, ask them here.
If you have more general questions about "a plan" for libconsensus, please ask them in the issue's thread.
I've been trying hard to effectively communicate about this work for more than a year and I believe I have miserably failed, not just communicating it to you. You can help me with your apparently many doubts if you turn them into concrete questions.

@jtimon
Copy link
Contributor Author

jtimon commented Sep 24, 2015

@jgarzik Please see #6714 (as discussed in IRC and the mailing list).

EDIT: also, it needed rebase again.

theuni and others added 7 commits September 24, 2015 22:16
…tion number

This is a no-op change. For now, everything passes MAX_BLOCK_SIZE / 60, so the
result matches what it would've before.

Tests use a number equal to the number of transactions where necessary,
to ensure that they're never rejected when blocksizesize isn't being tested.
This is a no-op change.

Tests use a value of std::numeric_limits<uint64_t>::max() where necessary, to ensure that they're never
rejected when size isn't being tested.
…sus.h (as functions)

The following are now tied to a chain rather than being defined as global
constants. Their values have not changed.

nMinTxSize
nMaxBlockSize
nMaxTxSize
nMaxBlockSigops
nCoinbaseMaturity

Also, for free (diff-wise):

Blocksize: Turn MAX_BLOCK_SIZE (nMaxBlockSize) and MAX_BLOCK_SIGOPS (nMaxBlockSigops) into functions

...which take Consensus::Params as parameter
This will be convenient to reduce the diff of any proposal that changes the blocksize as a hardfork
…tions:

-CheckBlockHeader
-ContextualCheckBlockHeader
-CheckBlock
-ContextualCheckBlock

Also add nTime parameter to CheckBlockHeader and CheckBlock.
Also use the oportunity to rename the functions inside the Consensus namespace.
@jtimon jtimon force-pushed the consensus-moveonly-0.12.99 branch from 23e8e95 to 88a3554 Compare September 24, 2015 20:37
@jtimon
Copy link
Contributor Author

jtimon commented Oct 20, 2015

Needs rebase, but since more documentation was demanded to continue with the libconsensus work, I will rebase and reopen #6625 instead, which is more related to testing and simplifying future potential blocksize-related changes than it is to the completion of libconsensus. It also contains 4 of the 7 commits contained in this PR, but doesn't have any code movement refactor.

To all the people who reviewed this PR, please re-review the rebased #6625 so that the review of this PR hasn't been completely in vain.

Closing this.

@jtimon
Copy link
Contributor Author

jtimon commented Nov 11, 2015

Sorry for insisting, but I would really appreciate some re-utACKs (or re-"I have been here") on #6625 before squashing it.
What's the motivation for #6625 if we're leaving the libconsensus movements for after 0.12 ? Apart from preventing any maxblocksize changes from getting in the way of libconsensus in the future and making any maxblocksize prototype branch easier to read, both #5970 and #6382 depend on #6625 and would become much easier to review and maintain if #6625 was merged.

@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants