Skip to content

Conversation

jamesob
Copy link
Contributor

@jamesob jamesob commented Jul 23, 2019

This is part of the assumeutxo project:

Parent PR: #15606
Issue: #15605
Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal


This change encapsulates UTXO set data within CChainState instances, removing global data pcoinsTip and pcoinsviewdb. This is necessary if we want to maintain multiple chainstates with their own rendering of the UTXO set.

We introduce a class CoinsViews which consolidates the construction of a CCoins* hierarchy.

This commit could be broken into smaller pieces, but it would require more ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor invocations.

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to split this into dumb refactoring and then hard-to-review changes, like the on in init.cpp?

An early commit (scripted-diff or not) could add ChainstateActive::CoinsTip, a dumb alias to return pcoinsTip; and replace the usages. The later commit could then make the other changes in this pull.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 23, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #16411 (Signet support by kallewoof)
  • #16400 ([refactor] Rewrite AcceptToMemoryPoolWorker() using smaller parts by sdaftuar)
  • #16324 (Get cs_main out of the critical path in ProcessMessages by TheBlueMatt)
  • #16323 (Call ProcessNewBlock() asynchronously by TheBlueMatt)
  • #15997 (refactor: GuessVerificationProgress requires cs_main lock by promag)
  • #13868 (Remove unused fScriptChecks parameter from CheckInputs by Empact)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from e56af43 to ceca2c9 Compare July 24, 2019 17:45
@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from ceca2c9 to cce18b8 Compare July 24, 2019 19:46
Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK ceca2c9. This looks good and is very straightforward. Feel free to ignore all my suggestions below! (They are not important and possibly not coherent.)

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch 2 times, most recently from 63a3e4e to d0e1e37 Compare July 24, 2019 21:51
@jamesob
Copy link
Contributor Author

jamesob commented Jul 24, 2019

Thanks for the feedback, @ryanofsky. I think I've incorporated all of it.

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d0e1e37 minus pcoinsTip removal.

Also I checked the other access to CoinsViews members via CoinsTip like in src/rpc/blockchain.cpp or src/rpc/rawtransaction.cpp. I've found every to be annotated or with a lock before, but was wandering if we shouldn't rely on EXCLUSIVE_LOCK_REQUIRED if possible and not AssertLockHeld. What's the exact diff between them ?

size_t cache_size_bytes,
bool in_memory,
bool should_wipe,
std::string leveldb_name = "chainstate");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm is there any risk than having a default leveldb_name cause database collisions for different CChainState at some point ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like something that would be caught pretty easily in tests if the confusion ever did happen, and I'm not sure if repeating "chainstate" in a few different places inline is any better. That said I'm happy to make this change if others agree it seems worthwhile.

@sipa
Copy link
Member

sipa commented Jul 25, 2019

EXCLUSIVE_LOCK_REQUIRED is checked at compile time, but only when the compiler has certain flags enabled (only on Clang, I believe).

AssertLockHeld is checked at runtime when lock debugging is compiled in (available for all compilers, but only enabled in debug builds).

@ariard
Copy link

ariard commented Jul 25, 2019

Yes EXCLUSIVE_LOCK_REQUIRED is defined in threadsafety.h and is part of Clang static analysis on Mac OS X. But reviewing last commit, e.g in ActivateBestChainStep we only use AssertLockHeld but in fact we should use Clang attribute (or maybe both). Is there a consistent policy on which one to favour ?

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from d0e1e37 to c860b44 Compare July 25, 2019 14:24
@jamesob
Copy link
Contributor Author

jamesob commented Jul 25, 2019

Thanks for the look, @ariard. Feedback pushed.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK c860b44. There have just been a lot of small changes since last review: rebase for bilingual_str, many new code comments and commit comments, more MakeUniques and c++ references replacing raw pointers, avoiding recursive cs_main lock in gettxoutsetinfo.

re: #16443 (comment)

Is there a consistent policy on which one to favour ?

It's good to use EXCLUSIVE_LOCK_REQUIRED wherever possible. AssertLockHeld is older and not used as much recently, but still useful for compilers other than clang and cases where clang annotations don't work (sometimes when there are private mutexes, locks in callbacks and virtual methods).

@maflcko
Copy link
Member

maflcko commented Jul 26, 2019

@maflcko
Copy link
Member

maflcko commented Jul 26, 2019

In commit 53c8d59 refactor: have CCoins* data managed under CChainState

   - The initialization order in init.cpp is changed slightly: the coinsdb (leveldb)
      creation is moved to be earlier than block index and genesis block loading,
      though this shouldn't have any noticeable effect.

Could you elaborate a bit more why it is save to flush the chainstate when shutdown is requested during the init sequence? As I read the code, previously we wouldn't flush and now we flush an empty db (potentially not having the best block hash set, because genesis is not loaded?).

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK c860b44, code looks good overall. Running into segfaults when testing

Show signature and timestamp

Signature:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

ACK c860b44ce3, code looks good overall. Running into segfaults when testing
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
pUi8YAwAnzGBHNruwZp6wvxhnfnew86jtl74PDidtnPwvGX9JiMOeY07UF70JOnO
OwTXN9IV7ZSAQklT0qik7QWAodG0h5TARUsmhnBfITrBTka81h+6JIN/ZTLq+abC
katI8VltU56K8HeXEcYTXuPWN3pjJAFlVphLPgvtxp5EwsFvykK6iebSKAKFfe5w
dCxSTuL/G/T+PAQNMXxAGS5MiheRFHu7I70ES9ddRuH4qGkVVutnjY9skajcX2+a
w9m1s/fFovci/bOaYCfQqfYlWXLHnOS+ewl9kGOPC3re6l3byY4+Lr19cmrDlNE+
qp3igws7FnoOJOjETY5D5sBvd+AikZcBk9uvVhWubPpJ5uLHQJactj/UigJtfGsJ
rKCpwX7oN30ZeSRf3vjCDUqO6HKKHih9dsPhQ8ikjswfqqTuMU0oPoNL+9PSaZi7
k9X0qcTpsC0M+Da+K2egehus507NFwpLLnMAWftLhzQoIxdt4/YHNEf0QT7P2L5F
5zZwImCS
=zVl4
-----END PGP SIGNATURE-----

Timestamp of file with hash ccd43c0d933c3fca4055f6f6006fddbebd4ec81c67d0810961187dc2dab9838d -

{
CBlockIndex *pindexDelete = m_chain.Tip();
CBlockIndex* pindexDelete = m_chain.Tip();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you removing the mempool lock annotation? Aren't the lock annotations in the header correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah you're right - dumb mistake on my part.

@@ -2297,7 +2326,8 @@ class ConnectTrace {
*
* The block is added to connectTrace if connection succeeds.
*/
bool CChainState::ConnectTip(CValidationState& state, const CChainParams& chainparams, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions &disconnectpool)
bool CChainState::ConnectTip(CValidationState& state, const CChainParams& chainparams, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool)
EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

@maflcko
Copy link
Member

maflcko commented Jul 26, 2019

I am seeing segmentation faults when shutting down during init:

==4778== Thread 9 bitcoin-shutoff:
==4778== Invalid read of size 4
==4778==    at 0x4953BC: FlushBlockFile(bool) (validation.cpp:1548)
==4778==    by 0x496B76: CChainState::FlushStateToDisk(CChainParams const&, CValidationState&, FlushStateMode, int) (validation.cpp:2061)
==4778==    by 0x497AB5: CChainState::ForceFlushStateToDisk() (validation.cpp:2115)
==4778==    by 0x32AE19: Shutdown(InitInterfaces&) (init.cpp:239)
==4778==    by 0x223F6F: BitcoinCore::shutdown() (bitcoin.cpp:161)
==4778==    by 0x58F4BF9: QObject::event(QEvent*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x4BFCAF5: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.12.4)
==4778==    by 0x4C05E7F: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.12.4)
==4778==    by 0x58C9AE7: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x58CCA92: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x591EE46: ??? (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x6F2EEDC: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6000.5)
==4778==  Address 0x4 is not stack'd, malloc'd or (recently) free'd
==4778== 
==4778== 
==4778== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==4778==  Access not within mapped region at address 0x4
==4778==    at 0x4953BC: FlushBlockFile(bool) (validation.cpp:1548)
==4778==    by 0x496B76: CChainState::FlushStateToDisk(CChainParams const&, CValidationState&, FlushStateMode, int) (validation.cpp:2061)
==4778==    by 0x497AB5: CChainState::ForceFlushStateToDisk() (validation.cpp:2115)
==4778==    by 0x32AE19: Shutdown(InitInterfaces&) (init.cpp:239)
==4778==    by 0x223F6F: BitcoinCore::shutdown() (bitcoin.cpp:161)
==4778==    by 0x58F4BF9: QObject::event(QEvent*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x4BFCAF5: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.12.4)
==4778==    by 0x4C05E7F: QApplication::notify(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.12.4)
==4778==    by 0x58C9AE7: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x58CCA92: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x591EE46: ??? (in /usr/lib64/libQt5Core.so.5.12.4)
==4778==    by 0x6F2EEDC: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.6000.5)

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from c860b44 to 8cd527d Compare July 26, 2019 16:06
@jamesob
Copy link
Contributor Author

jamesob commented Jul 26, 2019

Good catches, @MarcoFalke! I've pushed fixes, including e4b3247 to solve the init bug you found. There are a few ways of addressing that one but I think an early exit from FlushStateToDisk() is the cleanest since it doesn't involve partial construction of CoinsViews or introducing more have-I-initialized-yet state, but let me know if you think there's a better way.

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from 8cd527d to 4f44bd2 Compare July 26, 2019 16:40
@ryanofsky
Copy link
Contributor

In commit 53c8d59 refactor: have CCoins* data managed under CChainState

   - The initialization order in init.cpp is changed slightly: the coinsdb (leveldb)
      creation is moved to be earlier than block index and genesis block loading,
      though this shouldn't have any noticeable effect.

Could you elaborate a bit more why it is save to flush the chainstate when shutdown is requested during the init sequence?

I'm also curious about the answer to this. Would be good to say what's motivating the move in the comment. I guess I also had a similar question previously: #15606 (comment)

@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from 4f44bd2 to 4e307e5 Compare July 26, 2019 19:49
@jamesob
Copy link
Contributor Author

jamesob commented Jul 26, 2019

@ryanofsky @MarcoFalke good points. I've abandoned trying to reorder the init.cpp stuff and now have separated the construction of CChainState from CoinsViews to preserve the existing order.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 4e307e5. Only changes since last review are adding and using little InitCoinsViews / HasCoinsViews helpers so the init order doesn't have to change, and reverting the new FlushStateToDisk check which is no longer needed because the init order is not changing. Also, Marco's name is now added to the credits, but I can live with that.

-Thanks to Russell Yanofsky for helpful feedback.
+Thanks to Russell Yanofsky and Marco Falke for helpful feedback.

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 4e307e5 (reviewed changes since last one and ran tests)

@@ -314,7 +314,8 @@ bool CheckSequenceLocks(const CTxMemPool& pool, const CTransaction& tx, int flag
// Returns the script flags which should be checked for a given block
static unsigned int GetBlockScriptFlags(const CBlockIndex* pindex, const Consensus::Params& chainparams);

static void LimitMempoolSize(CTxMemPool& pool, size_t limit, unsigned long age) EXCLUSIVE_LOCKS_REQUIRED(pool.cs)
static void LimitMempoolSize(CTxMemPool& pool, size_t limit, unsigned long age)
EXCLUSIVE_LOCKS_REQUIRED(pool.cs, ::cs_main)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on above discussion on lock annotations, you may use also AssertLockHeld here and in UpdateTip
(also AcceptToMemoryPoolWithTime, DisconnectTip, ConnectTip)

@maflcko
Copy link
Member

maflcko commented Jul 31, 2019

ACK 4e307e5 (only checked that my name is in the credits, did not look at the changes)

Show signature and timestamp

Signature:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

ACK 4e307e5f5d676d01a94e1b645dcb0ee8574faf77 (only checked that my name is in the credits, did not look at the changes)
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
pUhusgwAu958HvbN9frbzwlJBJbL9+PABYM8kmB/+hArsWHs5J9vxPgDIyeGCsGN
fma8zMP/2Ru+YNBkr8fI8Dw24W7yrr+HliAzwV5czY9RFH+BJa4lON9lxs8Ni+07
cMwQe0I1mYYerOK8GEwmFqluyUpRx9EI5FF0gFsrmz4zKJI5KTGYv1XZ2ILLV4we
Zf3c58H0xLxi2RUywUIqNvYSYqsuJS1hf9bsvVnyzeJh0Uq84o74qnC9Y7Hx3Uy4
4gd4IzxUpiAeToE8KNpPfT2T+DmHNiuREqqmezUOdWkI1DN/RYY/jDORr2CysTvt
7mrrSpmNxbfTiboHZZvpxlFb94zQfxW08n9ex76z1N6aknbu7sTVW62Z+yQKU0LR
Nmcmxrg7gQ5uJU3B/cciM7TpJtHsOWHyUu1gDHGGBluK9kqTBt6jVU+XCwuJAUrU
hOgzIbf4qqIY3Bb5rq04RN/QBTDRZr43by8cX1HmyXNYXWaPQ2ZAVTdIgwYIPkjl
LLr6U7+n
=Lsbq
-----END PGP SIGNATURE-----

Timestamp of file with hash 055fc53ddab45b0500c4174e6690d5cf015f456b1806475966f66a2d74e0de0c -

This change encapsulates UTXO set data within CChainState instances, removing
global data `pcoinsTip` and `pcoinsviewdb`. This is necessary if we want to
maintain multiple chainstates with their own rendering of the UTXO set.

We introduce a class CoinsViews which consolidates the construction of a
CCoins* hierarchy. Construction of its various pieces (db, coinscatcher,
in-memory cache) is split up so that we avoid flushing bad state to disk if
startup is interrupted.

We also introduce `CChainState::CanFlushToDisk()` which tells us when it is
safe to flush the chainstate based on this partial construction.

This commit could be broken into smaller pieces, but it would require more
ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor
invocations.

Other changes:

- A parameter has been added to the CCoinsViewDB constructor that allows the
  name of the corresponding leveldb directory to be specified.

Thanks to Russell Yanofsky and Marco Falke for helpful feedback.
@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from cdf6afb to c41bcbd Compare August 15, 2019 15:10
i.e. any CoinsViews members. Adds a lock acquisition to `gettxoutsetinfo` RPC
to comply with added annotations.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
@jamesob jamesob force-pushed the 2019-07-au-coins-under-chainstate branch from c41bcbd to 582d2cd Compare August 15, 2019 15:20
@jamesob
Copy link
Contributor Author

jamesob commented Aug 15, 2019

@Sjors
Copy link
Member

Sjors commented Aug 15, 2019

reACK 582d2cd

@maflcko
Copy link
Member

maflcko commented Aug 15, 2019

ACK 582d2cd

Show signature and timestamp

Signature:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

ACK 582d2cd747
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE+rVPoUahrI9sLGYTzit1aX5ppUgFAlwqrYAACgkQzit1aX5p
pUinvAv+MV+6LkNCYGqrYG1HI3cm/fQYw8DmpC0nsUHkaEDHCfaqTvAimUoXxveq
zIb2gsCgbyujAM+zCTLlUBdE1v4stJ1fPMq57afluQNpEXrRSYttdy33YAI5D4Dy
QJ/Ny9kHTdTMukYe8eVnDqNWgwIgjfPkzFrhOw3vlN45e6aVQl7WX+yvaaB5v1/A
fg8eZ93c8ey4X0P6hDaT2jdVNNWidlnXZ2+bbZOqEKUfI0boJtQrcp4oxUkJ40kf
A7kFuwBvKyxUs8HWLeTUW4P39RHBSuhjewaHuBVneJpPOw7I3tYcOUtZS7Rv8CJC
ZZnj+GVLnLyAz0HMD/KCuLGpFV28oxgHdMfqRkNfGvFJFkAsEfyuFjHjWBe9DT9A
owlq2UGeBlMznMC2UK+GIX+5TIKDBShgoXbblu9x8x7R+s9iSRHmEiSEGtOst6qP
OU6mr2hOTB3qg1wywWHNbxfqpJiuP6TknWOFm7G5GnJcSrKUfO42RII0Q2E/oVQ2
QxgRxcan
=/V5U
-----END PGP SIGNATURE-----

Timestamp of file with hash 8c58c7b1c21f3568020ef04ac438a92aba830446a5172e171b37e02c744d64e0 -

@maflcko maflcko merged commit 582d2cd into bitcoin:master Aug 15, 2019
maflcko pushed a commit that referenced this pull request Aug 15, 2019
582d2cd Cover UTXO set access with lock annotations (James O'Beirne)
5693530 refactor: have CCoins* data managed under CChainState (James O'Beirne)
fae6ab6 refactor: pcoinsTip -> CChainState::CoinsTip() (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  This change encapsulates UTXO set data within CChainState instances, removing global data `pcoinsTip` and `pcoinsviewdb`. This is necessary if we want to maintain multiple chainstates with their own rendering of the UTXO set.

  We introduce a class CoinsViews which consolidates the construction of a CCoins* hierarchy.

  This commit could be broken into smaller pieces, but it would require more ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor invocations.

ACKs for top commit:
  Sjors:
    reACK 582d2cd
  MarcoFalke:
    ACK 582d2cd

Tree-SHA512: ec9d904fe5dca8cd2dc4b7916daa5d8bab30856dd4645987300f905e0a19f9919fce4f9d1ff03eda982943ca73e6e9a746be6cf53b46510de36e8c81a1eafba1
domob1812 added a commit to domob1812/namecoin-core that referenced this pull request Aug 19, 2019
Similarly to upstream's -segwitheight argument for tests, we allow also
controlling BIP16's activation height with -bip16height.  This is used
in name_multisig.py; we also include BIP16 in getblockchaininfo with
the other buried softforks.  Since BIP16 is not yet active in Namecoin
(and thus especially not activated from the start), this is useful at
least in the forseeable future.

Use ::ChainstateActive().CoinsTip() instead of pcoinsTip in the Namecoin
code.  See bitcoin/bitcoin#16443.

Rebranded the privkey / address versions in the updated descriptor_tests
unit test.
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Sep 24, 2020
…ce.cpp

Summary:
upcoming [[bitcoin/bitcoin#16443 | PR16443]] adds an `#include <txdb.h>` to validation.h which makes
zmq want leveldb symbols. this is the first step towards avoiding that and
a generally Good Thing To Do. `#include <primitives/block.h>` will do.

Test Plan:
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7550
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 1, 2020
…tate

Summary:
Upcoming Core [[bitcoin/bitcoin#16443 | PR16443]] adds `g_chainstate` to validation.cpp, this caused a bit of a havoc with initializaion order of pindexFinalized (also declared in validation.cpp) so I took this opportunity to encapsulate the finalizedBlockIndex pointer into CChainState

additionally:

* moved an assertion that cs_main lock must be held down to where the actual pointer is accessed, to guarantee that all code paths trigger that.

* ~~added an assert to prevent users from changing the finalizationBlockIndex pointer to nullptr~~

Test Plan:
  cmake .. -DENABLE_SANITIZERS=address
  ninja all check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Subscribers: deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7578
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 15, 2020
Summary:
Upcoming Core [[bitcoin/bitcoin#16443 | PR16443]] makes validation.h #include <txdb.h>, which has
the unwelcome effect of making zmq transitively #include <leveldb/db.h>
which cmake helpfully doesn't make available to it.

turns out zmqpublishnotifier.cpp (the only translation unit requiring
validation.h) needs it only for ReadBlockFrom disk.

This looks like a good place to start to cut things out of
the validation TU which is basically responsible for everything right
now.

This diff moves a few functions related to handling block files into the
(tentatively named) readblockfromdisk.cpp translation unit so that
zmqpublishnotifier.cpp no longer pulls leveldb transitively.

Test Plan:
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Subscribers: deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7579
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 16, 2020
Summary:
This aliasing makes subsequent commits easier to review; eventually CoinsTip()
will return the CCoinsViewCache managed by CChainState.

bitcoin/bitcoin@fae6ab6

---

Depends on D7578 and D7579

Partial backport of Core [[bitcoin/bitcoin#16443 | PR16443]]

Test Plan:
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7580
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 16, 2020
…State

Summary:
This change encapsulates UTXO set data within CChainState instances, removing
global data `pcoinsTip` and `pcoinsviewdb`. This is necessary if we want to
maintain multiple chainstates with their own rendering of the UTXO set.

We introduce a class CoinsViews which consolidates the construction of a
CCoins* hierarchy. Construction of its various pieces (db, coinscatcher,
in-memory cache) is split up so that we avoid flushing bad state to disk if
startup is interrupted.

We also introduce `CChainState::CanFlushToDisk()` which tells us when it is
safe to flush the chainstate based on this partial construction.

This commit could be broken into smaller pieces, but it would require more
ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor
invocations.

Other changes:

- A parameter has been added to the CCoinsViewDB constructor that allows the
  name of the corresponding leveldb directory to be specified.

Thanks to Russell Yanofsky and Marco Falke for helpful feedback.

---

bitcoin/bitcoin@5693530

Partial backport of Core [[bitcoin/bitcoin#16443 | PR16443]]

Depends on D7580

Test Plan:
  cmake .. -DENABLE_SANITIZERS=address
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7581
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 16, 2020
Summary:
i.e. any CoinsViews members. Adds a lock acquisition to `gettxoutsetinfo` RPC
to comply with added annotations.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>

---

bitcoin/bitcoin@582d2cd

Depends on D7581

Concludes backport of Core [[bitcoin/bitcoin#16443 | PR16443]]

Test Plan:
  cmake .. -DENABLE_SANITIZERS=address
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D7582
@jamesob jamesob mentioned this pull request Aug 5, 2021
18 tasks
kwvg added a commit to kwvg/dash that referenced this pull request Oct 10, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 10, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 16, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 16, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 16, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 21, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Oct 22, 2021
pravblockc pushed a commit to pravblockc/dash that referenced this pull request Nov 18, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

9 participants