Use CScheduler for wallet flushing, remove ThreadFlushWalletDB #9605

TheBlueMatt · 2017-01-20T21:04:26Z

Also uses std::function instead of boost::function.

sipa

Concept ACK

sipa · 2017-01-20T22:56:02Z

src/scheduler.h

@@ -23,7 +22,7 @@
 //
 // CScheduler* s = new CScheduler();
 // s->scheduleFromNow(doSomething, 11); // Assuming a: void doSomething() { }
-// s->scheduleFromNow(boost::bind(Class::func, this, argument), 3);
+// s->scheduleFromNow(std::bind(Class::func, this, argument), 3);
 // boost::thread* t = new boost::thread(boost::bind(CScheduler::serviceQueue, s));


Adapt comment here as well?

No, it still must run inside a boost::thread because it relies on boost::interruption_point. I believe @theuni is planning on cleaning that up.

well technically boost::thread takes a boost::function, so I assume if you uses std::bind it would create a boost::function around the std::function......I'll let cory do that when we kill boost::threads.

Yes, I have a local branch with CScheduler converted. I'm doing them in chunks, because the last huge PR proved to be too big to review/rebase. I'll submit that one next (#9566 is the current one).

sipa · 2017-01-20T23:24:56Z

I mean the boost::bind here.

theuni · 2017-01-21T07:09:25Z

Concept ACK!. I've considered doing this a few times as well.

laanwj · 2017-01-23T12:27:02Z

src/wallet/walletdb.cpp

@@ -768,67 +768,60 @@ DBErrors CWalletDB::ZapWalletTx(CWallet* pwallet, vector<CWalletTx>& vWtx)
    return DB_LOAD_OK;
 }

-void ThreadFlushWalletDB()
+void MaybeFlushWalletDB()


If you're renaming this function please, finally, change it to something else than FlushWalletDB. What is being done here is not merely a flush (those are done all the time as CWalletDB instances go out of scope) but more like "database consolidation", merging the log files into wallet.dat so that the wallet is self-contained. This has confused many people over time.

Done, called it "compact"

laanwj · 2017-01-23T12:27:55Z

Concept ACK

laanwj · 2017-01-26T09:05:46Z

Thank you! utACK 30f8d50

TheBlueMatt · 2017-02-08T18:20:31Z

Rebased. Note that this now forms the first of a rather long patchset to move wallet callbacks into a background thread without introducing races in APIs or wallet consistency issues.

theuni · 2017-03-06T23:13:37Z

utACK. Needs rebase.

TheBlueMatt · 2017-03-06T23:35:41Z

Rebased.

laanwj · 2017-03-07T10:01:14Z

utACK 0235be1

…lletDB 0235be1 Rename FlushWalletDB -> CompactWalletDB, add function description (Matt Corallo) 735d9b5 Use CScheduler for wallet flushing, remove ThreadFlushWalletDB (Matt Corallo) 73296f5 CScheduler boost->std::function, use millisecs for times, not secs (Matt Corallo) Tree-SHA512: c04f97beab65706c444c126be229d02887df9b0972d8fb15ca1f779ef0e628cf7ecef2bf533c650d9b44645b63e01de22f17266a05907e778938d64cc6e19de6

…FlushWalletDB 0235be1 Rename FlushWalletDB -> CompactWalletDB, add function description (Matt Corallo) 735d9b5 Use CScheduler for wallet flushing, remove ThreadFlushWalletDB (Matt Corallo) 73296f5 CScheduler boost->std::function, use millisecs for times, not secs (Matt Corallo) Tree-SHA512: c04f97beab65706c444c126be229d02887df9b0972d8fb15ca1f779ef0e628cf7ecef2bf533c650d9b44645b63e01de22f17266a05907e778938d64cc6e19de6

…llet 98d770f CScheduler boost->std::function, use millisecs for times, not secs (Matt Corallo) 67e068c Remove now unneeded ChainTip signal (furszy) bcdd3e9 Move ChainTip sapling update witnesses and nullifiers to BlockConnected/BlockDisconnected. (furszy) b799070 Take a CTransactionRef in AddToWalletIfInvolvingMe to avoid a copy (Matt Corallo) d77244c Remove dead-code tracking of requests for blocks we generated (Matt Corallo) 10ccbbf Hold cs_wallet for whole block [dis]connection processing (Matt Corallo) 1a45036 fix compiler warning member functions not marked as 'override' (furszy) d3867a7 An adaptation of Corallo's btc@461e49fee2935b1eb4d4ea7bae3023e655c0a6d8 (Matt Corallo) f5ac648 [Refactor] Move Sapling ChainTip signal notification loop to use the latest connectTrace class structure (furszy) 8704d9d Handle SyncTransaction in ActivateBestChain instead of ConnectTrace (Matt Corallo) 6dcb6b0 Keep conflictedTxs in ConnectTrace per-block (Matt Corallo) 50d3e0e Handle conflicted transactions directly in ConnectTrace (furszy) 27fb897 Make ConnectTrace::blocksConnected private, hide behind accessors (Matt Corallo) 60329da Add pblock to connectTrace at the end of ConnectTip, not start (Matt Corallo) e7c2789 Include missing #include in zmqnotificationinterface.h (Matt Corallo) 1b396b8 Move `NotifyEntryRemoved` to use the general interfaces::MakeHandler (furszy) 4cb5820 Better document usage of SyncTransaction (Alex Morcos) 21be4e2 Introduce MemPoolConflictRemovalTracker (Alex Morcos) 7326acb mempool: add notification for added/removed entries (Wladimir J. van der Laan) a8605d2 Clean up tx prioritization when conflict mined (Casey Rodarmor) e7db9ff remove internal tracking of mempool conflicts for reporting to wallet (Alex Morcos) 76c72c6 remove external usage of mempool conflict tracking (Alex Morcos) ef429af tests: Stop node before removing the notification file (furszy) 15a21c2 tests: write the notification to different files to avoid race condition (Chun Kuan Lee) 466e97a [Wallet] Simplify InMempool (furszy) 85e18f0 Rename FlushWalletDB -> CompactWalletDB, add function description (Matt Corallo) 00f36ea Use CScheduler for wallet flushing, remove ThreadFlushWalletDB (Matt Corallo) Pull request description: Revamped the validation interface interaction with the wallet, encapsulating and improving the mempool and block signaling and each of the wallet signals handler. Restructured the Sapling witnesses and nullifiers update under the new signals. Solved several bugs that found on the way as well (Check each commit description). This PR concludes with #1726 long road :). Pushing our repository around 2 years ahead in btc time in the mempool and validation interface areas (without including RBF). The new validation -> wallet interaction architecture will enable further, and much more user facing important, improvements for the syncing process, overall software responsiveness and multithreading locking issues. Adapted backports: dashpay#6464 --> Always clean up manual transaction prioritization (mempool) bitcoin#9240 --> Remove txConflicted (mempool) bitcoin#9371 --> Notify on removal (mempool) bitcoin#9605 --> Use CScheduler for wallet flushing, remove ThreadFlushWalletDB (walletdb) bitcoin#9725 --> CValidationInterface Cleanups (wallet, validation and validation interface) ACKs for top commit: random-zebra: utACK 98d770f Fuzzbawls: utACK 98d770f Tree-SHA512: 84c86567c2bff36b859b2ae73c558956a70dff0fffb4f73208708d92165b851bf42d35246410238c66c7a4b77e5bf51ec93885234a75fa48901fd182d2f70a28

fanquake added the Refactoring label Jan 20, 2017

sipa reviewed Jan 20, 2017

View reviewed changes

laanwj reviewed Jan 23, 2017

View reviewed changes

laanwj added this to the 0.15.0 milestone Feb 6, 2017

TheBlueMatt force-pushed the 2017-01-cscheduler-cleanups branch from 30f8d50 to dbb8072 Compare February 8, 2017 18:19

TheBlueMatt mentioned this pull request Feb 8, 2017

CValidationInterface Cleanups #9725

Merged

TheBlueMatt added 3 commits March 6, 2017 18:33

CScheduler boost->std::function, use millisecs for times, not secs

73296f5

Use CScheduler for wallet flushing, remove ThreadFlushWalletDB

735d9b5

Rename FlushWalletDB -> CompactWalletDB, add function description

0235be1

TheBlueMatt force-pushed the 2017-01-cscheduler-cleanups branch from dbb8072 to 0235be1 Compare March 6, 2017 23:35

laanwj merged commit 0235be1 into bitcoin:master Mar 7, 2017

PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Feb 5, 2019

fix compile error caused by bitcoin#9605

89d9521

PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Feb 5, 2019

fix compile error caused by bitcoin#9605

9a30671

PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Feb 5, 2019

implement bitcoin#9605 in Dash code seconds -> milliseconds

df8fab4

str4d mentioned this pull request Feb 7, 2020

Replace boost::function with std::function zcash/zcash#4324

Merged

random-zebra mentioned this pull request Aug 18, 2020

[Wallet][Refactoring] wallet/init refactoring backports PIVX-Project/PIVX#1811

Merged

furszy mentioned this pull request Jan 21, 2021

Revamping block validation interface interaction with wallet PIVX-Project/PIVX#2150

Merged

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use CScheduler for wallet flushing, remove ThreadFlushWalletDB #9605

Use CScheduler for wallet flushing, remove ThreadFlushWalletDB #9605

Uh oh!

TheBlueMatt commented Jan 20, 2017

Uh oh!

sipa left a comment

Uh oh!

sipa Jan 20, 2017

Uh oh!

TheBlueMatt Jan 20, 2017

Uh oh!

TheBlueMatt Jan 20, 2017

Uh oh!

theuni Jan 21, 2017

Uh oh!

sipa commented Jan 20, 2017 via email

Uh oh!

theuni commented Jan 21, 2017

Uh oh!

laanwj Jan 23, 2017

Uh oh!

TheBlueMatt Jan 23, 2017

Uh oh!

laanwj commented Jan 23, 2017

Uh oh!

laanwj commented Jan 26, 2017

Uh oh!

TheBlueMatt commented Feb 8, 2017

Uh oh!

theuni commented Mar 6, 2017

Uh oh!

TheBlueMatt commented Mar 6, 2017

Uh oh!

laanwj commented Mar 7, 2017

Uh oh!

Uh oh!

Use CScheduler for wallet flushing, remove ThreadFlushWalletDB #9605

Use CScheduler for wallet flushing, remove ThreadFlushWalletDB #9605

Uh oh!

Conversation

TheBlueMatt commented Jan 20, 2017

Uh oh!

sipa left a comment

Choose a reason for hiding this comment

Uh oh!

sipa Jan 20, 2017

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt Jan 20, 2017

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt Jan 20, 2017

Choose a reason for hiding this comment

Uh oh!

theuni Jan 21, 2017

Choose a reason for hiding this comment

Uh oh!

sipa commented Jan 20, 2017 via email

Uh oh!

theuni commented Jan 21, 2017

Uh oh!

laanwj Jan 23, 2017

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt Jan 23, 2017

Choose a reason for hiding this comment

Uh oh!

laanwj commented Jan 23, 2017

Uh oh!

laanwj commented Jan 26, 2017

Uh oh!

TheBlueMatt commented Feb 8, 2017

Uh oh!

theuni commented Mar 6, 2017

Uh oh!

TheBlueMatt commented Mar 6, 2017

Uh oh!

laanwj commented Mar 7, 2017

Uh oh!

Uh oh!