-
Notifications
You must be signed in to change notification settings - Fork 37.7k
[kernel 3b/n] Decouple {Dump,Load}Mempool
from ArgsManager
#25487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[kernel 3b/n] Decouple {Dump,Load}Mempool
from ArgsManager
#25487
Conversation
f20bff9
to
089d48e
Compare
Following up on #25290 (comment), wondering what clock I should be using there? SteadyClock? NodeClock? Also, could you sanity check my usage of the new clock facilities in 0ea35ed? Thanks! |
Jup, steady clock is fine, given that previously system clock was used and the result is only used for logging a duration (time difference).
Edit: See also #25499 |
@MarcoFalke Okay, I'll use |
The timestamps for mempool entries are mockable (GetTime), thus they are (supposed to be) using NodeClock. |
d43259e
to
05ff88a
Compare
Push 05ff88a:
I think this PR is ready for review! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left some nits (feel free to ignore)
05ff88a
to
ad862b8
Compare
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
ad862b8
to
e8434c8
Compare
e8434c8
to
ddf4920
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept + Approach ACK
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from db_wrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
The libbitcoin_kernel library should should not be trying to read options from the command line or make ArgsManager calls. Instead, it should get instead get options from simple function arguments and options structs. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager uses from kernel modules. This commit does not change behavior in any way.
Code in the libbitcoin_kernel library should not be calling ArgsManager methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager calls from kernel modules. This PR does not change behavior in any way.
Code in the libbitcoin_kernel library should not be calling ArgsManager methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager calls from kernel modules. This PR does not change behavior in any way.
Code in the libbitcoin_kernel library should not be calling ArgsManager methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager calls from kernel modules. This PR does not change behavior in any way.
Code in the libbitcoin_kernel library should not be calling ArgsManager methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This commit removes gArgs accesses from dbwrapper and txdb modules by defining appropriate options structs, and is a followup to PR's bitcoin#25290 bitcoin#25487 bitcoin#25527 which remove other ArgsManager calls from kernel modules. This PR does not change behavior in any way.
…and txdb aadd7c5 refactor, validation: Add ChainstateManagerOpts db options (Ryan Ofsky) 0352258 refactor, txdb: Use DBParams struct in CBlockTreeDB (Ryan Ofsky) c00fa1a refactor, txdb: Add CoinsViewOptions struct (Ryan Ofsky) 2eaeded refactor, dbwrapper: Add DBParams and DBOptions structs (Ryan Ofsky) Pull request description: Code in the libbitcoin_kernel library should not be calling `ArgsManager` methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This PR removes `gArgs` accesses from `dbwrapper` and `txdb` modules by defining appropriate options structs, and is a followup to PR's #25290 #25487 #25527 which remove other `ArgsManager` calls from kernel modules. This PR does not change behavior in any way. It is a simpler alternative to #25623 because the only thing it does is remove `gArgs` references from kernel code. It avoids other unnecessary changes like adding options to the kernel API (they can be added separately later). ACKs for top commit: TheCharlatan: Code review ACK aadd7c5 achow101: ACK aadd7c5 furszy: diff ACK aadd7c5 Tree-SHA512: 46dfd5d99ab3110492e7bba97a87122c831b8344caaf7dd2ebdb6e0ad6aa9174d4d1832d6f3a7465eda9294fe50defaa3c000afbbddc4e72838687df09a63ffd
…ystem.h aaced56 refactor: Move error() from util/system.h to logging.h (Ben Woosley) e7333b4 refactor: Extract util/exception from util/system (Ben Woosley) Pull request description: This pull request is part of the `libbitcoinkernel` project bitcoin/bitcoin#24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". These commits were originally authored by empact and are taken from their parent PR #25152. #### Context There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (bitcoin/bitcoin#25290, bitcoin/bitcoin#25487, bitcoin/bitcoin#25527, bitcoin/bitcoin#25862, bitcoin/bitcoin#26177, and bitcoin/bitcoin#27125). The `ArgsManager` is defined in `system.h`. #### Changes Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving some logging functions out of the `system.*` files. Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed. ACKs for top commit: MarcoFalke: re-ACK aaced56 🐍 Tree-SHA512: cb39f4cb7a77e7dc1887b1cbf340d53decab8880fc00878a2f12dc627fe67245b4aafd4cc31a9eab0fad1e5bb5d0eb4cdb8d501323ca200fa6ab7b201ae34aea
…arams functionality to kernel b3e78dc refactor: Don't use global chainparams in chainstatemanager method (TheCharlatan) 382b692 Split non/kernel chainparams (Carl Dong) edabbc7 Add factory functions for Main/Test/Sig/Reg chainparams (Carl Dong) d938098 Remove UpdateVersionBitsParameters (Carl Dong) 84b8578 Decouple RegTestChainParams from ArgsManager (Carl Dong) 76cd4e7 Decouple SigNetChainParams from ArgsManager (Carl Dong) Pull request description: This pull request is part of the `libbitcoinkernel` project bitcoin/bitcoin#24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". dongcarl is the original author of this patchset, these commits were taken from https://github.com/dongcarl/bitcoin/tree/2022-03-libbitcoinkernel-chainparams-args-only. #### Context The bitcoin kernel library currently relies on code containing user configurations through the `ArgsManager`. This is not optimal, since as a stand-alone library it should not rely on bitcoind's argument parsing logic. Instead, its interfaces should accept control and options structs that control the kernel library's desired configuration. Similar work towards decoupling the `ArgsManager` from the kernel has been done in bitcoin/bitcoin#25290, bitcoin/bitcoin#25487, bitcoin/bitcoin#25527 and bitcoin/bitcoin#25862. #### Changes By moving the `CChainParams` class definition into the kernel and giving it new factory functions `CChainParams::{RegTest,SigNet,Main,TestNet}`it can be constructed without an `ArgsManager` reference, unlike the current factory function `CreateChainParams`. The first few commits remove uses of `ArgsManager` within `CChainParams`. Then the `CChainParams` definition is moved to a new file in the `kernel/` subdirectory. ACKs for top commit: MarcoFalke: re-ACK b3e78dc 🛁 ryanofsky: Code review ACK b3e78dc. Only changes since last review were recent review suggestions. ajtowns: ACK b3e78dc Tree-SHA512: 3835aca1d3e3c75cc3303dd584bab3a77e58f6c678724a5e359fe4b0e17e0763a00931ee6191f516b9fde50496f59cc691f0709c0254206db3863bbf7ab2cacd
00e9b97 refactor: Move fs.* to util/fs.* (TheCharlatan) 106b46d Add missing fs.h includes (TheCharlatan) b202b3d Add missing cstddef include in assumptions.h (TheCharlatan) 18fb363 refactor: Extract util/fs_helpers from util/system (Ben Woosley) Pull request description: This pull request is part of the `libbitcoinkernel` project bitcoin/bitcoin#24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152. #### Context There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (bitcoin/bitcoin#25290, bitcoin/bitcoin#25487, bitcoin/bitcoin#25527, bitcoin/bitcoin#25862, bitcoin/bitcoin#26177, and bitcoin/bitcoin#27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in bitcoin/bitcoin#27238. #### Changes Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files. There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory. Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed. ACKs for top commit: hebasto: ACK 00e9b97 Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
…rom blockstorage 5ff63a0 refactor, blockstorage: Replace stopafterblockimport arg (TheCharlatan) 18e5ba7 refactor, blockstorage: Replace blocksdir arg (TheCharlatan) 02a0899 refactor, BlockManager: Replace fastprune from arg with options (TheCharlatan) a498d69 refactor/iwyu: Complete includes for blockmanager_args (TheCharlatan) f0bb102 refactor: Move functions to BlockManager methods (TheCharlatan) cfbb212 zmq: Pass lambda to zmq's ZMQPublishRawBlockNotifier (TheCharlatan) 8ed4ff8 refactor: Declare g_zmq_notification_interface as unique_ptr (TheCharlatan) Pull request description: The libbitcoin_kernel library should not rely on the `ArgsManager`, but rather use option structs that can be passed to the various classes it uses. This PR removes reliance on the `ArgsManager` from the `blockstorage.*` files. Like similar prior work, it uses the options struct in the `BlockManager` that can be populated with `ArgsManager` values. Some related prior work: bitcoin/bitcoin#26889 bitcoin/bitcoin#25862 bitcoin/bitcoin#25527 bitcoin/bitcoin#25487 Related PR removing blockstorage globals: bitcoin/bitcoin#25781 ACKs for top commit: ryanofsky: Code review ACK 5ff63a0. Since last ACK just added std::move and fixed commit title. Sorry for the noise! mzumsande: Code Review ACK 5ff63a0 Tree-SHA512: 4bde8fd140a40b97eca923e9016d85dcea6fad6fd199731f158376294af59c3e8b163a0725aa47b4be3519b61828044e0a042deea005e0c28de21d8b6c3e1ea7
…base from kernel library db77f87 scripted-diff: move settings to common namespace (TheCharlatan) c27e4bd move-only: Move settings to the common library (TheCharlatan) c2dae5d kernel: Remove chainparams, chainparamsbase, args, settings from kernel library (TheCharlatan) 05870b1 refactor: Remove gArgs access from validation.cpp (TheCharlatan) 8789b11 refactor: Add path argument to FindSnapshotChainstateDir (TheCharlatan) ef95be3 refactor: Add stop_at_height option in ChainstateManager (TheCharlatan) Pull request description: This pull request is part of the `libbitcoinkernel` project #27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". --- This completes the removal of the node's chainparams, chainparamsbase, args and settings files and their respective classes from the kernel library. This is the last pull request in a long series working towards decoupling the `ArgsManager` and the `gArgs` global from kernel code. These prior pull requests are: #26177 #27125 #25527 #25487 #25290 ACKs for top commit: MarcoFalke: lgtm ACK db77f87 🍄 hebasto: ACK db77f87, I have reviewed the code and it looks OK. ryanofsky: Code review ACK db77f87. Looks great! Tree-SHA512: cbfbd705d056f2f10f16810d4f869eb152362fff2c5ddae5e1ac6785deae095588e52ad48b29d921962b085e51de1e0ecab6e50f46149ffe3c16250608a2c93a
This is part of the
libbitcoinkernel
project: #24303, https://github.com/bitcoin/bitcoin/projects/18This PR moves
{Dump,Load}Mempool
into its ownkernel/mempool_persist
module and introducesArgsManager
node::
helpers innode/mempool_persist_args
to remove the scattered calls toGetBoolArg("-persistmempool", DEFAULT_PERSIST_MEMPOOL)
.More context can be gleaned from the commit messages.
One thing I was reflecting on as I wrote this was that in the long run, I think we should probably invert the validation <-> mempool relationship. Instead of mempool not depending on validation, it might make more sense to have validation not depend on mempool. Not super urgent since
libbitcoinkernel
will include both validation and mempool, but perhaps something for the future.