-
Notifications
You must be signed in to change notification settings - Fork 37.7k
kernel: Remove util/system from kernel library, interface_ui from validation. #27636
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kernel: Remove util/system from kernel library, interface_ui from validation. #27636
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK. I think adding the level of indirection here is right approach, but I left some suggestions below which I think would make the implementation less awkward and more future-proof
0b159b9
to
63e915a
Compare
Thank you for the suggestions @ryanofsky! Updated 0b159b9 -> 63e915a (chainstateRmKernelUiInterface_0 -> chainstateRmKernelUiInterface_1, compare).
|
Concept ACK. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK 63e915a.
The approach implemented here introduces some indirection, which makes the code a bit harder to read.
As for me, it does not look like a problem at all. Good function naming works well :D
In commit 7355679:
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -4659,7 +4659,7 @@ void Chainstate::LoadExternalBlockFile(
}
}
- NotifyHeaderTip(*this);
+ ::NotifyHeaderTip(*this);
if (!blocks_with_unknown_parent) continue;
Also mind considering the IWYU output regarding src/node/validation_notifications.{h,cpp}
?
I added the global scope resolution where it was required, because of the naming conflict with the |
virtual void notifyBlockTip(SynchronizationState state, CBlockIndex* index) = 0; | ||
virtual void notifyHeaderTip(SynchronizationState state, int64_t height, int64_t timestamp, bool presync) = 0; | ||
virtual void showProgress(const std::string& title, int nProgress, bool resume_possible) = 0; | ||
virtual void doWarning(const bilingual_str& warning) = 0; | ||
virtual bool fatalError(const std::string& strMessage, bilingual_str user_message = {}) = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should these methods be non-const
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, as far as I am aware, const
virtual methods are bad practice. See: #25337 (comment).
63e915a
to
97b0e28
Compare
Thank you for the review @hebasto, Updated 63e915a -> 97b0e28 (chainstateRmKernelUiInterface_1 -> chainstateRmKernelUiInterface_2, compare). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: https://api.cirrus-ci.com/v1/task/4542005133443072/logs/ci.log:
|
97b0e28
to
2f9c2d2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review ACK 7d3b350
I'm still familiarising myself with the libbitcoinkernel project so for now I've mostly focused on the code being correct, readable, ... and not so much on architecture (which is arguably the most important thing here) etc. All the changes made sense, and are very well structured and readable.
I've left a few nits (+here) mostly regarding includes, all of them can be safely ignored and definitely shouldn't stand in the way of this PR making progress.
general nit: could we stick to camelcase naming for new functions, e.g. (edit: TheCharlatan pointed me to this section in the developer notes which I wasn't aware of: https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#internal-interface-naming-style)Notifications::headerTip()
-> Notification::HeaderTip()
?
Thank you for the review!
I'll fix these if I have to push again, otherwise will address them in follow ups. |
re-ACK 7d3b350 (no change) 🎋 Show signatureSignature:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…ion code. 6eb33bd kernel: Add fatalError method to notifications (TheCharlatan) 7320db9 kernel: Add flushError method to notifications (TheCharlatan) 3fa9094 scripted-diff: Rename FatalError to FatalErrorf (TheCharlatan) edb55e2 kernel: Pass interrupt reference to chainman (TheCharlatan) e2d680a util: Add SignalInterrupt class and use in shutdown.cpp (TheCharlatan) Pull request description: Get rid of all `ShutdownRequested` calls in validation code by introducing an interrupt object that applications can use to cancel long-running kernel operations. Replace all `AbortNode` calls in validation code with new fatal error and flush error notifications so kernel applications can be notified about failures and choose how to handle them. --- This pull request is part of the `libbitcoinkernel` project #27587 https://github.com/orgs/bitcoin/projects/3 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". The pull request mostly allows dropping the kernel dependency on shutdown.cpp. The only dependency left after this is a `StartShutdown` call which will be removed in followup PR #27711. This PR also drops the last reference to the `uiInterface` global in kernel code. The process of moving the `uiInterface` out of the kernel was started in #27636. This pull request contains a subset of patches originally proposed in #27711. It will be part of a series of changes required to make handling of interrupts (or in other words the current shutdown procedure) in the kernel library more transparent and less reliable on global mutable state. The set of patches contained here was originally proposed by @ryanofsky [here](#27711 (comment)). ACKs for top commit: achow101: light ACK 6eb33bd hebasto: ACK 6eb33bd, I have reviewed the code and it looks OK. ryanofsky: Code review ACK 6eb33bd. No changes since last review other than rebase. Tree-SHA512: 7d2d05fa4805428a09466d43c11ae32946cbb25aa5e741b1eec9cd142e4de4bb311e13ebf1bb125ae490c9d08274f2d56c93314e10f3d69e7fec7445e504987c
The following changes considered: - bitcoin/bitcoin#27419 - bitcoin/bitcoin#27491 - bitcoin/bitcoin#27576 - bitcoin/bitcoin#27636
The following changes considered: - bitcoin/bitcoin#27419 - bitcoin/bitcoin#27491 - bitcoin/bitcoin#27576 - bitcoin/bitcoin#27636
The following changes considered: - bitcoin/bitcoin#27419 - bitcoin/bitcoin#27491 - bitcoin/bitcoin#27576 - bitcoin/bitcoin#27636
Pull request description: Sync with the main repo up to the latest bitcoin/bitcoin@ab42b2e, which includes the recent changes in the CI. There is no downloadable artifacts support for now. It will be done in a separated PR(s). Additionally: - The code was adjusted to reflect changes from [PR27419](bitcoin/bitcoin#27419), [PR27491](bitcoin/bitcoin#27491), [PR27576](bitcoin/bitcoin#27576) and [PR27636](bitcoin/bitcoin#27636). - Fixed `modernize-use-default-member-init` clang-tidy warnings. - The ARM task has been temporarily disabled until the issue with the depends cache is resolved. Guix builds: ``` e92b8c4c3298165edb1a0e85ee516d52c81af1269405dcbc6520e63069de2363 guix-build-b3261144c892/output/aarch64-linux-gnu/SHA256SUMS.part 939c6c002490d5649bdbfabacd20cd2270b41b20b7b3a254c9fcd5780209900d guix-build-b3261144c892/output/aarch64-linux-gnu/bitcoin-b3261144c892-aarch64-linux-gnu-debug.tar.gz b3c1383fb394997378997bdd2933965cf4ecc694143b4703108ff6ecb946696c guix-build-b3261144c892/output/aarch64-linux-gnu/bitcoin-b3261144c892-aarch64-linux-gnu.tar.gz f43fedf3af666d35e83b84e63cfe19f315f74f01296982f47d8c159385c3b03c guix-build-b3261144c892/output/arm-linux-gnueabihf/SHA256SUMS.part 73b89b0487e8eee474a6c9c96ae0e7ad635cccc332fc062eb5d4ff5555356c3e guix-build-b3261144c892/output/arm-linux-gnueabihf/bitcoin-b3261144c892-arm-linux-gnueabihf-debug.tar.gz b4518dd9396f316de8d7de5181b8b5d1083e0afa9081625c37117472d2559380 guix-build-b3261144c892/output/arm-linux-gnueabihf/bitcoin-b3261144c892-arm-linux-gnueabihf.tar.gz 0213e754408e2a032cef61a946354656f5b5f755f85aeac1ce4b37f1d22528e6 guix-build-b3261144c892/output/arm64-apple-darwin/SHA256SUMS.part 11bc1be1f53dad337565f3c556dd69abc2d702a31e661359daad6ff89225c794 guix-build-b3261144c892/output/arm64-apple-darwin/bitcoin-b3261144c892-arm64-apple-darwin-unsigned.dmg 558d8e805420c7a348759df6f559ca349953646aa28840efafe5a3d245ea917f guix-build-b3261144c892/output/arm64-apple-darwin/bitcoin-b3261144c892-arm64-apple-darwin-unsigned.tar.gz e679ce3f1c80aff11a5eab8890efbd0d396a851875fbd6f93f32eef5cdf06813 guix-build-b3261144c892/output/arm64-apple-darwin/bitcoin-b3261144c892-arm64-apple-darwin.tar.gz 0cb346390dc6620593b1af5b6669ddc3c1a8d2219a51b1697747c5ab24069c27 guix-build-b3261144c892/output/dist-archive/bitcoin-b3261144c892.tar.gz ac8bd2d58d9d0ebe2da1c8efa2d57bd97c3ef2b2590c758edbc4919808c528c5 guix-build-b3261144c892/output/powerpc64-linux-gnu/SHA256SUMS.part cdf8252fa8aca6da61ff6926de5c7e2e6560ab046049c84c26ba44823f83236a guix-build-b3261144c892/output/powerpc64-linux-gnu/bitcoin-b3261144c892-powerpc64-linux-gnu-debug.tar.gz 3b8b5f53d365b5bf962ecd7def9f06b6f13af0e5c9ef69c6d028f1ed772459be guix-build-b3261144c892/output/powerpc64-linux-gnu/bitcoin-b3261144c892-powerpc64-linux-gnu.tar.gz b44e688d233dcb46a7d6d0b1d97979335d3cc559d16190cc5cd647add79298d2 guix-build-b3261144c892/output/powerpc64le-linux-gnu/SHA256SUMS.part ae5c19afefd523cdc171a3f9aa9f707870fd99749c01c01166086619dfd95ece guix-build-b3261144c892/output/powerpc64le-linux-gnu/bitcoin-b3261144c892-powerpc64le-linux-gnu-debug.tar.gz bb581b1444fa1686f8889248af13d1859f2915091cd640bc522185d5ad83e13d guix-build-b3261144c892/output/powerpc64le-linux-gnu/bitcoin-b3261144c892-powerpc64le-linux-gnu.tar.gz bdca0a3c19b5a9a5c72b2b43b07050678d960009d3fa80cf7e0689d508346974 guix-build-b3261144c892/output/riscv64-linux-gnu/SHA256SUMS.part b0b9c91abe2ad0b5ab3b0bfd10c90133d8d75b50aef0a6a98ac2c2ae4219eaa8 guix-build-b3261144c892/output/riscv64-linux-gnu/bitcoin-b3261144c892-riscv64-linux-gnu-debug.tar.gz fcce0ea00f1d9df136dd677cbc468183faa92bd4bfcd4a77cd1c70f1b894b5f0 guix-build-b3261144c892/output/riscv64-linux-gnu/bitcoin-b3261144c892-riscv64-linux-gnu.tar.gz 7be84969950bb9570522be5a37551c01698cd3fb65eca3988fc9bd6867460552 guix-build-b3261144c892/output/x86_64-apple-darwin/SHA256SUMS.part 25203f50aa6a344ad1c6c4a44a48082440bb0af9bf38f0d60506569f216d1672 guix-build-b3261144c892/output/x86_64-apple-darwin/bitcoin-b3261144c892-x86_64-apple-darwin-unsigned.dmg 16c5baaf6d00ed43b0611c86c2d4555d500b3896daa1daac6a567bc2611c39f6 guix-build-b3261144c892/output/x86_64-apple-darwin/bitcoin-b3261144c892-x86_64-apple-darwin-unsigned.tar.gz 86662f39c29b013b576e6555ecb6cbbc98eaa08532a541e22a7ed6b1baf87209 guix-build-b3261144c892/output/x86_64-apple-darwin/bitcoin-b3261144c892-x86_64-apple-darwin.tar.gz fbbc0ad2376431fdc5b214fd63f24a6da907d87f6f11e0833def50c0d45772cd guix-build-b3261144c892/output/x86_64-linux-gnu/SHA256SUMS.part cba8d700f746a6063809570e45d6dc3d5e60ad5f1a28e0f41f8beed8b546a7b1 guix-build-b3261144c892/output/x86_64-linux-gnu/bitcoin-b3261144c892-x86_64-linux-gnu-debug.tar.gz 0a32985a1e26e13ce883a85e4a92cc68bf51ce096f2f6d74ea499a9fa662d7d0 guix-build-b3261144c892/output/x86_64-linux-gnu/bitcoin-b3261144c892-x86_64-linux-gnu.tar.gz 0bd4cc64cd6ad733cdef87cd74d5034e79dd250b72795cebf9c2c63500509457 guix-build-b3261144c892/output/x86_64-w64-mingw32/SHA256SUMS.part 6ed8f2e6c6cf1992d156672707cd2c254754051f88223dd052a9cd9078d84789 guix-build-b3261144c892/output/x86_64-w64-mingw32/bitcoin-b3261144c892-win64-debug.zip 1ea6d7660652e20b2b1529e406be1f606745d35f6a179b006335a19a19aa9a5b guix-build-b3261144c892/output/x86_64-w64-mingw32/bitcoin-b3261144c892-win64-setup-unsigned.exe 41b0f8cbac614e8c555921de60b25a73a75e6bed025de98ca40d3db48c5db6b1 guix-build-b3261144c892/output/x86_64-w64-mingw32/bitcoin-b3261144c892-win64-unsigned.tar.gz 5c68d711782e76f9e4be93b5468c505f022b72ca299532b200e58fe1e51343b1 guix-build-b3261144c892/output/x86_64-w64-mingw32/bitcoin-b3261144c892-win64.zip ``` Top commit has no ACKs. Tree-SHA512: dd18cfb2cfd6fd45b35bef8a0397bccc0752ce946b304bae986006ff09a9a183d6222b0f607e4dd3373992814ae0e61d5ba63cb54fef9a288152edef3d7ea81d
…e time (#5693) ## Issue being fixed or feature implemented Some headers include other heavy headers, such as `logging.h`, `tinyformat.h`, `iostream`. These headers are heavy and increase compilation time on scale of whole project drastically because can be used in many other headers. ## What was done? Moved many heavy includes from headers to cpp files to optimize compilation time. In some places added forward declarations if it is reasonable. As side effect removed 2 circular dependencies: ``` "llmq/debug -> llmq/dkgsessionhandler -> llmq/debug" "llmq/debug -> llmq/dkgsessionhandler -> llmq/dkgsession -> llmq/debug" ``` ## How Has This Been Tested? Run build 2 times before refactoring and after refactoring: `make clean && sleep 10s; time make -j18` Before refactoring: ``` real 5m37,826s user 77m12,075s sys 6m20,547s real 5m32,626s user 76m51,143s sys 6m24,511s ``` After refactoring: ``` real 5m18,509s user 73m32,133s sys 6m21,590s real 5m14,466s user 73m20,942s sys 6m17,868s ``` ~5% of improvement for compilation time. That's not huge, but that's worth to get merged There're several more refactorings TODO but better to do them later by backports: - bitcoin#27636 - bitcoin#26286 - bitcoin#27238 - and maybe this one: bitcoin#28200 ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone
Summary: This commit is part of the libbitcoinkernel project and seeks to remove the ChainstateManager's and, more generally, the kernel library's dependency on interface_ui with options methods in this and the following few commits. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. Define a new kernel notification class with virtual methods for notifying about internal kernel events. Create a new file in the node library for defining a function creating the default set of notification methods such that these do not need to be re-defined all over the codebase. As a first step, add a `blockTip` method, wrapping `uiInterface.NotifyBlockTip`. This is a backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@447761c Test Plan: With bitcoin-chainstate: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16204
Summary: This commit is part of the libbitcoinkernel project and seeks to remove the ChainstateManager's and, more generally, the kernel library's dependency on interface_ui with options methods in this and the following few commits. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@84d7145 Depends on D16204 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16205
Summary: This commit is part of the libbitcoinkernel project and seeks to remove the ChainstateManager's and, more generally, the kernel library's dependency on interface_ui with options methods in this and the following few commits. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@4452707 Depends on D16205 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D16206
Summary: This commit is part of the libbitcoinkernel project and seeks to remove the ChainstateManager's and, more generally, the kernel library's dependency on interface_ui with options methods in this and the following few commits. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. The AlertNotify function is moved out of the validation.cpp file, which removes its dependency on interface_ui as well as util/system This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@f871c69 Depends on D16206 Notes: - I removed the util/system include from validation in D16151, but it was actually needed for `void runCommand(const std::string &strCommand)`. After this diff, we actually no longer need it in validation.cpp - we don't use AlertNotify for the same thing, so there are minor differences: we never backported [[bitcoin/bitcoin#10464 | core#10464]], we don't use a bilingual_str, we don't call SetMiscWarning so we don't need warning.h in kernel_notification.cpp Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16207
Summary: With the previous move of AlertNotify out of the validation file, and thus out of the kernel library, ScheduleBatchPriority is the last remaining function used by the kernel library from util/system. Move it to its own file, such that util/system can be moved out of the util library in the following few commits. Moving util/system out of the kernel library removes further networking as well as shell related code from it. This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@9ec5da3 Depends on D16207 Test Plan: With bitcoin-chainstate `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16208
Summary: It makes sense to keep util::insert and util::AnyPtr functions in util/, not common/ as these are header-only template functions with no dependencies, and there's no particular reason they shouldn't be used by kernel code even if they aren't used right now. This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@44de325 Depends on D16210 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16213
Summary: It makes sense to keep util::insert and util::AnyPtr functions in util/, not common/ as these are header-only template functions with no dependencies, and there's no particular reason they shouldn't be used by kernel code even if they aren't used right now. This is a partial backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@7eee356 Depends on D16213 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16214
Summary: Since the kernel library no longer depends on the system file, move it to the common library instead in accordance to the diagram in doc/design/libraries.md. This is a move-only commit. This concludes backport of [[bitcoin/bitcoin#27636 | core#27636]] bitcoin/bitcoin@7d3b350 Depends on D16216 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D16217
This pull request is part of the
libbitcoinkernel
project #27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".It removes the kernel library's dependency on
util/system
andinterface_ui
.util/system
contains networking and shell-related code that should not be part of the kernel library. The following pull requests preparedutil/system
for this final step: #27419 #27254 #27238.interface_ui
defines functions for a more general node interface and has a dependency onboost/signals2
. After applying the patches from this pull request, the kernel's reliance on boost is down toboost::multiindex
.The approach implemented here introduces some indirection, which makes the code a bit harder to read. Any suggestions for improving or reworking this pull request to make it more concise, or even reworking it into a more proper interface, are appreciated.