Skip to content

Conversation

TheCharlatan
Copy link
Contributor

@TheCharlatan TheCharlatan commented Mar 10, 2023

This pull request is part of the libbitcoinkernel project #24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". These commits were originally authored by empact and are taken from their parent PR #25152.

Context

There is an ongoing effort to decouple the ArgsManager used for command line parsing user-provided arguments from the libbitcoinkernel library (#25290, #25487, #25527, #25862, #26177, and #27125). The ArgsManager is defined in system.h.

Changes

Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on system.h (and thus the ArgsManager definition) by moving some logging functions out of the system.* files.

Further commits splitting more functionality out of system.h are still in #25152 and will be submitted in separate PRs once this PR has been processed.

@DrahtBot
Copy link
Contributor

DrahtBot commented Mar 10, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK MarcoFalke
Concept ACK fanquake
Stale ACK davidgumberg

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #26177 (refactor / kernel: Move non-gArgs chainparams functionality to kernel by TheCharlatan)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@TheCharlatan TheCharlatan marked this pull request as draft March 10, 2023 11:18
@TheCharlatan TheCharlatan marked this pull request as ready for review March 10, 2023 12:34
@fanquake
Copy link
Member

Concept ACK

@davidgumberg
Copy link
Contributor

ACK 485ea8d

Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review ACK 4aa0ec6 🐸

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: review ACK 4aa0ec6a8617ac77a19e49d8cf081b7a7f6404ae 🐸
Q9UDjbl3uLbiI3hg8GAjCLa+KcpMXFHY4zmCz2T39dQ6BsSrY7132Xgo9wCcDbaRQQsRJs5AfEjE9uCDi9P0AA==

Empact added 2 commits March 13, 2023 17:09
This is a minimal extraction of a single function, but also the only use
of std::exception in util/system.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
error is a low-level function with a sole dependency on LogPrintf, which
is defined in logging.h

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
@TheCharlatan
Copy link
Contributor Author

Updated 4aa0ec6 -> aaced56 (splitSystemLogging_0 -> splitSystemLogging_1, compare) to fix header inclusion order addressing @MarcoFalke's #27238 (review).

@maflcko
Copy link
Member

maflcko commented Mar 13, 2023

re-ACK aaced56 🐍

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: re-ACK aaced5633b81b2f08b993106a527e2a0e1d663c1 🐍
qOnBQIyWxM4Wdga+t5Kp/TH3mEC8A8Rq0S+G4hMBYPBENLw7mwmhC9r3o1wUw9NWbmakHCuycScGpqGf0603Bg==

@fanquake fanquake merged commit b175bdb into bitcoin:master Mar 14, 2023
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Mar 14, 2023
@Empact
Copy link
Contributor

Empact commented Mar 14, 2023

Thanks @TheCharlatan for picking this up. 👍

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Mar 15, 2023
fanquake added a commit to bitcoin-core/gui that referenced this pull request Apr 3, 2023
00e9b97 refactor: Move fs.* to util/fs.* (TheCharlatan)
106b46d Add missing fs.h includes (TheCharlatan)
b202b3d Add missing cstddef include in assumptions.h (TheCharlatan)
18fb363 refactor: Extract util/fs_helpers from util/system (Ben Woosley)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project bitcoin/bitcoin#24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152.

  #### Context

  There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (bitcoin/bitcoin#25290, bitcoin/bitcoin#25487, bitcoin/bitcoin#25527, bitcoin/bitcoin#25862, bitcoin/bitcoin#26177, and bitcoin/bitcoin#27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in bitcoin/bitcoin#27238.

  #### Changes

  Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files.

  There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory.

  Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed.

ACKs for top commit:
  hebasto:
    ACK 00e9b97

Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
fanquake added a commit to bitcoin-core/gui that referenced this pull request May 30, 2023
…ibrary, interface_ui from validation.

7d3b350 refactor: Move system from util to common library (TheCharlatan)
7eee356 refactor: Split util::AnyPtr into its own file (TheCharlatan)
44de325 refactor: Split util::insert into its own file (TheCharlatan)
9ec5da3 refactor: Move ScheduleBatchPriority to its own file (TheCharlatan)
f871c69 kernel: Add warning method to notifications (TheCharlatan)
4452707 kernel: Add progress method to notifications (TheCharlatan)
84d7145 kernel: Add headerTip method to notifications (TheCharlatan)
447761c kernel: Add notification interface (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project bitcoin/bitcoin#27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".

  ---

  It removes the kernel library's dependency on `util/system` and `interface_ui`. `util/system` contains networking and shell-related code that should not be part of the kernel library. The following pull requests prepared `util/system` for this final step: bitcoin/bitcoin#27419 bitcoin/bitcoin#27254 bitcoin/bitcoin#27238.

  `interface_ui` defines functions for a more general node interface and has a dependency on `boost/signals2`. After applying the patches from this pull request, the kernel's reliance on boost is down to `boost::multiindex`.

  The approach implemented here introduces some indirection, which makes the code a bit harder to read. Any suggestions for improving or reworking this pull request to make it more concise, or even reworking it into a more proper interface, are appreciated.

ACKs for top commit:
  MarcoFalke:
    re-ACK 7d3b350 (no change) 🎋
  stickies-v:
    Code Review ACK 7d3b350
  hebasto:
    re-ACK 7d3b350, only last two commits dropped since my [recent](bitcoin/bitcoin#27636 (review)) review.

Tree-SHA512: c8cfc698dc9d78e20191c444708f2d957501229abe95e5806106d1126fb9c5fbcee686fb55645658c0107ce71f10646f37a2fdf7fde16bbf22cbf1ac885dd08d
PastaPastaPasta pushed a commit to dashpay/dash that referenced this pull request Nov 17, 2023
…e time (#5693)

## Issue being fixed or feature implemented
Some headers include other heavy headers, such as `logging.h`,
`tinyformat.h`, `iostream`. These headers are heavy and increase
compilation time on scale of whole project drastically because can be
used in many other headers.

## What was done?
Moved many heavy includes from headers to cpp files to optimize
compilation time.
In some places  added forward declarations if it is reasonable.

As side effect removed 2 circular dependencies:
```
"llmq/debug -> llmq/dkgsessionhandler -> llmq/debug"
"llmq/debug -> llmq/dkgsessionhandler -> llmq/dkgsession -> llmq/debug"
```


## How Has This Been Tested?
Run build 2 times before refactoring and after refactoring: `make clean
&& sleep 10s; time make -j18`

Before refactoring:
```
real    5m37,826s
user    77m12,075s
sys     6m20,547s

real    5m32,626s
user    76m51,143s
sys     6m24,511s
```

After refactoring:
```
real    5m18,509s
user    73m32,133s
sys     6m21,590s

real    5m14,466s
user    73m20,942s
sys     6m17,868s
```

~5% of improvement for compilation time. That's not huge, but that's
worth to get merged

There're several more refactorings TODO but better to do them later by
backports:
 - bitcoin#27636
 - bitcoin#26286
 - bitcoin#27238
 - and maybe this one: bitcoin#28200


## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
@bitcoin bitcoin locked and limited conversation to collaborators Mar 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Status: Done or Closed or Rethinking
Development

Successfully merging this pull request may close these issues.

6 participants