Issue for discussion about ways multiprocess functionality could be packaged and released. Trying to figure out which of the 3 release options described below makes the most sense.

Binaries

One goal of the multiprocess project has been to provide minimal binaries that only include node, wallet, or gui code, which spawn or connect to other processes as needed to provide other functionality.

The idea implemented in #10102 is to have 3 binaries:

• bitcoin-node - contains node code and libraries (leveldb), and no wallet or gui code
• bitcoin-wallet - contains wallet code and libraries (sqlite), and no node or gui code
• bitcoin-gui - contains gui code and libraries (qt), and no node or wallet code

Which are built differently than current binaries:

• bitcoind - contains node code+libraries (leveldb) and wallet code+libraries (sqlite)
• bitcoin-qt - contains node code+libraries (leveldb), wallet code+libraries (sqlite), and gui code+libraries (qt)

Original release plan

My original idea implementing this was for there to be a separate multiprocess bitcoin release with the multiprocess binaries.

The main release would be unchanged, including existing bitcoind and bitcoin-qt binaries, but there would be a separate release containing bitcoin-node, bitcoin-wallet and bitcoin-gui binaries. The main release would be available in the normal place like bitcoincore.org/bin/bitcoin-core-27.1/, and the multiprocess release would be available alongside, at bitcoincore.org/bin/bitcoin-core-multiprocess-27.1/ or someplace similar. Then, after a release cycle or two, the multiprocess release could replace the main release if it was working well.

But this approach has some drawbacks, namely that it would be a burden for maintainers to create separate releases and make them accessible, and that it could be confusing for users to encounter two releases with different binaries. So it doesn't seem like this approach is optimal (at least not for general users, maybe it would be ok for miners using the IPC mining interface).

Release options overview

Taking a step back, it seems like there are 3 possible options for releasing multiprocess functionality:

1. Side-release: create a separate multiprocess release for multiprocess binaries, as described above.
2. Side-binaries: add multiprocess binaries (bitcoin-node, bitcoin-wallet, bitcoin-gui) to existing releases alongside existing binaries (bitcoind and bitcoin-qt).
3. Combined binaries: add multiprocess functionality to existing bitcoind and bitcoin-qt binaries that can be enabled/disabled with runtime arguments.
4. Minimally combined binaries: add limited multiprocess functionality to existing bitcoind and bitcoin-qt binaries that can be enabled/disabled with runtime arguments. Difference from option 3 is that IPC side-binaries would also exist and bitcoind and bitcoin-qt would not contain IPC code, but automatically exec() IPC binaries when -ipcbind or other IPC options were enabled.

Release options comparison

From discussion in recent PR's and offline, it seems like approach (3) might be favored, but it is worth considering pros and cons of the three approaches.

1. Side-release

   • Pros:
     • Opt-in: Keeps original binaries unchanged for users who don't want multiprocess functionality.
     • Minimal binaries: Node binary contains only node code not wallet code, gui binary contains no node or wallet code.
   • Cons:
     • Extra work: It's more work for maintainers and signers to create an extra set of packages and make them accessible.
     • Complexity: Posting a separate release could be confusing for users.

2. Side-binaries:

   • Pros:
     • Opt-in and Minimal binaries: same as previous
     • Easy to implement: Implemented in #30975 and basically requires no changes other than flipping cmake options.
   • Cons:
     • Confusing: This would probably be the most confusing approach for users. They would see one release with bitcoind, bitcoin-qt, bitcoin-node, bitcoin-wallet, bitcoin-gui binaries and not be able to easily understand how they are intended to be used.
   • Potential mitigations:

     • We would probably put multiprocess binaries in another folder (like multiprocess/bin/ instead of bin/) so they are clearly labeled and aren't confused with the default binaries.

     • We might want to rethink our approach to packaging binaries to begin with. We are already shipping a substantial collection of binaries: bitcoin-cli, bitcoind, bitcoin-qt, bitcoin-tx, bitcoin-util, bitcoin-wallet, and test_bitcoin which users are expected to call individually, and are probably already somewhat confusing. By contrast, git ships dozens of binaries and nobody is confused by them because they live in a libexec directory and are wrapped with single git command that finds and executes the right ones. Maybe taking git as inspiration we could have add a unified bitcoin command that looks like:

       bitcoin gui [OPTIONS]             # Start a gui
       bitcoin daemon [OPTIONS]          # Start a daemon
       bitcoin rpc [OPTIONS] COMMAND     # Call an rpc
       bitcoin wallet [OPTIONS] COMMAND  # Access a wallet
       bitcoin test [OPTIONS]            # Run tests
       bitcoin help
       # ... and more for bitcoin-util, bitcoin-tx ...

       Update: wrapper command is implemented in #31375

3. Combined binaries:

   • Pros:
     • Compatibility: Users could download and invoke bitcoind and bitcoin-qt binaries as before, not know they support IPC, not use IPC unless it's switched on by default, and there would be runtime options to turn it on or off.
   • Cons:
     • Bigger binaries and more dependencies: Binaries will both contain node, wallet, and IPC code. There will be no non-IPC binaries users can deploy and run separately if they don't use IPC, and no node binary users can deploy and run separately if they don't use the wallet, or want to run the wallet and node on different hosts.
     • Requires code changes: Will require some thinking about how to add runtime options to #10102 to control whether bitcoind should spawn separate wallet process or run wallet code internally, and similarly whether bitcoin-gui should use a separate node process or run node code in the same process. Will also require updating the build system.

4. Minimally combined binaries:

   • Pros:
     • Combines pros listed in option 1 with pros listed in option 3.
   • Cons:
     • Some implementation messiness: A downside of this approach is options will need to be parsed twice. For example bitcoind will need to read arguments and config file to detect if any IPC features were requested, then it will need to exec() bitcoin-node which will read the same options and configuration again. In general this doesn't seem as nice and clean as having a single wrapper binary like multiprocess: Add bitcoin wrapper executable #31375 that calls all the other binaries, and can show help displaying what subcommands are available.

As mentioned, approach (3) seems favored so far, though personally I like approach (2), especially if we could have a unified bitcoin command.

Creating this issue to get feedback and document any decisions we make.

This issue is part of the process separation project.