Skip to content

Conversation

Sjors
Copy link
Owner

@Sjors Sjors commented Jul 17, 2024

This is an alternative approach to #68. Rather than integrating the Template Provider directly into Bitcoin Core, we make it an entirely separate application that communicates via IPC, see design/multiprocess.md.

Usage

Compile as usual, multiprocess is enabled by default.

Alternatively get a recent binary, see (unofficial, by me) releases ending in "ICP".

Start node:

build/src/bitcoin-node -chain=testnet4 -ipcbind=unix

Start the Template Provider:

build/bin/bitcoin-mine -chain=testnet4 -debug=sv2 -loglevel=sv2:trace

See general Stratum v2 usage instructions in bitcoin#29432.

Rationale, PRs this builds on

Upstream PRs used:

There are multiple ways to create such an application. It could be built from scratch in Rust, using bits and pieces of the Stratum Reference Implementation SRI. It could be a c++ application built from scratch, perhaps using some libraries from Bitcoin Core.

But this PR goes about in a slightly different way. Similar to Elements or Knots it's a set of patches on top of the full Bitcoin Core codebase. This allows me to leverage the build system, subclass Transport and use various other useful bits.

This lets me reuse almost all code from the integrated Template Provider bitcoin#29432. Both build direct on:

I modified the guix script to only produce bitcoin-node (for now) and bitcoin-mine binaries.

Goal

The goal is to drop bitcoin-node from the release here. Ideally users would install Bitcoin Core in the manner they're familiar with. They would then install bitcoin-mine separately and it should just work(tm), at most having to add a line to bitcoin.conf to turn IPC on.

This won't work until we ship a release with multiprocess and the mining interface. See bitcoin#30437 for progress towards that.

Rebase hint

Rebase hint:

git reset --hard 2024/07/sv2-tp-common
git fetch ryanofsky
git merge ryanofsky/pr/mine

git cherry-pick sjors/sv2-ipc~12^..sjors/sv2-ipc

@Sjors
Copy link
Owner Author

Sjors commented Jul 17, 2024

@ryanofsky I slightly mangled e373642 because of bitcoin@4e1a434#r1681193702. I'll clean that up later.

@Sjors
Copy link
Owner Author

Sjors commented Jul 17, 2024

Incorporated the changes from bitcoin#30437 (comment) and cleaned up commit history.

@Sjors
Copy link
Owner Author

Sjors commented Jul 18, 2024

@ryanofsky do you have a commit somewhere that adds libmultiprocess to the guix build?

@Sjors Sjors force-pushed the sv2-ipc branch 2 times, most recently from c75792b to 91720b7 Compare July 18, 2024 09:26
@Sjors
Copy link
Owner Author

Sjors commented Jul 18, 2024

Getting a cryptic error: '__NVCC___WORKAROUND_GUARD' is not defined error for the multiprocess job: https://cirrus-ci.com/task/4940413969104896?logs=ci#L2225

Somehow related to the Template Provider depending on boost/multi_index via its use of the mempool.

I can't make sense of the personality(old_personality | ADDR_NO_RANDOMIZE) error thrown by MSAN and TSAN, but it's possible that's related to my self-hosted CI setup and/or the modification in 3319668.

@Sjors Sjors force-pushed the 2024/07/sv2-tp-common branch 2 times, most recently from db58076 to 1ff4660 Compare July 18, 2024 12:24
@Sjors Sjors force-pushed the sv2-ipc branch 3 times, most recently from 4354e96 to 3aa6f0a Compare July 18, 2024 13:11
@Sjors Sjors force-pushed the 2024/07/sv2-tp-common branch from 1ff4660 to ee010f5 Compare July 18, 2024 13:16
@ryanofsky
Copy link

re: #48 (comment)

@ryanofsky do you have a commit somewhere that adds libmultiprocess to the guix build?

I don't, since I haven't really experimented with the guix build. Maybe simplest approach to start with would enable the multiprocess build option and build and package all the multiprocess executables including bitcoin-tp and bitcoin-node. I know in the longer run you want to have separate releases as described in bitcoin#30437 (comment).

re: #48 (comment)

Getting a cryptic error: '__NVCC___WORKAROUND_GUARD' is not defined error for the multiprocess job: https://cirrus-ci.com/task/4940413969104896?logs=ci#L2225

Somehow related to the Template Provider depending on boost/multi_index via its use of the mempool.

This is a -Wundef error caused by bitcoin#29876 that happens when a boost header is included from a source file that is compiled with -I /path/to/boost/headers rather than -isystem /path/to/boost/headers. The fix for this is probably:

-bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags)
+bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags) $(BOOST_CPPFLAGS)

to add the right -isystem option.

I can't make sense of the personality(old_personality | ADDR_NO_RANDOMIZE) error thrown by MSAN and TSAN, but it's possible that's related to my self-hosted CI setup and/or the modification in 3319668.

Not sure about this error. Link seems to be https://cirrus-ci.com/task/4759864851824640?logs=ci#L2486

@Sjors Sjors force-pushed the 2024/07/sv2-tp-common branch 2 times, most recently from 19cc9f1 to dabc32c Compare July 19, 2024 08:59
@Sjors Sjors force-pushed the sv2-ipc branch 3 times, most recently from 6c24b45 to a8bb152 Compare July 19, 2024 13:55
@Sjors
Copy link
Owner Author

Sjors commented Jul 19, 2024

I'm getting the MSAN / TSAN error inconsistently on my self-hosted CI. Does not seem related to this PR. Trying to reproduce in isolation in #51.

TheCharlatan and others added 2 commits August 18, 2025 11:19
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Suggested by Sjors Provoost <sjors@sprovoost.nl> in
bitcoin#30437 (comment)
ryanofsky added a commit to bitcoin/bitcoin that referenced this pull request Aug 20, 2025
ce7d94a doc: add release note (Sjors Provoost)
71f29d4 doc: update build and dependencies docs for IPC (Sjors Provoost)
3cbf747 cmake: set ENABLE_IPC by default (Sjors Provoost)
32a90e1 ci: use bitcoin-node for one depends job (Sjors Provoost)
b333cc1 ci: build one depends job without multiprocess (Sjors Provoost)
16bce9a build: depends makes libmultiprocess by default (Sjors Provoost)

Pull request description:

  Have depends make libmultiprocess by default. This PR causes the following behavior changes:

  1. **bitcoin-node and bitcoin-gui binaries are included in releases**, due to `ENABLE_IPC` option being switched on by default in depends builds
  2. `ENABLE_IPC` is also switched on by default in non-depends builds (instructions updated, #33190 does this as a standalone PR)
  3. Various changes to CI: switching on `ENABLE_IPC` on in most configurations and using `bitcoin-node` binary (`bitcoin -m`) for functional tests in two of them.
  4. The `bitcoin-node` and `bitcoin-gui` are added to `Maintenance.cmake` (since they're now in the release)

  This PR doesn't need to do all of these things at once. However it's is simpler, avoids code churn (especially in CI), and  probably less confusing to make all these changes in the same PR.

  Windows is not supported yet, so `ENABLE_IPC` is off by default for it. It can be enabled after #32387.

  The initial main use case for IPC is to enable experimental support for the Mining IPC interface. A working example of a Stratum v2 Template Provider client using this interface can be found here: Sjors#48.

  See #31756 for discussion of when this should happen. Supersedes #30975.

  ## Wait what, why?

  The [Stratum v2 spec](https://stratumprotocol.org/specification) has been around for a few years now, mostly stable but with [ongoing activity](https://github.com/stratum-mining/sv2-spec/commits/main/) to clarify and fix more subtle issues encountered by implementers. Most of the implementation is built in Rust in a project called the Stratum Reference Implementation ([SRI](https://github.com/stratum-mining/stratum)).

  [Braiins](https://demand.work) added Stratum v2 support to both their (custom) firmware and pool several years ago, though they have fallen behind on recent spec changes (update: it seems they've fixed that). Apparently [new hardware is underway](#31802 (comment)) that supports Stratum v2 without the need for custom firmware.

  [DMND pool](https://www.dmnd.work) is Stratum v2 native from the start and employs several of the SRI developers (they haven't fully launched though). The industry is rather secretive, but apparently [there is more underway](#31802 (comment)).

  What does Bitcoin Core have to do with this? Well, in Stratum v2 jargon we are the Template Provider.

  Or at least, the Template Provider role needs us to make block templates. Initially back in 2023 the plan was to have Bitcoin Core implement this role entirely, see #23049. It would speak the sv2 encrypted message protocol. In fact the spec was designed around this assumption, making sure to only use cryptographic primitives already in our codebase.

  I took over that effort in late 2023, but during 2024 it became quite clear there was [strong resistance](#29432 (review)) to the idea of including all this new code, opening another network ports, etc.

  At the same time there was the long running multiprocess / IPC project #10102, and the idea was born to apply that here: instead of including Stratum v2 specific stuff, we offer a general Mining interface via an IPC connection that can e.g. push out fresh block templates as fees rise above a threshold (something not possible and/or very inefficient with `getblocktemplate`). A client sidecar application then sits between the Stratum v2 world and our node.

  Currently there's only one such sidecar application, maintained by me, and reusing the same codebase from the integrated approach. An attempt has been made to connect to our interface from Rust bitcoin-core/libmultiprocess#174, which would pave the way for SRI include the Template Provider role. Plebhash below indicates he's also working on that: #31802 (comment).

  So with this new approach in mind, between mid 2024 until spring 2025, I introduced a new Mining interface (#30200 - #31785). At the same time Russ Ryanosky worked on more tight integration of [libmultiprocess](https://github.com/bitcoin-core/libmultiprocess), including making it a subtree in #31741. See [design/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/multiprocess.md).

  Meanwhile I've been maintaining a fork of Bitcoin Core that includes the Template Provider, in the original integrated approach (Sjors#68) as well as an IPC + sidecar variant (Sjors#48). I've been shipping [regular releases](https://github.com/Sjors/bitcoin/releases), mostly after bug fixes or major rebases. The SRI team has been testing both variants, though the "official" [instruction on their web page](https://stratumprotocol.org/developers) is to stick to integrated version. Bug reports on [my repo fork](https://github.com/Sjors/bitcoin/issues?q=is%3Aissue) as well as on the [SRI repo](https://github.com/stratum-mining/stratum/issues?q=is%3Aissue%20%20label%3A%22template%20provider%22) are evidence of actual testing happening.

  But as Pavlenex writes below:

  > one recurring feedback I kept getting regardless of the size/type of miner is that the need to run a forked version of Bitcoin Core remains a significant barrier to adoption

  This PR gets rids of that significant barrier. People can download a "pristine" version of Bitcoin Core and the only change is to start it with `bitcoin node -m -ipcconnect=unix` instead of the usual `bitcoind`.

  Once that's released, I can dramatically simplify my sidecar codebase (Sjors#48) by removing pretty much all Bitcoin Core code  that it doesn't need. My plan is to then make that a separate repository, which should be much easier to contribute to. I can then also make (deterministically built) signed releases, while making it clear that sidecar code has nothing to do with Bitcoin Core. Perhaps later on SRI implements the same and I can stop maintaining that project.

  Conceptually the situation will be a lot clearer;
  - today: download forked version of `bitcoind` (or a forked version of `bitcoin-node`, plus `bitcoin-mine`), install SRI stuff
  - tomorrow: download Bitcoin Core v30, install `bitcoin-mine` and SRI
  - future: download Bitcoin Core v30 and SRI

  <details>
  <summary>
  Guix hashes:
  </summary>

  ```
  find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  6dbf29baecb1d1593087ef1306ae7c78aa160c8beb04dc016e02549ae2d6d90d  guix-build-ce7d94a492e6/output/aarch64-linux-gnu/SHA256SUMS.part
  4b465e5e8f9652c176aa57cfe5c289267c28c3a3c684034a9ce471b529b95275  guix-build-ce7d94a492e6/output/aarch64-linux-gnu/bitcoin-ce7d94a492e6-aarch64-linux-gnu-debug.tar.gz
  85bc6fa008b83419d96443d9dcc212b46f0992387fd58fd2dda5da76536ee22c  guix-build-ce7d94a492e6/output/aarch64-linux-gnu/bitcoin-ce7d94a492e6-aarch64-linux-gnu.tar.gz
  5ed9ea52a8bd55361d2d9c01fbd1b25ec9970530c2776e6c1959424ba1689f52  guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/SHA256SUMS.part
  2e483011fac64462d3aa000b577c3c05c825506032d879e39612e096d7a6c65b  guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/bitcoin-ce7d94a492e6-arm-linux-gnueabihf-debug.tar.gz
  7ff1e3ba54944a2be89dd7d68cb91dff6f8950de9d7b521e15dfb746965f81bd  guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/bitcoin-ce7d94a492e6-arm-linux-gnueabihf.tar.gz
  abdf89e701b21b8c1238a8cec46aeaa55e0c3a0b88ad718636e89cde9813ca08  guix-build-ce7d94a492e6/output/arm64-apple-darwin/SHA256SUMS.part
  fb55cff0296cd5474811fe5cedcf28603628729dd085eeefa007c72582459b33  guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-codesigning.tar.gz
  e9aa566b1e79c467d7987b7c68fa608db788e6ddf89c4d90e524cd47b4faaf86  guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-unsigned.tar.gz
  bb428fc62a1230a55f49fa3b5c7ba8d588e8fed491357f890d5a6724a38b14e9  guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-unsigned.zip
  5ef4b75e94b2c8265fbc588bbb42467a84438af969fddac0ea61ced3e4113345  guix-build-ce7d94a492e6/output/dist-archive/bitcoin-ce7d94a492e6.tar.gz
  4f55d56a108c8f312a502cd5dfdf0840b091861a6d502df31caf4636a203697a  guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/SHA256SUMS.part
  66c5b1242c60e37098885a00e24efe19baee4afcd2e3d6407207523d8872f055  guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/bitcoin-ce7d94a492e6-powerpc64-linux-gnu-debug.tar.gz
  d9dbbee7217544eda26e77158cd82caeaef2b40fb9fc7033323e7ffe64264109  guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/bitcoin-ce7d94a492e6-powerpc64-linux-gnu.tar.gz
  d9b808cc5685c819abcebb4ace65f003ebc4bfedf3fca046b34de37994358782  guix-build-ce7d94a492e6/output/riscv64-linux-gnu/SHA256SUMS.part
  eeeea470b1cf76515bfae14c779a3ea356d89f719d1fef1a81e8f0d6b04ab747  guix-build-ce7d94a492e6/output/riscv64-linux-gnu/bitcoin-ce7d94a492e6-riscv64-linux-gnu-debug.tar.gz
  9993da4eb51618b8bd25ec88cc576496720e5589315e9eba6f3ddab25f9c3e60  guix-build-ce7d94a492e6/output/riscv64-linux-gnu/bitcoin-ce7d94a492e6-riscv64-linux-gnu.tar.gz
  1b5a676580e0e79598d182f6ebbb05fb8aee2381edc3c09c042cae2600f448ab  guix-build-ce7d94a492e6/output/x86_64-apple-darwin/SHA256SUMS.part
  9152122d95a34d5df75305c6883c87707e7b09033fffd08e264d703ed177ef12  guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-codesigning.tar.gz
  2793f75730dbef6bdf12b5ed7135e22ed21178abff2926dee92843837d4ab544  guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-unsigned.tar.gz
  e89aafd7e4a330a41f470e8f0a91ea876fad7d19547b404600867413f1a8ccb7  guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-unsigned.zip
  955b27f881927a86da3c566357ad8ca68dbe17e9652bde8c482a57ceacba92cb  guix-build-ce7d94a492e6/output/x86_64-linux-gnu/SHA256SUMS.part
  fd012be97bdf5c75ac12ddef21526bfdb5e17ecc77cde9c34d832194b0dc3293  guix-build-ce7d94a492e6/output/x86_64-linux-gnu/bitcoin-ce7d94a492e6-x86_64-linux-gnu-debug.tar.gz
  0ecf7f80e9049369760d0e27fe6c026391ab25eae0f42336bef43e51a2621726  guix-build-ce7d94a492e6/output/x86_64-linux-gnu/bitcoin-ce7d94a492e6-x86_64-linux-gnu.tar.gz
  2e8085f5fecc246d841b0bf6f28ecd0684a6cee49252fc88c1019d7586c7b7a2  guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/SHA256SUMS.part
  c60041e8137eda352557254c5f67fb83eeb97ecfec342ee528451bd44ee4523a  guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-codesigning.tar.gz
  b1be6b2f4de1c69c2e0e4de6dd97a4891ae9eb50d89435ef47247b5a187915a9  guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-debug.zip
  bfe143f41a20c537145c7044aca889b28efe19072b0150042a3bd865983b3d7e  guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-setup-unsigned.exe
  94a906b83d84db7b25f7e3cfdce2a2030243f2ee6cc70b1fc088459f0b2f382d  guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-unsigned.zip
  ```

  </details>

ACKs for top commit:
  ryanofsky:
    Code review ACK ce7d94a. This was just rebased to fix a conflict since last review.
  josibake:
    ACK ce7d94a
  achow101:
    ACK ce7d94a
  ismaelsadeeq:
    ACK ce7d94a and tested again on macOS by building via depends and source.
  janb84:
    ACK ce7d94a

Tree-SHA512: f7ab72933854e9dfce5746cdf764944bc26eec815f97cd0aa6b54fa499c3cccb1b678861ef5c1c793de28153d46bbb6e4d1b9aa0652163b74262e2d55ec8b813
@Sjors Sjors force-pushed the 2024/07/sv2-tp-common branch from 29a9c8c to 0e01528 Compare August 25, 2025 13:00
@Sjors Sjors force-pushed the 2024/07/sv2-tp-common branch 2 times, most recently from 597513e to ce5f206 Compare August 25, 2025 14:58
Sjors added 3 commits August 25, 2025 17:25
1b8d4a6f1e Merge bitcoin-core/libmultiprocess#194: mpgen: Work around c++20 / capnproto 0.8 incompatibility
f1fad396bf Merge bitcoin-core/libmultiprocess#195: ci: Add openbsd
eed42f210d ci: Bump all tasks to actions/checkout@v5
486a510bbe ci: Remove ancient and problematic -lstdc++fs in mpexample
dd40897efe Add missing thread include
98414e7d28 ci: Add openbsd
dc3ba22046 cmake, doc: Add check for CVE-2022-46149
cb170d4913 Merge bitcoin-core/libmultiprocess#193: build: require CapnProto 0.7.0 or better
8ceeaa6ae4 ci: Add olddeps job to test old dependencies versions
c4cb758ecc mpgen: Work around c++20 / capnproto 0.8 incompatibility
30930dff7b build: require CapnProto 0.7.0 or better

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 1b8d4a6f1e54b92708bd2ad627ec6d440a1daf3d
@Sjors
Copy link
Owner Author

Sjors commented Aug 25, 2025

bitcoin#31802 landed!

Last rebase before I'm going to split off the sv2 functionality into a fresh codebase.

@Sjors
Copy link
Owner Author

Sjors commented Aug 27, 2025

This code is now part of https://github.com/Sjors/sv2-tp; feedback and pull requests are welcome there!

I'll be tagging a release shortly. Usage will be the same as in this PR, except:

  1. the bitcoin-mine command was renamed to sv2-tp
  2. you need to bring your own Bitcoin Core node (I'll make a preview release of v30)

@Sjors Sjors closed this Aug 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants