bitcoin-cli: Add -ipcconnect option #32297

ryanofsky · 2025-04-17T13:55:32Z

This implements an idea from sipa in #28722 (comment) to allow bitcoin-cli to connect to the node via IPC instead of TCP, if the ENABLE_IPC cmake option is enabled and the node has been started with -ipcbind.

This feature can be tested with:

build/bin/bitcoin-node -regtest -ipcbind=unix -debug=ipc
build/bin/bitcoin-cli -regtest -ipcconnect=unix -getinfo

The -ipconnect parameter can also be omitted, since this change also makes bitcoin-cli prefer IPC over HTTP by default, and falling back to HTTP if an IPC connection can't be established.

This PR is part of the process separation project.

DrahtBot · 2025-04-17T13:55:35Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32297.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	janb84, BrandonOdiwuor, pinheadmz
Concept ACK	laanwj, jlest01, yancyribbens, shahsb, sipa

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33229 (multiprocess: Don't require bitcoin -m argument when IPC options are used by ryanofsky)
#33201 (Add functional test for IPC interface by sipa)
#33142 (test: Run bench sanity checks in parallel with functional tests by maflcko)
#33117 (Interfaces: Expose UTXO Snapshot Loading and Add Progress Notifications by D33r-Gee)
#30437 (ipc: add bitcoin-mine test program by ryanofsky)
#29641 (scripted-diff: Use LogInfo over LogPrintf [WIP, NOMERGE, DRAFT] by maflcko)
#29409 (multiprocess: Add capnp wrapper for Chain interface by ryanofsky)
#28690 (build: Introduce internal kernel library by TheCharlatan)
#17783 (common: Disallow calling IsArgSet() on ALLOW_LIST options by ryanofsky)
#17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
#17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
#17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

src/bitcoin-cli.cpp

laanwj · 2025-04-17T14:40:55Z

Concept ACK, it'd be useful to have an utility that can talk to the node through IPC and making bitcoin-cli double for that prevents code duplication.

jlest01 · 2025-04-17T18:28:29Z

Concept ACK, as it can make communication between the CLI and the node simpler and configuration-free.

pinheadmz · 2025-04-17T18:58:45Z

concept ACK, cool idea and will review. Does this close #5029 (adding unix sockets to the bitcoind rpc api) or does it only apply to bitcoin-node? Or is IPC protocol different from RPC and must be defined in the client?

ryanofsky · 2025-04-17T19:16:42Z

concept ACK, cool idea and will review. Does this close #5029 (adding unix sockets to the bitcoind rpc api) or does it only apply to bitcoin-node? Or is IPC protocol different from RPC and must be defined in the client?

It uses capnproto, so it may or may not achieve goals of #5029 depending on what they are. This PR does make it possible to call RPC methods over a unix socket, but in order to do that you need to use capnproto which is not available everywhere. For example there wouldn't be an easy way for our python test framework to use the socket directly unless it depended on https://github.com/capnproto/pycapnp. So another solution that uses a another protocol or just uses http over the socket might be preferable to this approach. But the first commit here could be useful for supporting other approaches as well.

ryanofsky

Updated df40f46 -> 538521a (pr/ipc-cli.2 -> pr/ipc-cli.3, compare) handling -rpcconnect option, adding a test, making various cleanups, adding comments, and fixing CI failures in non-ipc builds
Updated 538521a -> bd29de3 (pr/ipc-cli.3 -> pr/ipc-cli.4, compare) fixing compile error in intermediate commit, https://github.com/bitcoin/bitcoin/actions/runs/14538988101/job/40793056407?pr=32297, fixing functional test failure when socket path is too long https://cirrus-ci.com/task/6103052337283072, and adding new test to cover "bitcoin-cli was not built with IPC support" error

src/bitcoin-cli.cpp

yancyribbens · 2025-04-19T12:37:46Z

Concept ACK

maflcko · 2025-04-21T12:11:10Z

Looks like those are real typos that should be fixed:

LLM Linter (✨ experimental)

Possible typos and grammar issues:

The following typos/grammar issues were found in added lines:

1. Extra “unix’” in the –ipcconnect help text
   Replace
   “…node.sock unix’ but fall back to TCP…”
   with
   “…node.sock’ but fall back to TCP…”

2. Double negative in JSONErrorReply comment
   Replace
   “…when a request cannot not be parsed,…”
   with
   “…when a request cannot be parsed,…”

luke-jr · 2025-04-21T22:37:17Z

This looks like it uses JSON-RPC inside Capnproto? Why not just use straight Capnproto?

ryanofsky · 2025-04-21T23:31:42Z

This looks like it uses JSON-RPC inside Capnproto? Why not just use straight Capnproto?

Yes it is skipping the HTTP part of the protocol but keeping the JSON part. The capnproto interface this exposes looks like:

interface Rpc $Proxy.wrap("interfaces::Rpc") {
    executeRpc @0 (context :Proxy.Context, request :Text, uri :Text, user :Text) -> (result :Text);
}

where request and response above are JSON text strings with the expected JSON-RPC fields. The schema could be unwrapped further, or the data could be sent in some binary json format depending on what your goal is but that interface was the simplest way I could think of implementing this feature.

shahsb

Concept ACK

laanwj · 2025-04-24T12:25:16Z

The schema could be unwrapped further, or the data could be sent in some binary json format depending on what your goal is but that interface was the simplest way I could think of implementing this feature.

Right, Say, representing the entire RPC protocol as cap'n'proto structures directly would be interesting (anologous to how c-lightning has a gRPC version of their RPC), but this would be a huge project for which the first step would be the formal definition of the RPC protocol (including specific-enough types) so that that can be automatically generated.

And this would make it harder to make a cli tool, not easier, because the client needs to know the protocol to convert its command line to suitable calls.

So sending the JSON inside the interface seems like a good, low-impact way to implement this tool.

sipa · 2025-04-30T14:45:47Z

Concept ACK

With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to start a node listening on an IPC socket, instead of needing to choose which node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to pass to it. The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297 (interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in their test setup.

Expose ENABLE_IPC build option to function tests so new tests for bitcoin-mine in bitcoin#30437 and bitcoin-cli in bitcoin#32297 can test IPC-only features.

Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`

With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to start a node listening on an IPC socket, instead of needing to choose which node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to pass to it. The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297 (interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in their test setup.

Expose ENABLE_IPC build option to function tests so new tests for bitcoin-mine in bitcoin#30437 and bitcoin-cli in bitcoin#32297 can test IPC-only features.

Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`

With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to start a node listening on an IPC socket, instead of needing to choose which node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to pass to it. The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297 (interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in their test setup.

…coin` binary test: Provide path to `bitcoin` binary Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`

Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`

With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to start a node listening on an IPC socket, instead of needing to choose which node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to pass to it. The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297 (interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in their test setup.

Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in bitcoin#30437 and `bitcoin-cli` in bitcoin#32297 to find the `bitcoin` binary and call `bitcoin -m` to start nodes with IPC support. This way the new tests can run whenever the ENABLE_IPC build option is enabled, instead of only running when the `BITCOIN_CMD` environment variable is set to `bitcoin -m`

With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to start a node listening on an IPC socket, instead of needing to choose which node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to pass to it. The eliminates boilerplate code bitcoin#30437 (interface_ipc_mining.py), bitcoin#32297 (interface_ipc_cli.py), and bitcoin#33201 (interface_ipc.py) previously needed in their test setup.

Add a MakeBasicInit() function so simpler standalone IPC clients like bitcoin-mine in bitcoin#30437 and bitcoin-cli in bitcoin#32297 that only initiate IPC connections without exposing any IPC interfaces themselves can to avoid needing to implement their own specialized interfaces::Init subclasses.