Stratum v2 connman #30332

Sjors · 2024-06-25T10:50:04Z

Based on #30315 and #30205, parent PR #29432.

This PR introduces Sv2Connman which is somewhat similar to CConnman. It uses the Sv2Transport introduced in #30315 to enable incoming connections from other Stratum v2 roles.

It's main target user is the Template Provider role introduced in #29432.

There may be other Stratum v2 roles we want to support in the future.

A remaining issue is that the code in ThreadSv2Handler reuses a lot of code from CConnman::SocketHandlerConnected. I'd like to find a way to extract this common functionality and put it somewhere else.

TODO:

use mock sockets from test: add mocked Sock that can read/write custom data and/or CNetMessages #30205
make CConnman::SocketHandlerConnected (partially) reusable

DrahtBot · 2024-06-25T10:50:07Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#30205 (test: add mocked Sock that can read/write custom data and/or CNetMessages by vasild)
#26812 (test: add end-to-end tests for CConnman and PeerManager by vasild)
#26697 (logging: use bitset for categories by LarryRuane)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot · 2024-06-25T11:46:58Z

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/26648048745}

…t rpc bug fix a74b0f9 Have testBlockValidity hold cs_main instead of caller (Sjors Provoost) f6dc6db refactor: use CHECK_NONFATAL to avoid single-use symbol (Sjors Provoost) 5fb2b70 Drop unneeded lock from createNewBlock (Sjors Provoost) 75ce763 refactor: testBlockValidity make out argument last (Sjors Provoost) 83a9bef Add missing include for mining interface (Sjors Provoost) Pull request description: Followups from #30200 Fixes: - `std::unique_ptr` needs `#include <memory>` (noticed while working on #30332, which has fewer includes than its parent PR that I originally tested with) - Drop lock from createNewBlock that was spuriously added - Have testBlockValidity hold cs_main instead of caller (also fixes a race condition in test-only code) Refactor: - Use CHECK_NONFATAL to avoid single-use symbol (refactor) - move output argument `state` to the end of `testBlockValidity`, see #30200 (comment) ACKs for top commit: AngusP: Code Review ACK a74b0f9 itornaza: Tested ACK a74b0f9 ryanofsky: Code review ACK a74b0f9. Just new error string is added since last review, and a commit message was updated Tree-SHA512: 805e133bb59303fcee107d6f02b3e2761396c290efb731a85e6a29ae56b4b1b9cd28ada9629e979704dcfd98cf35034e7e6b618e29923049eb1eca2f65630e41

DrahtBot · 2024-07-01T19:14:00Z

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/26898716492}

DrahtBot · 2024-07-02T18:37:30Z

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/26955543098}

TheBlueMatt · 2024-07-11T15:10:09Z

This seems like a lot of additional code that is very Sv2-specific. Is there no way to DRY a lot of this code up with connman itself? @pinheadmz may also have some thoughts here as I believe he's looking at replacing libevent with stuff that uses the common Core socket code anyway.

Sjors · 2024-07-11T15:22:55Z

DRY a lot of this code up with connman itself?

Yes, I think the key is to make the copy-pasted parts of CConnman::SocketHandlerConnected reusable.

pinheadmz · 2024-07-12T18:00:00Z

I am working on a libevent-replacing HTTP server using netbase primitives and yeah, I have some functions that look a lot like the sv2connamn in this branch.

I think it would be cool if possible to abstract the mechanisms in ThreadSocketHandler to work on "abstract clients" and I'd be happy to review and collaborate on that.

Sjors · 2024-07-15T11:41:57Z

Removed the accidentally included src/common/sv2_messages.cpp.

src/common/sv2_connman.cpp

Sjors · 2024-07-17T07:21:04Z

Now using mock sockets #30205, thanks @vasild!

Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

This commit adds the simplest stratum v2 message. The remaining messages are introduced in later commits. Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

This allows us to subclass Transport.

Implemented starting from a copy of V2Transport and the V2TransportTester, modifying it to fit Stratum v2 and Noise Protocol requirements. Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com> Co-Authored-By: Fi3

Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

…lass This allows reusing them in other mocked implementations. Also move the implementation (method definitions) to `test/util/net.cpp` to make the header `test/util/net.h` easier to follow.

…o it And also allows gradually providing the data to be returned by `Recv()` and sending and receiving net messages (`CNetMessage`).

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

Sjors · 2024-07-19T19:10:02Z

I moved this PR to my own fork at Sjors#50 to reduce CI load, which apparently was becoming too much: #30475 (comment)

maflcko · 2024-07-22T09:22:05Z

reduce CI load

Some more machines were added a few weeks back to deal with increased overall activity, so I think the load should be handleable now, unless someone pushes to 10 different pull requests every 20 minutes in a loop.

Looks like you fixed it in the meantime, but if CI load comes up again, you may create an issue so that it can be fixed.

This was referenced Jun 25, 2024

Stratum v2 Template Provider (take 3) #29432

Closed

Mining interface followups, reduce cs_main locking, test rpc bug fix #30335

Merged

Sjors force-pushed the 2024/06/sv2_connection branch from 5a5f754 to 89b876c Compare June 25, 2024 11:46

DrahtBot added the CI failed label Jun 25, 2024

DrahtBot removed the CI failed label Jun 25, 2024

This was referenced Jun 26, 2024

kernel: De-globalize validation caches #30141

Merged

Testnet4 including PoW difficulty adjustment fix #29775

Merged

Broadcast own transactions only via short-lived Tor or I2P connections #29415

Open

logging: use bitset for categories #26697

Closed

Sjors force-pushed the 2024/06/sv2_connection branch 2 times, most recently from 5dd22ee to b0b4851 Compare June 28, 2024 15:23

DrahtBot mentioned this pull request Jun 28, 2024

Stratum v2 Noise Protocol #29346

Closed

Sjors mentioned this pull request Jul 1, 2024

test: add mocked Sock that can read/write custom data and/or CNetMessages #30205

Merged

Sjors force-pushed the 2024/06/sv2_connection branch from b0b4851 to 10e8373 Compare July 1, 2024 15:28

DrahtBot added the CI failed label Jul 1, 2024

Sjors force-pushed the 2024/06/sv2_connection branch from 10e8373 to 45908be Compare July 2, 2024 08:21

DrahtBot removed the CI failed label Jul 2, 2024

glozow added the Mining label Jul 2, 2024

Sjors force-pushed the 2024/06/sv2_connection branch 3 times, most recently from 1bef7f4 to 441b081 Compare July 2, 2024 18:37

DrahtBot added the CI failed label Jul 2, 2024

Sjors force-pushed the 2024/06/sv2_connection branch from 441b081 to 4cc9cfe Compare July 2, 2024 18:49

Sjors changed the title ~~Stratum v2 connectivity~~ Stratum v2 connman Jul 2, 2024

Sjors force-pushed the 2024/06/sv2_connection branch from c1509ca to a3a0637 Compare July 11, 2024 10:35

Sjors force-pushed the 2024/06/sv2_connection branch from a3a0637 to 0459ea8 Compare July 11, 2024 14:14

DrahtBot removed the CI failed label Jul 11, 2024

Sjors mentioned this pull request Jul 13, 2024

ipc: add bitcoin-mine test program #30437

Open

Sjors force-pushed the 2024/06/sv2_connection branch from 0459ea8 to 04e89ba Compare July 15, 2024 11:41

vasild reviewed Jul 16, 2024

View reviewed changes

src/common/sv2_connman.cpp Outdated Show resolved Hide resolved

Sjors force-pushed the 2024/06/sv2_connection branch from 04e89ba to 648f793 Compare July 17, 2024 07:19

This was referenced Jul 17, 2024

Stratum v2 Template Provider common functionality #30475

Closed

Stratum v2 Template Provider via IPC interface (multiprocess) Sjors/bitcoin#48

Closed

DrahtBot mentioned this pull request Jul 18, 2024

test: add end-to-end tests for CConnman and PeerManager #26812

Open

Sjors and others added 10 commits July 19, 2024 09:00

Add sv2 log category for Stratum v2

59eacd2

Add sv2 noise protocol

f69fd62

Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

Add sv2 message CoinbaseOutputDataSize

176795a

This commit adds the simplest stratum v2 message. The remaining messages are introduced in later commits. Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

Convert between Sv2NetMsg and CSerializedNetMsg

c4f2ef9

This allows us to subclass Transport.

Introduce Sv2Transport

938570a

Implemented starting from a copy of V2Transport and the V2TransportTester, modifying it to fit Stratum v2 and Noise Protocol requirements. Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com> Co-Authored-By: Fi3

Add sv2 SETUP_CONNECTION messages

750f75a

Co-Authored-By: Christopher Coverdale <chris.coverdale24@gmail.com>

Add strings for Sv2MsgType

fc81013

test: put the generic parts from StaticContentsSock into a separate c…

88e7fbd

…lass This allows reusing them in other mocked implementations. Also move the implementation (method definitions) to `test/util/net.cpp` to make the header `test/util/net.h` easier to follow.

test: add a mocked Sock that allows inspecting what has been Send() t…

d1e3e3b

…o it And also allows gradually providing the data to be returned by `Recv()` and sending and receiving net messages (`CNetMessage`).

Add Sv2Connman

3cf779b

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

Sjors force-pushed the 2024/06/sv2_connection branch from 648f793 to 3cf779b Compare July 19, 2024 07:10

Sjors mentioned this pull request Jul 19, 2024

Stratum v2 connman Sjors/bitcoin#50

Closed

3 tasks

Sjors closed this Jul 19, 2024

bitcoin locked and limited conversation to collaborators Jul 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Stratum v2 connman #30332

Stratum v2 connman #30332

Uh oh!

Sjors commented Jun 25, 2024 •

edited

Loading

Uh oh!

DrahtBot commented Jun 25, 2024 •

edited

Loading

Uh oh!

DrahtBot commented Jun 25, 2024

Uh oh!

DrahtBot commented Jul 1, 2024

Uh oh!

DrahtBot commented Jul 2, 2024

Uh oh!

TheBlueMatt commented Jul 11, 2024

Uh oh!

Sjors commented Jul 11, 2024

Uh oh!

pinheadmz commented Jul 12, 2024

Uh oh!

Sjors commented Jul 15, 2024

Uh oh!

Uh oh!

Sjors commented Jul 17, 2024

Uh oh!

Sjors commented Jul 19, 2024

Uh oh!

maflcko commented Jul 22, 2024

Uh oh!

Uh oh!

Stratum v2 connman #30332

Stratum v2 connman #30332

Uh oh!

Conversation

Sjors commented Jun 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Jun 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage

Reviews

Conflicts

Uh oh!

DrahtBot commented Jun 25, 2024

Uh oh!

DrahtBot commented Jul 1, 2024

Uh oh!

DrahtBot commented Jul 2, 2024

Uh oh!

TheBlueMatt commented Jul 11, 2024

Uh oh!

Sjors commented Jul 11, 2024

Uh oh!

pinheadmz commented Jul 12, 2024

Uh oh!

Sjors commented Jul 15, 2024

Uh oh!

Uh oh!

Sjors commented Jul 17, 2024

Uh oh!

Sjors commented Jul 19, 2024

Uh oh!

maflcko commented Jul 22, 2024

Uh oh!

Uh oh!

Sjors commented Jun 25, 2024 •

edited

Loading

DrahtBot commented Jun 25, 2024 •

edited

Loading