Skip to content

Conversation

pinheadmz
Copy link
Member

Introduces a new low-level socket manager SockMan as an abstract class with virtual functions for implementing higher-level networking protocols like HTTP. This is the next step in #31194

This is a minimal, alternative version of #30988 ("Split CConnman") without any changes to working code (P2P is not affected). It adds a stripped-down version of the SockMan introduced in that pull request that implements only what is needed for the HTTP server implemented in #32061 (i.e. no I2P stuff and for now, no outbound connection stuff). Exclusions from the original SockMan pull request can be checked with:

git diff vasild/sockman \
src/common/sockman.h \
src/common/sockman.cpp

The commit order has been flipped quite a bit because the original PR incrementally pulls logic out of net wheras this PR builds a new system from the bottom-up. Otherwise I tried to keep all the SockMan code in order so reviewers of the original PR would be familiar with it.

It also adds unit tests by introducing a SocketTestingSetup which mocks network sockets by returning a queue of DynSock from CreateSock(). HTTP server tests in #32061 will be rebased on this framework as well.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jun 13, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32747.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK w0xlt, hodlinator
Stale ACK vasild

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

  • “an unique id” → “a unique id” [“unique” starts with a consonant sound, so “a” is correct]
  • “datasent” → “data sent” [missing space between “data” and “sent”]

drahtbot_id_4_m

@DrahtBot
Copy link
Contributor

🚧 At least one of the CI tasks failed.
Task MSan, depends: https://github.com/bitcoin/bitcoin/runs/44070031695
LLM reason (✨ experimental): MemorySanitizer detected use of uninitialized memory during sockman_tests, causing CI failure.

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@Sjors
Copy link
Member

Sjors commented Jun 16, 2025

I was able to switch Sjors#50 to use this instead of #30988 without have to change anything. So it makes sense to me to focus on this PR first, as it's smaller. Thoughts @vasild?

Copy link
Contributor

@w0xlt w0xlt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.
Are the functions in this PR to replace Connam functions like CConnman::BindListenPort, CConnman::InitBinds, etc...?

@pinheadmz
Copy link
Member Author

replace Connman functions

In this PR they are just protocol-agnostic copies of those ConnMan functions. Just the socket stuff without the Bitcoin stuff. We could still proceed with the reactor in #30988 after this is merged and actually replace the socket stuff in ConnMan with SockMan but this PR is minimized with the focus being the socket needs of the HTTP server in #32061

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK 5f79411

TestSockMan sockman;

// This address won't actually get used because we stubbed CreateSock()
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9299d5d SockMan: introduce class and implement binding to listening socket

nit, ensure Lookup() succeeded before continuing because below addr_bind.value() is used unconditionally:

BOOST_REQUIRE(addr.has_value());

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines 17 to 29
/** Get the bind address for a socket as CService. */
static CService GetBindAddress(const Sock& sock)
{
CService addr_bind;
struct sockaddr_storage sockaddr_bind;
socklen_t sockaddr_bind_len = sizeof(sockaddr_bind);
if (!sock.GetSockName((struct sockaddr*)&sockaddr_bind, &sockaddr_bind_len)) {
addr_bind.SetSockAddr((const struct sockaddr*)&sockaddr_bind, sockaddr_bind_len);
} else {
LogPrintLevel(BCLog::NET, BCLog::Level::Warning, "getsockname failed\n");
}
return addr_bind;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetBindAddress() is the same as in net.cpp. It is nice to have this PR remove 0 lines, but I think it is better to make an exception and move the function from net.cpp to netbase.{h,cpp} and use that from both net.cpp and common/sockman.cpp.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes thanks, done by inserting a move-only commit

SocketHandlerConnected(io_readiness);

// Accept new connections from listening sockets.
SocketHandlerListening(io_readiness.events_per_sock);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

674a5ff SockMan: handle connected sockets: read data from socket

In the commit message: s/conencts/connects/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

// Received data is written here by the SockMan I/O thread
// and tested by the main thread.
Mutex m_received_mutex;
std::vector<uint8_t> m_received;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

674a5ff SockMan: handle connected sockets: read data from socket

m_received would better be per-client:

std::unordered_map<Id, std::vector<uint8_t>> m_received;

and then adjust EventGotData() to plug the data in the client's slot:

m_received[id].assign(data.begin(), data.end());

and GetReceivedData() to get the data for the client:

GetReceivedData(Id id)
{
    return m_received[id];
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, taken. I was hoping to get away with only ever using one client in this test but this makes more sense for coverage anyway ;-)

return m_connected.erase(id) > 0;
}

ssize_t SockMan::SendBytes(Id id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3f796b6 SockMan: handle connected sockets: write data to socket

In the commit message: s/recevied/received/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member Author

@pinheadmz pinheadmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase to address review by @vasild THANKS!

TestSockMan sockman;

// This address won't actually get used because we stubbed CreateSock()
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)};
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines 17 to 29
/** Get the bind address for a socket as CService. */
static CService GetBindAddress(const Sock& sock)
{
CService addr_bind;
struct sockaddr_storage sockaddr_bind;
socklen_t sockaddr_bind_len = sizeof(sockaddr_bind);
if (!sock.GetSockName((struct sockaddr*)&sockaddr_bind, &sockaddr_bind_len)) {
addr_bind.SetSockAddr((const struct sockaddr*)&sockaddr_bind, sockaddr_bind_len);
} else {
LogPrintLevel(BCLog::NET, BCLog::Level::Warning, "getsockname failed\n");
}
return addr_bind;
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes thanks, done by inserting a move-only commit

// Received data is written here by the SockMan I/O thread
// and tested by the main thread.
Mutex m_received_mutex;
std::vector<uint8_t> m_received;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, taken. I was hoping to get away with only ever using one client in this test but this makes more sense for coverage anyway ;-)

SocketHandlerConnected(io_readiness);

// Accept new connections from listening sockets.
SocketHandlerListening(io_readiness.events_per_sock);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return m_connected.erase(id) > 0;
}

ssize_t SockMan::SendBytes(Id id,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@vasild vasild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 3e7abce

@DrahtBot DrahtBot requested a review from w0xlt June 25, 2025 15:57
@pinheadmz pinheadmz requested review from Sjors and removed request for w0xlt July 2, 2025 18:08
@pinheadmz pinheadmz requested a review from hodlinator July 15, 2025 18:05
Copy link
Member

@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Began reviewing, then realized that I was reviewing the same code twice with the two "modernize" commits following "original" commits.

Propose to either (1) make clear in the commit message where the code is being pulled from, and that it will be re-styled afterward, or (2) squash the modernization commits into the previous ones. Thereby easing life a bit for your reviewers :)

@@ -17,6 +17,7 @@
#include <pubkey.h>
#include <stdexcept>
#include <test/util/random.h>
#include <test/util/net.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

535daaf nit, sort

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

@@ -0,0 +1,152 @@
// Copyright (c) 2021-2022 The Bitcoin Core developers
Copy link
Member

@jonatack jonatack Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

535daaf (edit, or is this due to the code already existing, if yes, please mention this in the commit message)

Suggested change
// Copyright (c) 2021-2022 The Bitcoin Core developers
// Copyright (c) 2025-present The Bitcoin Core developers

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, removed the years entirely which seems to be the style for new files.

@DrahtBot DrahtBot requested a review from w0xlt July 23, 2025 22:42
Introduce a new low-level socket managing class `SockMan`.

BindListenPort() is copied from CConnMan in net.cpp and will
be modernized in the next commit.

Unit-test it with a new class `SocketTestingSetup` which mocks
`CreateSock()` and will enable mock client I/O in future commits.

`SockMan` and `SocketTestingSetup` are designed to be generic and
reusbale for higher-level network protocol implementation and testing.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
pinheadmz and others added 9 commits August 7, 2025 11:29
It was copied verbatim from `CConnman::BindListenPort()` in the previous
commit. Modernize its variables and style and log the error messages
from the caller. Also categorize the informative messages to the "net"
category because they are quite specific to the networking layer.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
AcceptConnection() is mostly copied from CConmann in net.cpp
and will be modernized in the following commit.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Socket handling methods are copied from CConnMan:

`CConnman::GenerateWaitSockets()` goes to
`SockMan::GenerateWaitSockets()`.

`CConnman::ThreadSocketHandler()` and
`CConnman::SocketHandler()` are combined into
`SockMan::ThreadSocketHandler()`.

`CConnman::SocketHandlerListening()` goes to
`SockMan::SocketHandlerListening()`.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
`CConnman::SocketHandlerConnected()` copied to
`SockMan::SocketHandlerConnected()`.

Testing this requires adding a new feature to the SocketTestingSetup,
inserting a "request" payload into the mock client that connects
to us.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Sockets-touching bits from `CConnman::SocketSendData()` copied to
`SockMan::SendBytes()`.

Testing this requires adding a new feature to the SocketTestingSetup,
returning the DynSock I/O pipes from the mock socket so the received
data can be checked.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Copy from some parts of `CConnman::SocketHandlerConnected()` and
`CConnman::ThreadSocketHandler()` to:
`EventIOLoopCompletedForOne(id)` and `EventIOLoopCompletedForAll()`.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Copy link
Member Author

@pinheadmz pinheadmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3e7abce -> 598bee6

Address nits from @jonatack including commit messages that indicate code is copied from CConnman and will be modernized in following commits.

@@ -17,6 +17,7 @@
#include <pubkey.h>
#include <stdexcept>
#include <test/util/random.h>
#include <test/util/net.h>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

@@ -0,0 +1,152 @@
// Copyright (c) 2021-2022 The Bitcoin Core developers
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, removed the years entirely which seems to be the style for new files.

Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK 598bee6

Concept

Agree with exploring this avenue of introducing SockMan without touching CConnman. Given @theuni's comment in #30988 (comment), I'm still curious how he would suggest improving it as far as event-loop/send-receive/threading goes.

Commit approach

Think it's fine so far to pair the commits (copy+modernize, ...). It enables reviewers to review the copying and then diffing over 2 commits at once to see what changed in combination.

Suggestions branch

598bee6...hodlinator:bitcoin:pr/32747_suggestions

Ordered my suggestion commits starting from:

  • Trivial - Example: use std::byte (44bf80c).
  • Slightly controversial - Example: decreased shared_ptr mentioned above.
  • More arbitrary - Example: rename ConnectionSockets => ConnectionSocket (650ff1d), probably there are reasons I don't know for the naming.

Hodlinator's shared_ptr corner

shared_ptr signals heavily ambiguous ownership. It seems like if there is a socket manager, it should be the one controlling the lifetime of all of them. But since the socket code is already so shared_ptr-heavy, it's more like a global network API with some objects with value-like semantics thrown in. And maybe I can learn to live with that for now. Few socket operations entail looping over a vector of them anyway, which weakens the argument for centralizing ownership. Wrapping them in shared_ptr means they can contain inlined data that is "pinned" to that place in memory with decreased risk of being moved and causing issues (even though that could be trivially worked around through internal pointer indirection).

Comment on lines +120 to +128
/**
* This is signaled when network activity should cease.
*/
CThreadInterrupt interruptNet;

/**
* List of listening sockets.
*/
std::vector<std::shared_ptr<Sock>> m_listen;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we avoid public data in the initial version?

    void InterruptNet() { interruptNet(); }

    const std::vector<std::shared_ptr<Sock>>& ListenSockets() const { return m_listen; }

Alternatively make them protected and expose them in TestSockMan.

Comment on lines +21 to +26
std::vector<std::pair<Id, CService>> m_connections;

// Received data is written here by the SockMan I/O thread
// and tested by the main thread.
Mutex m_received_mutex;
std::unordered_map<Id, std::vector<uint8_t>> m_received;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well:

Suggested change
std::vector<std::pair<Id, CService>> m_connections;
// Received data is written here by the SockMan I/O thread
// and tested by the main thread.
Mutex m_received_mutex;
std::unordered_map<Id, std::vector<uint8_t>> m_received;
std::vector<std::pair<Id, CService>> m_connections GUARDED_BY(m_connections_mutex);
// Received data is written here by the SockMan I/O thread
// and tested by the main thread.
Mutex m_received_mutex;
std::unordered_map<Id, std::vector<uint8_t>> m_received GUARDED_BY(m_received_mutex);

Comment on lines +64 to +66
};
virtual void EventGotEOF(Id id) override {};
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
};
virtual void EventGotEOF(Id id) override {};
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {};
}
virtual void EventGotEOF(Id id) override {}
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {}

// This address won't actually get used because we stubbed CreateSock()
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)};
BOOST_REQUIRE(addr_bind.has_value());
bilingual_str strError;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits:

  1. Should be snake_case.
  2. Could be put with the block that uses it.


// Connect to the socket with a mock client (a DynSock) and send pre-loaded data.
// Returns the I/O pipes from the mock client so we can read response datasent to it.
std::shared_ptr<DynSock::Pipes> ConnectClient(const std::vector<uint8_t>& data);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More modern:

Suggested change
std::shared_ptr<DynSock::Pipes> ConnectClient(const std::vector<uint8_t>& data);
std::shared_ptr<DynSock::Pipes> ConnectClient(std::span<const std::byte> data);

* @param[in] id Connection for which the data arrived.
* @param[in] data Received data.
*/
virtual void EventGotData(Id id, std::span<const uint8_t> data) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
virtual void EventGotData(Id id, std::span<const uint8_t> data) = 0;
virtual void EventGotData(Id id, std::span<const std::byte> data) = 0;

EXCLUSIVE_LOCKS_REQUIRED(!m_received_mutex)
{
LOCK(m_received_mutex);
m_received[id].assign(data.begin(), data.end());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be more realistic to account for network stream being broken up:

Suggested change
m_received[id].assign(data.begin(), data.end());
auto& vec{m_received[id]};
vec.insert(vec.end(), data.begin(), data.end());

Implemented a slightly tangential change in my suggestions branch to make DynSock[::Pipe] support simulating partial I/O in the vein of TCP/IP streams, see a5e7157.


// Create the Mock Connected Socket that represents a client.
// It needs I/O pipes but its queue can remain empty
std::unique_ptr<DynSock> connected_socket{std::make_unique<DynSock>(connected_socket_pipes, std::make_shared<DynSock::Queue>())};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Managed to remove shared_ptr usage internally in DynSock, see commit 7e894f3 from my suggestions branch.

std::vector<uint8_t> actually_received(expected_response_size);
while (!std::ranges::equal(actually_received, sockman.m_respond)) {
// Read the data received by the mock socket
ssize_t bytes_read = pipes->send.GetBytes((void *)actually_received.data(), expected_response_size);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented simulating partial transfer of stream in 6a74e1e (requires parent commit).

Comment on lines +145 to +147
virtual bool EventNewConnectionAccepted(Id id,
const CService& me,
const CService& them) = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could add an EventConnectionClosed() to mirror this? See 0ca4fac.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants