-
Notifications
You must be signed in to change notification settings - Fork 37.7k
Introduce SockMan ("lite"): low-level socket handling for HTTP #32747
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32747. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsNo conflicts as of last run. LLM Linter (✨ experimental)Possible typos and grammar issues:
drahtbot_id_4_m |
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
55a329a
to
2d79cd7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK.
Are the functions in this PR to replace Connam functions like CConnman::BindListenPort
, CConnman::InitBinds
, etc...?
In this PR they are just protocol-agnostic copies of those ConnMan functions. Just the socket stuff without the Bitcoin stuff. We could still proceed with the reactor in #30988 after this is merged and actually replace the socket stuff in ConnMan with SockMan but this PR is minimized with the focus being the socket needs of the HTTP server in #32061 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK 5f79411
TestSockMan sockman; | ||
|
||
// This address won't actually get used because we stubbed CreateSock() | ||
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
9299d5d SockMan: introduce class and implement binding to listening socket
nit, ensure Lookup()
succeeded before continuing because below addr_bind.value()
is used unconditionally:
BOOST_REQUIRE(addr.has_value());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
src/common/sockman.cpp
Outdated
/** Get the bind address for a socket as CService. */ | ||
static CService GetBindAddress(const Sock& sock) | ||
{ | ||
CService addr_bind; | ||
struct sockaddr_storage sockaddr_bind; | ||
socklen_t sockaddr_bind_len = sizeof(sockaddr_bind); | ||
if (!sock.GetSockName((struct sockaddr*)&sockaddr_bind, &sockaddr_bind_len)) { | ||
addr_bind.SetSockAddr((const struct sockaddr*)&sockaddr_bind, sockaddr_bind_len); | ||
} else { | ||
LogPrintLevel(BCLog::NET, BCLog::Level::Warning, "getsockname failed\n"); | ||
} | ||
return addr_bind; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GetBindAddress()
is the same as in net.cpp
. It is nice to have this PR remove 0 lines, but I think it is better to make an exception and move the function from net.cpp
to netbase.{h,cpp}
and use that from both net.cpp
and common/sockman.cpp
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yes thanks, done by inserting a move-only commit
SocketHandlerConnected(io_readiness); | ||
|
||
// Accept new connections from listening sockets. | ||
SocketHandlerListening(io_readiness.events_per_sock); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
674a5ff SockMan: handle connected sockets: read data from socket
In the commit message: s/conencts/connects/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
src/test/sockman_tests.cpp
Outdated
// Received data is written here by the SockMan I/O thread | ||
// and tested by the main thread. | ||
Mutex m_received_mutex; | ||
std::vector<uint8_t> m_received; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
674a5ff SockMan: handle connected sockets: read data from socket
m_received
would better be per-client:
std::unordered_map<Id, std::vector<uint8_t>> m_received;
and then adjust EventGotData()
to plug the data in the client's slot:
m_received[id].assign(data.begin(), data.end());
and GetReceivedData()
to get the data for the client:
GetReceivedData(Id id)
{
return m_received[id];
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, taken. I was hoping to get away with only ever using one client in this test but this makes more sense for coverage anyway ;-)
return m_connected.erase(id) > 0; | ||
} | ||
|
||
ssize_t SockMan::SendBytes(Id id, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3f796b6 SockMan: handle connected sockets: write data to socket
In the commit message: s/recevied/received/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rebase to address review by @vasild THANKS!
TestSockMan sockman; | ||
|
||
// This address won't actually get used because we stubbed CreateSock() | ||
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
src/common/sockman.cpp
Outdated
/** Get the bind address for a socket as CService. */ | ||
static CService GetBindAddress(const Sock& sock) | ||
{ | ||
CService addr_bind; | ||
struct sockaddr_storage sockaddr_bind; | ||
socklen_t sockaddr_bind_len = sizeof(sockaddr_bind); | ||
if (!sock.GetSockName((struct sockaddr*)&sockaddr_bind, &sockaddr_bind_len)) { | ||
addr_bind.SetSockAddr((const struct sockaddr*)&sockaddr_bind, sockaddr_bind_len); | ||
} else { | ||
LogPrintLevel(BCLog::NET, BCLog::Level::Warning, "getsockname failed\n"); | ||
} | ||
return addr_bind; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yes thanks, done by inserting a move-only commit
src/test/sockman_tests.cpp
Outdated
// Received data is written here by the SockMan I/O thread | ||
// and tested by the main thread. | ||
Mutex m_received_mutex; | ||
std::vector<uint8_t> m_received; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, taken. I was hoping to get away with only ever using one client in this test but this makes more sense for coverage anyway ;-)
SocketHandlerConnected(io_readiness); | ||
|
||
// Accept new connections from listening sockets. | ||
SocketHandlerListening(io_readiness.events_per_sock); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
return m_connected.erase(id) > 0; | ||
} | ||
|
||
ssize_t SockMan::SendBytes(Id id, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 3e7abce
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Began reviewing, then realized that I was reviewing the same code twice with the two "modernize" commits following "original" commits.
Propose to either (1) make clear in the commit message where the code is being pulled from, and that it will be re-styled afterward, or (2) squash the modernization commits into the previous ones. Thereby easing life a bit for your reviewers :)
@@ -17,6 +17,7 @@ | |||
#include <pubkey.h> | |||
#include <stdexcept> | |||
#include <test/util/random.h> | |||
#include <test/util/net.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
535daaf nit, sort
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, thanks
src/test/sockman_tests.cpp
Outdated
@@ -0,0 +1,152 @@ | |||
// Copyright (c) 2021-2022 The Bitcoin Core developers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
535daaf (edit, or is this due to the code already existing, if yes, please mention this in the commit message)
// Copyright (c) 2021-2022 The Bitcoin Core developers | |
// Copyright (c) 2025-present The Bitcoin Core developers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, removed the years entirely which seems to be the style for new files.
Introduce a new low-level socket managing class `SockMan`. BindListenPort() is copied from CConnMan in net.cpp and will be modernized in the next commit. Unit-test it with a new class `SocketTestingSetup` which mocks `CreateSock()` and will enable mock client I/O in future commits. `SockMan` and `SocketTestingSetup` are designed to be generic and reusbale for higher-level network protocol implementation and testing. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
It was copied verbatim from `CConnman::BindListenPort()` in the previous commit. Modernize its variables and style and log the error messages from the caller. Also categorize the informative messages to the "net" category because they are quite specific to the networking layer. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
AcceptConnection() is mostly copied from CConmann in net.cpp and will be modernized in the following commit. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Socket handling methods are copied from CConnMan: `CConnman::GenerateWaitSockets()` goes to `SockMan::GenerateWaitSockets()`. `CConnman::ThreadSocketHandler()` and `CConnman::SocketHandler()` are combined into `SockMan::ThreadSocketHandler()`. `CConnman::SocketHandlerListening()` goes to `SockMan::SocketHandlerListening()`. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
`CConnman::SocketHandlerConnected()` copied to `SockMan::SocketHandlerConnected()`. Testing this requires adding a new feature to the SocketTestingSetup, inserting a "request" payload into the mock client that connects to us. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Sockets-touching bits from `CConnman::SocketSendData()` copied to `SockMan::SendBytes()`. Testing this requires adding a new feature to the SocketTestingSetup, returning the DynSock I/O pipes from the mock socket so the received data can be checked. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Copy from some parts of `CConnman::SocketHandlerConnected()` and `CConnman::ThreadSocketHandler()` to: `EventIOLoopCompletedForOne(id)` and `EventIOLoopCompletedForAll()`. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -17,6 +17,7 @@ | |||
#include <pubkey.h> | |||
#include <stdexcept> | |||
#include <test/util/random.h> | |||
#include <test/util/net.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, thanks
src/test/sockman_tests.cpp
Outdated
@@ -0,0 +1,152 @@ | |||
// Copyright (c) 2021-2022 The Bitcoin Core developers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, removed the years entirely which seems to be the style for new files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK 598bee6
Concept
Agree with exploring this avenue of introducing SockMan
without touching CConnman
. Given @theuni's comment in #30988 (comment), I'm still curious how he would suggest improving it as far as event-loop/send-receive/threading goes.
Commit approach
Think it's fine so far to pair the commits (copy+modernize, ...). It enables reviewers to review the copying and then diffing over 2 commits at once to see what changed in combination.
Suggestions branch
598bee6...hodlinator:bitcoin:pr/32747_suggestions
Ordered my suggestion commits starting from:
- Trivial - Example: use
std::byte
(44bf80c). - Slightly controversial - Example: decreased
shared_ptr
mentioned above. - More arbitrary - Example: rename
ConnectionSockets
=>ConnectionSocket
(650ff1d), probably there are reasons I don't know for the naming.
Hodlinator's shared_ptr
corner
shared_ptr
signals heavily ambiguous ownership. It seems like if there is a socket manager, it should be the one controlling the lifetime of all of them. But since the socket code is already so shared_ptr
-heavy, it's more like a global network API with some objects with value-like semantics thrown in. And maybe I can learn to live with that for now. Few socket operations entail looping over a vector
of them anyway, which weakens the argument for centralizing ownership. Wrapping them in shared_ptr
means they can contain inlined data that is "pinned" to that place in memory with decreased risk of being moved and causing issues (even though that could be trivially worked around through internal pointer indirection).
/** | ||
* This is signaled when network activity should cease. | ||
*/ | ||
CThreadInterrupt interruptNet; | ||
|
||
/** | ||
* List of listening sockets. | ||
*/ | ||
std::vector<std::shared_ptr<Sock>> m_listen; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we avoid public data in the initial version?
void InterruptNet() { interruptNet(); }
const std::vector<std::shared_ptr<Sock>>& ListenSockets() const { return m_listen; }
Alternatively make them protected
and expose them in TestSockMan
.
std::vector<std::pair<Id, CService>> m_connections; | ||
|
||
// Received data is written here by the SockMan I/O thread | ||
// and tested by the main thread. | ||
Mutex m_received_mutex; | ||
std::unordered_map<Id, std::vector<uint8_t>> m_received; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might as well:
std::vector<std::pair<Id, CService>> m_connections; | |
// Received data is written here by the SockMan I/O thread | |
// and tested by the main thread. | |
Mutex m_received_mutex; | |
std::unordered_map<Id, std::vector<uint8_t>> m_received; | |
std::vector<std::pair<Id, CService>> m_connections GUARDED_BY(m_connections_mutex); | |
// Received data is written here by the SockMan I/O thread | |
// and tested by the main thread. | |
Mutex m_received_mutex; | |
std::unordered_map<Id, std::vector<uint8_t>> m_received GUARDED_BY(m_received_mutex); |
}; | ||
virtual void EventGotEOF(Id id) override {}; | ||
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
}; | |
virtual void EventGotEOF(Id id) override {}; | |
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {}; | |
} | |
virtual void EventGotEOF(Id id) override {} | |
virtual void EventGotPermanentReadError(Id id, const std::string& errmsg) override {} |
// This address won't actually get used because we stubbed CreateSock() | ||
const std::optional<CService> addr_bind{Lookup("0.0.0.0", 0, false)}; | ||
BOOST_REQUIRE(addr_bind.has_value()); | ||
bilingual_str strError; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nits:
- Should be snake_case.
- Could be put with the block that uses it.
|
||
// Connect to the socket with a mock client (a DynSock) and send pre-loaded data. | ||
// Returns the I/O pipes from the mock client so we can read response datasent to it. | ||
std::shared_ptr<DynSock::Pipes> ConnectClient(const std::vector<uint8_t>& data); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More modern:
std::shared_ptr<DynSock::Pipes> ConnectClient(const std::vector<uint8_t>& data); | |
std::shared_ptr<DynSock::Pipes> ConnectClient(std::span<const std::byte> data); |
* @param[in] id Connection for which the data arrived. | ||
* @param[in] data Received data. | ||
*/ | ||
virtual void EventGotData(Id id, std::span<const uint8_t> data) = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
virtual void EventGotData(Id id, std::span<const uint8_t> data) = 0; | |
virtual void EventGotData(Id id, std::span<const std::byte> data) = 0; |
EXCLUSIVE_LOCKS_REQUIRED(!m_received_mutex) | ||
{ | ||
LOCK(m_received_mutex); | ||
m_received[id].assign(data.begin(), data.end()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be more realistic to account for network stream being broken up:
m_received[id].assign(data.begin(), data.end()); | |
auto& vec{m_received[id]}; | |
vec.insert(vec.end(), data.begin(), data.end()); |
Implemented a slightly tangential change in my suggestions branch to make DynSock[::Pipe]
support simulating partial I/O in the vein of TCP/IP streams, see a5e7157.
|
||
// Create the Mock Connected Socket that represents a client. | ||
// It needs I/O pipes but its queue can remain empty | ||
std::unique_ptr<DynSock> connected_socket{std::make_unique<DynSock>(connected_socket_pipes, std::make_shared<DynSock::Queue>())}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Managed to remove shared_ptr
usage internally in DynSock
, see commit 7e894f3 from my suggestions branch.
std::vector<uint8_t> actually_received(expected_response_size); | ||
while (!std::ranges::equal(actually_received, sockman.m_respond)) { | ||
// Read the data received by the mock socket | ||
ssize_t bytes_read = pipes->send.GetBytes((void *)actually_received.data(), expected_response_size); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implemented simulating partial transfer of stream in 6a74e1e (requires parent commit).
virtual bool EventNewConnectionAccepted(Id id, | ||
const CService& me, | ||
const CService& them) = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could add an EventConnectionClosed()
to mirror this? See 0ca4fac.
Introduces a new low-level socket manager
SockMan
as an abstract class with virtual functions for implementing higher-level networking protocols like HTTP. This is the next step in #31194This is a minimal, alternative version of #30988 ("Split CConnman") without any changes to working code (P2P is not affected). It adds a stripped-down version of the
SockMan
introduced in that pull request that implements only what is needed for the HTTP server implemented in #32061 (i.e. no I2P stuff and for now, no outbound connection stuff). Exclusions from the originalSockMan
pull request can be checked with:The commit order has been flipped quite a bit because the original PR incrementally pulls logic out of
net
wheras this PR builds a new system from the bottom-up. Otherwise I tried to keep all theSockMan
code in order so reviewers of the original PR would be familiar with it.It also adds unit tests by introducing a
SocketTestingSetup
which mocks network sockets by returning a queue ofDynSock
fromCreateSock()
. HTTP server tests in #32061 will be rebased on this framework as well.