Skip to content

Conversation

theuni
Copy link
Member

@theuni theuni commented Nov 11, 2016

This contains a more complete version of #9117, as well as the changes promised to @TheBlueMatt in #8708. They're combined here because the PushMessage changes require that the SendVersion is sane. This will be my last time messing with PushMessage, and it's nearing the end of the net changes needed outside of actual net code...

The changes here:

  • Don't send messages after fDisconnect is set. (feeler connections are a TODO exception).
  • Don't use CSerializeData for outgoing data, which incurred a memory_cleanse for each message. Messages aren't secret, and going forward, if they are they'll be encrypted.
  • Add a thin, featureless serializer to turn any set of args into a byte vector.
  • Add an go-between message maker so that CConnman sees only serialized messages for pushing.
  • That means that the caller is now responsible for determining the node's send version. That's a good thing, it can now move to CNodeState as later step.
  • CConnman attaches the header and hashes the byte vector. @jonasschnelli: This should be very easy to adapt to bip151. A new flag SERIALIZE_BIP151_ENCRYPTED or so can be added and used similar to SERIALIZE_TRANSACTION_NO_WITNESS.

@@ -6502,9 +6509,13 @@ bool SendMessages(CNode* pto, CConnman& connman)
const Consensus::Params& consensusParams = Params().GetConsensus();
{
// Don't send anything until we get its version message
if (pto->nVersion == 0)
if (!pto->fSuccessfullyConnected || pto->fDisconnect)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Don't send anything until we get its version message

Comment is out-of-date?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dcousens the version message sets fSuccessfullyConnected, but I guess that's not as clear after this change.

How about "don't send anything until we're successfully connected and ok with the peer's version" ?

Copy link
Contributor

@dcousens dcousens Nov 11, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we are still referencing version here to explain why, then fSuccessfullyConnected isn't appropriate.
Otherwise, just say "don't send anything until we're successfully connected".

@theuni
Copy link
Member Author

theuni commented Nov 11, 2016

Grr, will fix up the Windows build problem in the morning.

Copy link

@venzen venzen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

netmessagemaker and handling of initial Version message in main.cpp are good improvements.

@TheBlueMatt
Copy link
Contributor

Concept ACK, though I'm not convinced there arent too many copies when making the message and passing it into CConnman, but will audit when you fix up build, etc.

@theuni
Copy link
Member Author

theuni commented Nov 12, 2016

@TheBlueMatt there's only 1 copy (other than the serialization itself), and it comes from combining the header and payload. See the note here: https://github.com/bitcoin/bitcoin/pull/9128/files#diff-9a82240fe7dfe86564178691cc57f2f1R2590

Either way, it for sure cuts down on copies compared to before.

@@ -5723,8 +5730,8 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv,
pfrom->id,
FormatStateMessage(state));
if (state.GetRejectCode() < REJECT_INTERNAL) // Never send AcceptToMemoryPool's internal codes over P2P
connman.PushMessage(pfrom, NetMsgType::REJECT, strCommand, (unsigned char)state.GetRejectCode(),
state.GetRejectReason().substr(0, MAX_REJECT_MESSAGE_LENGTH), inv.hash);
connman.PushMessage(pfrom, msgMaker.Make(NetMsgType::REJECT, strCommand, (unsigned char)state.GetRejectCode(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: indent

@jonasschnelli
Copy link
Contributor

Concept ACK.

@laanwj
Copy link
Member

laanwj commented Nov 15, 2016

winsok needs an extra cast:

../../src/net.cpp: In function ‘size_t SocketSendData(CNode*)’:
../../src/net.cpp:777:131: error: invalid conversion from ‘const value_type* {aka const unsigned char*}’ to ‘const char*’ [-fpermissive]
         int nBytes = send(pnode->hSocket, &data[pnode->nSendOffset], data.size() - pnode->nSendOffset, MSG_NOSIGNAL | MSG_DONTWAIT);    

@theuni theuni force-pushed the connman-send branch 2 times, most recently from 9d4bf93 to 6d331d6 Compare November 16, 2016 04:09
@theuni
Copy link
Member Author

theuni commented Nov 16, 2016

Fixed several things:

  • Fixed @jonasschnelli's whitespace nit
  • Fixed win32 build, broken due to missing cast
  • Addressed @TheBlueMatt's complaints:
    • Back to using pnode->nVersion == 0 to detect lack of version message
    • Added a change to hold off on setting fDisconnect for feelers until later, to avoid attempting to send messages to a disconnected node. The behavior is kept the same here, but imo it should be changed as a follow-up.
    • zero copies when sending to the net. CSerializedNetMsg now advises how much padding it may possibly need. Its copy ctor and copy assignment operator are deleted to ensure no copies.
    • CVectorInserter::pad and CVectorOverWriter::skip. I'm not sure if these are exactly what you needed, but I took a guess :)

@laanwj
Copy link
Member

laanwj commented Nov 17, 2016

I'm testing this + #9125

@sipa
Copy link
Member

sipa commented Nov 17, 2016

Needs rebase after #9075.

{
return nType;
}
void skip(size_t nSize)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSizeComputer uses 'seek' for this function (inspired by the dd command, where skip is something you do on input, and seek is something you do on output).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Will change.

*
* Note that the referenced vector must already be large enough to hold all data to be serialized.
*/
class CVectorOverWriter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced there is a need to separate CVectorOverWriter and and CVectorInserter. Couldn't you have a single one, which a constructor that initialized the write pointer at the beginning, and one that initializes it at the end?

Writes would involve first checking whether we'd not go out of bounds, and resizing if necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that should work fine. Will change.

Copy link
Contributor

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again didnt do a full review, just looked over the actual code changes and came up with a few comments...will do a full review when they're addressed.


unsigned int nSize = strm.size() - CMessageHeader::HEADER_SIZE;
LogPrint("net", "sending %s (%d bytes) peer=%d\n", SanitizeString(sCommand.c_str()), nSize, pnode->id);
uint256 hash = Hash(msg.data.data() + msg.header_pad_size, msg.data.data() + msg.data.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldnt this happen inside CNetMsgMaker::Make? That way we can just have a BIP151MsgMaker?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was left out of CNetMsgMaker because I assumed that we'd want CNode making the decision about how to wrap the data for sending. If it's to be encrypted, we'll need some per-node state vars.

So I guess it depends whether encryption status and state are to end up somewhere in main like CNodeState, or in CNode. Since (for once) this seems like something CConnman should know about but nobody else should have to care, I figured CNode was the likely place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should mention: I've gone back and forth a few times on this (probably in this PR even). If anyone has a strong preference for a different approach that makes sense, I'm fine with changing it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isnt in CNode, though? If it were in CNode I'd be OK with that, but putting it in CConnman seems like /the/ layer violation this PR intends to fix?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result of this PR is that CConnman now receives only dumb bytes with no knowledge of what they are. All internal structures could change without affecting the net side.

Dealing with wrapping the raw bytes up for transmission is something different, and arguably not something that the bitcoin side should have to know/care about. If you have a different model in your mind, please suggest it.

CNode is slowly moving towards an internal dumb storage class. So in this case, I don't make much distinction.

@@ -2581,30 +2581,17 @@ void CNode::AskFor(const CInv& inv)
mapAskFor.insert(std::make_pair(nRequestTime, inv));
}

CDataStream CConnman::BeginMessage(CNode* pnode, int nVersion, int flags, const std::string& sCommand)
void CConnman::PushMessage(CNode* pnode, CSerializedNetMsg msg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be CSerializedNetMsg&&? It seems super strange that we're secretly destroying the object passed in, but only because the CSerializedNetMsg class doesnt have a copy-constructor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Copy link
Member

@sipa sipa Nov 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing secret is being done. This would fail, for example:

CSerializedNetMsg x = ...;
pconman->PushMessage(pnode, x);

as the binding of the actual parameter x would need a copy constructor to assign to the formal parameter msg.

Passing a && would be preferable, though. The move construction here is slower, and only adds the possibility of passing in something that is implicitly convertible to a CSerializedNetMsg (which afaik doesn't exist).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was done this way originally so that a message could be serialized once and sent to multiple peers. If an rvalue is required, we can't do that.

I don't think there's a use-case for that at the moment, though, so changing to && is fine by me.

{
CSerializedNetMsg msg;
msg.data.assign(CSerializedNetMsg::header_pad_size, 0);
msg.command = std::move(sCommand);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Over-use of std::move, much? Can we stick with const std::string& in the function declaration and then use operator=() to make the copy instead of creating a new object just to std::move the data over?
(Also, could definitely be missing some part of rvalue stuff, here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is standard c++11 pass-by-value-by-default. If you pass in a const-reference, a copy is guaranteed since we're storing the result. If you pass by value, it can be moved and the copy can be skipped.

Not sure why you'd prefer a copy over 2 cheap moves.

Arguably NetMsgType should become a
std::array<char, CMessageHeader::COMMAND_SIZE>
so we can do away with the char arrays/strings, but that's a change for another day.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, didnt realize it would avoid a copy there (though I'd be OK copying the sCommand field...), anyway, comment withdrawn.

@theuni
Copy link
Member Author

theuni commented Nov 18, 2016

Rebased and updated for comments from @sipa and @TheBlueMatt.

  • Combined the vector serializers into CVectorWriter as requested.
  • skip() -> seek()
  • Added tests for CVectorWriter.
  • PushMessage requires an rvalue

@sipa
Copy link
Member

sipa commented Nov 18, 2016 via email

@TheBlueMatt
Copy link
Contributor

Heh, well if you consider CNode to be a part of CConnman, then the type-of-message-to-send logic should be in net_processing...either way CConnman shouldnt know anything about the serialization of messages on the wire.

@maflcko
Copy link
Member

maflcko commented Nov 21, 2016

Needs rebase

@laanwj
Copy link
Member

laanwj commented Nov 23, 2016

Code review ACK 02d463d
Have been testing an earlier revision on a node for 6 days without issues.

@theuni
Copy link
Member Author

theuni commented Nov 23, 2016

updated after a discussion with @TheBlueMatt and @sipa today, I think we're all finally aligned.

@laanwj Sadly removing the "skb" was part of the compromise as it was a slight layer violation. We instead use 2 send()s (later scatter/gather) to avoid the copy.

@sipa
Copy link
Member

sipa commented Nov 24, 2016

Needs rebase, sorry!

This way we're not relying on messages going out after fDisconnect has been
set.

This should not cause any real behavioral changes, though feelers should
arguably disconnect earlier in the process. That can be addressed in a later
functional change.
…onnect

Also, send reject messages earlier in SendMessages(), so that disconnections are
processed earlier.

These changes combined should ensure that no message is ever sent after
fDisconnect is set.
@theuni
Copy link
Member Author

theuni commented Nov 24, 2016

Rebased after ca8549d.

@sipa
Copy link
Member

sipa commented Nov 24, 2016

utACK 09a92c27d46a62d10aed95b742e891e204dcee4e

@paveljanik
Copy link
Contributor

ACK 09a92c2

Copy link
Contributor

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You missed one fDisconnect check in SendMessages (the last one - feefilter, maybe a rebase issue?).

* @param[in] nVersionIn Serialization Version (including any flags)
* @param[in] vchDataIn Referenced byte vector to overwrite/append
* @param[in] nPosIn Starting position. Vector index where writes should start. If the index is past the end, the
* vector will first be resized to fit. So to append, use vec.size().
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is very confusing...if nPos is 1 and vchData.size() is 1, the documentation implies that vchData will be resized so that nPos isnt past-the-end, but it will not be. Instead, vchData is resized so that, at maximum, nPos is 1-past-the-end.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, yes. Will clarify.

}
size_t nMessageSize = msg.data.size();
size_t nTotalSize = nMessageSize + CMessageHeader::HEADER_SIZE;
LogPrint("net", "sending %s (%d bytes) peer=%d\n", SanitizeString(msg.command.c_str()), nTotalSize, pnode->id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You changed the print - should print nMessageSize, not nTotalSize.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I assumed it'd never say "sending 0 bytes", but you're right, I'll avoid a behavioral change here.

CVectorWriter is useful for overwriting or appending an existing byte vector.

CNetMsgMaker is a shortcut for creating messages on-the-fly which are suitable
for pushing to CConnman.
This fixes one of the last major layer violations in the networking stack.

The network side is no longer in charge of message serialization, so it is now
decoupled from Bitcoin structures. Only the header is serialized and attached
to the payload.
@theuni
Copy link
Member Author

theuni commented Nov 25, 2016

Updated and squashed for @TheBlueMatt's comments. The diff from before is very straightforward:

cory@cory-i7:~/dev/bitcoin/src(connman-send)$ git diff theuni/connman-send
diff --git a/src/main.cpp b/src/main.cpp
index 4520afb..5595381 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -6983,7 +6983,7 @@ bool SendMessages(CNode* pto, CConnman& connman)
         // Message: feefilter
         //
         // We don't want white listed peers to filter txs to us if we have -whitelistforcerelay
-        if (!pto->fDisconnect && pto->nVersion >= FEEFILTER_VERSION && GetBoolArg("-feefilter", DEFAULT_FEEFILTER) &&
+        if (pto->nVersion >= FEEFILTER_VERSION && GetBoolArg("-feefilter", DEFAULT_FEEFILTER) &&
             !(pto->fWhitelisted && GetBoolArg("-whitelistforcerelay", DEFAULT_WHITELISTFORCERELAY))) {
             CAmount currentFilter = mempool.GetMinFee(GetArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000).GetFeePerK();
             int64_t timeNow = GetTimeMicros();
diff --git a/src/net.cpp b/src/net.cpp
index e05c174..ce87150 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -2616,7 +2616,7 @@ void CConnman::PushMessage(CNode* pnode, CSerializedNetMsg&& msg)
 {
     size_t nMessageSize = msg.data.size();
     size_t nTotalSize = nMessageSize + CMessageHeader::HEADER_SIZE;
-    LogPrint("net", "sending %s (%d bytes) peer=%d\n",  SanitizeString(msg.command.c_str()), nTotalSize, pnode->id);
+    LogPrint("net", "sending %s (%d bytes) peer=%d\n",  SanitizeString(msg.command.c_str()), nMessageSize, pnode->id);
 
     std::vector<unsigned char> serializedHeader;
     serializedHeader.reserve(CMessageHeader::HEADER_SIZE);
diff --git a/src/streams.h b/src/streams.h
index ef22620..b508784 100644
--- a/src/streams.h
+++ b/src/streams.h
@@ -81,8 +81,8 @@ class CVectorWriter
  * @param[in]  nTypeIn Serialization Type
  * @param[in]  nVersionIn Serialization Version (including any flags)
  * @param[in]  vchDataIn  Referenced byte vector to overwrite/append
- * @param[in]  nPosIn Starting position. Vector index where writes should start. If the index is past the end, the
- *                    vector will first be resized to fit. So to append, use vec.size().
+ * @param[in]  nPosIn Starting position. Vector index where writes should start. The vector will initially
+ *                    grow as necessary to  max(index, vec.size()). So to append, use vec.size().
 */
     CVectorWriter(int nTypeIn, int nVersionIn, std::vector<unsigned char>& vchDataIn, size_t nPosIn) : nType(nTypeIn), nVersion(nVersionIn), vchData(vchDataIn), nPos(nPosIn)
     {

@TheBlueMatt
Copy link
Contributor

TheBlueMatt commented Nov 25, 2016

utACK c7be56d

@sipa sipa merged commit c7be56d into bitcoin:master Nov 25, 2016
sipa added a commit that referenced this pull request Nov 25, 2016
c7be56d net: push only raw data into CConnman (Cory Fields)
2ec935d net: add CVectorWriter and CNetMsgMaker (Cory Fields)
b7695c2 net: No need to check individually for disconnection anymore (Cory Fields)
fedea8a net: don't send any messages before handshake or after requested disconnect (Cory Fields)
d74e352 net: Set feelers to disconnect at the end of the version message (Cory Fields)
codablock pushed a commit to codablock/dash that referenced this pull request Jan 17, 2018
c7be56d net: push only raw data into CConnman (Cory Fields)
2ec935d net: add CVectorWriter and CNetMsgMaker (Cory Fields)
b7695c2 net: No need to check individually for disconnection anymore (Cory Fields)
fedea8a net: don't send any messages before handshake or after requested disconnect (Cory Fields)
d74e352 net: Set feelers to disconnect at the end of the version message (Cory Fields)
andvgal pushed a commit to energicryptocurrency/gen2-energi that referenced this pull request Jan 6, 2019
c7be56d net: push only raw data into CConnman (Cory Fields)
2ec935d net: add CVectorWriter and CNetMsgMaker (Cory Fields)
b7695c2 net: No need to check individually for disconnection anymore (Cory Fields)
fedea8a net: don't send any messages before handshake or after requested disconnect (Cory Fields)
d74e352 net: Set feelers to disconnect at the end of the version message (Cory Fields)
CryptoCentric pushed a commit to absolute-community/absolute that referenced this pull request Feb 25, 2019
c7be56d net: push only raw data into CConnman (Cory Fields)
2ec935d net: add CVectorWriter and CNetMsgMaker (Cory Fields)
b7695c2 net: No need to check individually for disconnection anymore (Cory Fields)
fedea8a net: don't send any messages before handshake or after requested disconnect (Cory Fields)
d74e352 net: Set feelers to disconnect at the end of the version message (Cory Fields)
random-zebra added a commit to PIVX-Project/PIVX that referenced this pull request Sep 7, 2020
30d5c66 net: correct addrman logging (Fuzzbawls)
8a2b7fe Don't send layer2 messages to peers that haven't completed the handshake (Fuzzbawls)
dc10100 [bugfix] Making tier two thread interruptable. (furszy)
2ae76aa Move CNode::addrLocal access behind locked accessors (Fuzzbawls)
470482f Move CNode::addrName accesses behind locked accessors (Fuzzbawls)
35365e1 Move [clean|str]SubVer writes/copyStats into a lock (Fuzzbawls)
d816a86 Make nServices atomic (Matt Corallo)
8a66add Make nStartingHeight atomic (Matt Corallo)
567c9b5 Avoid copying CNodeStats to make helgrind OK with buggy std::string (Matt Corallo)
aea5211 Make nTimeConnected const in CNode (Matt Corallo)
cf46680 net: fix a few races. (Fuzzbawls)
c916fcf net: add a lock around hSocket (Cory Fields)
cc8a93c net: rearrange so that socket accesses can be grouped together (Cory Fields)
6f731dc Do not add to vNodes until fOneShot/fFeeler/fAddNode have been set (Matt Corallo)
07c8d33 Ensure cs_vNodes is held when using the return value from FindNode (Matt Corallo)
110a44b Delete some unused (and broken) functions in CConnman (Matt Corallo)
08a12e0 net: log an error rather than asserting if send version is misused (Cory Fields)
cd8b82c net: Disallow sending messages until the version handshake is complete (Fuzzbawls)
54b454b net: don't run callbacks on nodes that haven't completed the version handshake (Cory Fields)
2be6877 net: deserialize the entire version message locally (Fuzzbawls)
444f599 Dont deserialize nVersion into CNode (Fuzzbawls)
f30f10e net: remove cs_vRecvMsg (Fuzzbawls)
5812f9e net: add a flag to indicate when a node's send buffer is full (Fuzzbawls)
5ec4db2 net: Hardcode protocol sizes and use fixed-size types (Wladimir J. van der Laan)
de87ea6 net: Consistent checksum handling (Wladimir J. van der Laan)
d4bcd25 net: push only raw data into CConnman (Cory Fields)
b79e416 net: add CVectorWriter and CNetMsgMaker (Cory Fields)
63c51d3 net: No need to check individually for disconnection anymore (Cory Fields)
07d8c7b net: don't send any messages before handshake or after fdisconnect (Cory Fields)
9adfc7f net: Set feelers to disconnect at the end of the version message (Cory Fields)
f88c06c net: handle version push in InitializeNode (Cory Fields)
04d39c8 net: construct CNodeStates in place (Cory Fields)
40a6c5d net: remove now-unused ssSend and Fuzz (Cory Fields)
681c62d drop the optimistic write counter hack (Cory Fields)
9f939f3 net: switch all callers to connman for pushing messages (Cory Fields)
8f9011d connman is in charge of pushing messages (Cory Fields)
f558bb7 serialization: teach serializers variadics (Cory Fields)
01ea667 net: Use deterministic randomness for CNode's nonce, and make it const (Cory Fields)
de1ad13 net: constify a few CNode vars to indicate that they're threadsafe (Cory Fields)
34050a3 Move static global randomizer seeds into CConnman (Pieter Wuille)
1ce349f net: add a flag to indicate when a node's process queue is full (Fuzzbawls)
5581b47 net: add a new message queue for the message processor (Fuzzbawls)
701b578 net: rework the way that the messagehandler sleeps (Fuzzbawls)
7e55dbf net: Add a simple function for waking the message handler (Cory Fields)
47ea844 net: record bytes written before notifying the message processor (Cory Fields)
ffd4859 net: handle message accounting in ReceiveMsgBytes (Cory Fields)
8cee696 net: log bytes recv/sent per command (Fuzzbawls)
754400e net: set message deserialization version when it's time to deserialize (Fuzzbawls)
d2b8e0a net: make CMessageHeader a dumb storage class (Fuzzbawls)
cc24eff net: remove redundant max sendbuffer size check (Fuzzbawls)
32ab0c0 net: wait until the node is destroyed to delete its recv buffer (Cory Fields)
6e3f71b net: only disconnect if fDisconnect has been set (Cory Fields)
1b0beb6 net: make GetReceiveFloodSize public (Cory Fields)
229697a net: make vRecvMsg a list so that we can use splice() (Fuzzbawls)
d2d71ba net: fix typo causing the wrong receive buffer size (Cory Fields)
50bb09d Add test-before-evict discipline to addrman (Ethan Heilman)

Pull request description:

  This is a combination of multiple upstream PRs focused on optimizing the P2P networking flow after the introduction of CConnman encapsulation, and a few older PRs that were previously missed to support the later optimizations. The PRs are as follows:

  - bitcoin#9037 - net: Add test-before-evict discipline to addrman
  - bitcoin#5151 - make CMessageHeader a dumb storage class
  - bitcoin#6589 - log bytes recv/sent per command
  - bitcoin#8688 - Move static global randomizer seeds into CConnman
  - bitcoin#9050 - net: make a few values immutable, and use deterministic randomness for the localnonce
  - bitcoin#8708 - net: have CConnman handle message sending
  - bitcoin#9128 - net: Decouple CConnman and message serialization
  - bitcoin#8822 - net: Consistent checksum handling
  - bitcoin#9441 - Net: Massive speedup. Net locks overhaul
  - bitcoin#9609 - net: fix remaining net assertions
  - bitcoin#9626 - Clean up a few CConnman cs_vNodes/CNode things
  - bitcoin#9698 - net: fix socket close race
  - bitcoin#9708 - Clean up all known races/platform-specific UB at the time PR was opened
    - Excluded bitcoin/bitcoin@512731b and bitcoin/bitcoin@d8f2b8a, to be done in a separate PR

ACKs for top commit:
  furszy:
    code ACK 30d5c66 , testnet sync from scratch went well and tested with #1829 on top as well and all good.
  furszy:
     mainnet sync went fine, ACK 30d5c66 .
  random-zebra:
    ACK 30d5c66 and merging...

Tree-SHA512: 09689554f53115a45f810b47ff75d887fa9097ea05992a638dbb6055262aeecd82d6ce5aaa2284003399d839b6f2c36f897413da96cfa2cd3b858387c3f752c1
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.