-
Notifications
You must be signed in to change notification settings - Fork 37.8k
banlist.dat: store banlist on disk #6310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Nice feature. RE -storebanlist, the system should default to storing. Users may disable. Or don't create an option at all -- same as CAddrDB, user may delete file to clear the list. |
I'd also vote for treating this the same as peers.dat and don't add an option for it. |
return error("%s: Failed to open file %s", __func__, pathBanlist.string()); | ||
|
||
// use file size to size memory buffer | ||
int fileSize = boost::filesystem::file_size(pathBanlist); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this is best to use an int here... what is file_size returning?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you expect users with more then 2GB of banned node data?
I have taken this approach from CAddrDB 1:1. But i agree file sizes should be held in uint64_t
or in size_t
if they are not serialized/shared anywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's plenty of space, but we should be 100% sure we don't overflow IMHO? Yeah there are more parts in the code that don't do that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right: to be precise, file sizes should be uint64_t
, not size_t
. Remember that size_t depends on the address width, it is for in-memory sizes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. Changed from int
to uint64_t
(also in CAddDB
).
This seems risky in my opinion, because not all IPs are static, and they don't necessarily always belong to the same node. |
dexX7: that is why all bans are temporary.
jonasschnelli: I think defaulting to storing them is perfectly fine (or
even not offering the option).
|
@jgarziks way of resetting the banlist (by removing banlist.dat) is very straightforward and doesn't need any explanation. I'd like to avoid another cmd arg option to not end up in having thousands of tiny options. Node misbehaving bans are by default 24h. All other bans needs conscious actions from the users. |
Concept ACK, also agree that defaulting to store bans is perfectly fine (so is not offering an option). As any kind of persistence commits to a format it is good to think forward a bit what should be included: would it make sense to add e.g. a "ban source" enum field, that specifies whether the ban is automatic or manual? |
9c4b16b
to
1fe4661
Compare
I like the idea of storing the "ban source". Because we now keep/dump a serialized |
Right. That would also allow for versioning in serialization.
This might be overdoing it; at least I don't see much added value in per-port bans, at the least they're easily circumvented (for incoming connections: just reconnect and you get a different source port, for outgoing connections simply rebind to a different port). |
@laanwj Think about per port bans with incoming Tor/proxy connections, they're all 127.0.0.1 and when I currently ban a single 127.0.0.1 with a specific port all further connection attempts are blocked until bantime is over. Not sure if this is to solve in any way. |
@Diapolo That simply won't work. As I try to explain above, source ports aren't unique nor identifying. |
@laanwj Indeed, my fault... but should be warned when banning 127.0.0.1 or do you assume people know what they are banning? |
cc60893
to
adf76c8
Compare
Followed and implemented @laanwj s proposal. Now the |
adf76c8
to
0386ba4
Compare
UniValue rec(UniValue::VOBJ); | ||
rec.push_back(Pair("address", (*it).first.ToString())); | ||
rec.push_back(Pair("banned_untill", (*it).second)); | ||
rec.push_back(Pair("banned_untill", banEntry.nBanUntil)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/untill/until/
I suppose?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/untill/until/ I suppose?
Fixed typo.
utACK |
fResult = true; | ||
} | ||
} | ||
return fResult; | ||
} | ||
|
||
void CNode::Ban(const CNetAddr& addr, int64_t bantimeoffset, bool sinceUnixEpoch) { | ||
void CNode::Ban(const CNetAddr& addr, const BanReason &banReason, int64_t bantimeoffset, bool sinceUnixEpoch) { | ||
CSubNet subNet(addr.ToString()+(addr.IsIPv4() ? "/32" : "/128")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hhmm at some point we should add a CSubNet constructor or factory function that makes a subnet with one unique CNetAddr
(netmask all ones), and use that, instead of going through a string. Not a big deal, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hhmm at some point we should add a CSubNet constructor or factory function that makes a subnet with one unique CNetAddr (netmask all ones), and use that, instead of going through a string. Not a big deal, though.
Right. Added (see latest commit). If we don't add it now, it probably will never be added.
0386ba4
to
67511db
Compare
I have an unsubmitted patch that also adds "remembering" of ban scores associated by IP address across disconnects over a time window (24h I think). That is, it prevents malicious nodes from carrying out sawtooth attacks where they can reset their ban score by disconnecting and reconnecting before being banned. There are pros (prevents malicious behavior) and cons (tightens up the rules and increases the likelihood of NAT exit hosts getting banned due to lots of small accumulations of ban points over time.) Is there any interest in this? I can open a new pull if so... |
67511db
to
79f375f
Compare
@ajweiss Not sure about that. At least make sure that the associated structure is size-limited. Especially with IPv6 it will be extremely easy to fill memory that way. But this needs to be discussed somewhere else as it is not related to this pull. |
memset(netmask, 255, sizeof(netmask)); | ||
|
||
std::vector<CNetAddr> vIP; | ||
if (!LookupHost(addr.ToString().c_str(), vIP, 1, fAllowLookup)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. But this seems a bit circuitous. Let's do just:
CSubNet::CSubNet(const CNetAddr &addr, bool fAllowLookup):
valid(addr.IsValid())
{
memset(netmask, 255, sizeof(netmask));
network = addr;
}
Also the fAllowLookup parameter is unnecessary. It should never be needed to look up anything when going from an already binary address.
Edit: added initialization for valid()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right. Wasn't aware of the LookupHost()
in the CNetAddr::CNetAddr()
constructor.
Fixed and pushed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right! The isValid must be set explicit. Fixed.
- added a reason enum for a ban - added creation time for a ban Using CBanEntry as container will keep banlist.dat extenable.
a4ea221
to
9dde3bd
Compare
9dde3bd
to
177a0e4
Compare
banlist.dat: store banlist on disk Cherry-picked from the following upstream PRs: - bitcoin/bitcoin#6310 - bitcoin/bitcoin#6315 - Only the new signal and net changes, no QT - bitcoin/bitcoin#7350 - bitcoin/bitcoin#7458 Part of #2074.
banlist.dat: store banlist on disk Cherry-picked from the following upstream PRs: - bitcoin/bitcoin#6310 - bitcoin/bitcoin#6315 - Only the new signal and net changes, no QT. - bitcoin/bitcoin#7350 - bitcoin/bitcoin#7458 - bitcoin/bitcoin#7696 - bitcoin/bitcoin#7959 - bitcoin/bitcoin#7906 - Only the ban-related commits. - bitcoin/bitcoin#8392 - Only the second commit. - bitcoin/bitcoin#8085 - Only the second commit. - bitcoin/bitcoin#10564 Part of #2074.
banlist.dat: store banlist on disk Cherry-picked from the following upstream PRs: - bitcoin/bitcoin#6310 - bitcoin/bitcoin#6315 - Only the new signal and net changes, no QT. - bitcoin/bitcoin#7350 - bitcoin/bitcoin#7458 - bitcoin/bitcoin#7696 - bitcoin/bitcoin#7959 - bitcoin/bitcoin#7906 - Only the ban-related commits. - bitcoin/bitcoin#8392 - Only the second commit. - bitcoin/bitcoin#8085 - Only the second commit. - bitcoin/bitcoin#10564 Part of #2074.
banlist.dat: store banlist on disk Cherry-picked from the following upstream PRs: - bitcoin/bitcoin#6310 - bitcoin/bitcoin#6315 - Only the new signal and net changes, no QT. - bitcoin/bitcoin#7350 - bitcoin/bitcoin#7458 - bitcoin/bitcoin#7696 - bitcoin/bitcoin#7959 - bitcoin/bitcoin#7906 - Only the ban-related commits. - bitcoin/bitcoin#8392 - Only the second commit. - bitcoin/bitcoin#8085 - Only the second commit. - bitcoin/bitcoin#10564 - bitcoin/bitcoin#13061 - Wasn't needed when I first made this backport in 2017, but it is now! Part of #2074.
banlist.dat: store banlist on disk Cherry-picked from the following upstream PRs: - bitcoin/bitcoin#6310 - bitcoin/bitcoin#6315 - Only the new signal and net changes, no QT. - bitcoin/bitcoin#7350 - bitcoin/bitcoin#7458 - bitcoin/bitcoin#7696 - bitcoin/bitcoin#7959 - bitcoin/bitcoin#7906 - Only the ban-related commits. - bitcoin/bitcoin#8392 - Only the second commit. - bitcoin/bitcoin#8085 - Only the second commit. - bitcoin/bitcoin#10564 - bitcoin/bitcoin#13061 - Wasn't needed when I first made this backport in 2017, but it is now! Part of #2074.
Currently, the list of banned nodes are only held in memory. A restart of bitcoind would reset the set of banned nodes.
This PR introduces a new filestore (
banlist.dat
) which stores the IPs/Subnets of the banned nodes more or less identical to the CAddrDB (peers.dat
).Also added within this PR is a feature that removes unused banned node entries because the
banned_till
time has expired.includes tests for the banlist disk storage.
Unsure if we should introduce a new cmd arg
-storebanlist
to optionally allow storing of banned node data. It could be, that some users expecting a sweep of all banned nodes when they restart bitcoind.