Skip to content

Conversation

jonasschnelli
Copy link
Contributor

Currently, the list of banned nodes are only held in memory. A restart of bitcoind would reset the set of banned nodes.

This PR introduces a new filestore (banlist.dat) which stores the IPs/Subnets of the banned nodes more or less identical to the CAddrDB (peers.dat).

Also added within this PR is a feature that removes unused banned node entries because the banned_till time has expired.

includes tests for the banlist disk storage.

Unsure if we should introduce a new cmd arg -storebanlist to optionally allow storing of banned node data. It could be, that some users expecting a sweep of all banned nodes when they restart bitcoind.

@jgarzik
Copy link
Contributor

jgarzik commented Jun 19, 2015

Nice feature. RE -storebanlist, the system should default to storing. Users may disable.

Or don't create an option at all -- same as CAddrDB, user may delete file to clear the list.

@Diapolo
Copy link

Diapolo commented Jun 19, 2015

I'd also vote for treating this the same as peers.dat and don't add an option for it.

return error("%s: Failed to open file %s", __func__, pathBanlist.string());

// use file size to size memory buffer
int fileSize = boost::filesystem::file_size(pathBanlist);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is best to use an int here... what is file_size returning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you expect users with more then 2GB of banned node data?
I have taken this approach from CAddrDB 1:1. But i agree file sizes should be held in uint64_t or in size_t if they are not serialized/shared anywhere.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's plenty of space, but we should be 100% sure we don't overflow IMHO? Yeah there are more parts in the code that don't do that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right: to be precise, file sizes should be uint64_t, not size_t. Remember that size_t depends on the address width, it is for in-memory sizes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Changed from int to uint64_t (also in CAddDB).

@dexX7
Copy link
Contributor

dexX7 commented Jun 26, 2015

@jgarzik: RE -storebanlist, the system should default to storing. Users may disable.

This seems risky in my opinion, because not all IPs are static, and they don't necessarily always belong to the same node.

@sipa
Copy link
Member

sipa commented Jun 26, 2015 via email

@jonasschnelli
Copy link
Contributor Author

@jgarziks way of resetting the banlist (by removing banlist.dat) is very straightforward and doesn't need any explanation. I'd like to avoid another cmd arg option to not end up in having thousands of tiny options.

Node misbehaving bans are by default 24h. All other bans needs conscious actions from the users.
The chance that your node accidentally get an IP out of a dynamic range, where the previous owner managed to add his bitcoin-core on a ban-list, is very rar. And then, the chance that all nodes did ban this IP or that you connect to a node which conscious added your new IP to the banlist is even rarer
We have 3,706,452,992 assignable public IPv4 addresses.

@laanwj
Copy link
Member

laanwj commented Jun 26, 2015

Concept ACK, also agree that defaulting to store bans is perfectly fine (so is not offering an option).

As any kind of persistence commits to a format it is good to think forward a bit what should be included: would it make sense to add e.g. a "ban source" enum field, that specifies whether the ban is automatic or manual?

@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch from 9c4b16b to 1fe4661 Compare June 26, 2015 13:22
@jonasschnelli
Copy link
Contributor Author

I like the idea of storing the "ban source". Because we now keep/dump a serialized std::map<CSubNet, int64_t> everything beyond that would be a bigger change. We should probably introduce a simple class for the banlist entry like class CBanEntry { int64_t bantime, enum banSource }. This might also be capable of allowing ban distinction between multiple bans on the same IP but with different ports (but would mean moving from std::map to a vector).

@laanwj
Copy link
Member

laanwj commented Jun 26, 2015

We should probably introduce a simple class for the banlist entry like class CBanEntry { int64_t bantime, enum banSource }

Right. That would also allow for versioning in serialization.

This might also be capable of allowing ban distinction between multiple bans on the same IP but with different ports

This might be overdoing it; at least I don't see much added value in per-port bans, at the least they're easily circumvented (for incoming connections: just reconnect and you get a different source port, for outgoing connections simply rebind to a different port).

@Diapolo
Copy link

Diapolo commented Jun 26, 2015

@laanwj Think about per port bans with incoming Tor/proxy connections, they're all 127.0.0.1 and when I currently ban a single 127.0.0.1 with a specific port all further connection attempts are blocked until bantime is over. Not sure if this is to solve in any way.

@laanwj
Copy link
Member

laanwj commented Jun 26, 2015

@Diapolo That simply won't work. As I try to explain above, source ports aren't unique nor identifying.

@Diapolo
Copy link

Diapolo commented Jun 26, 2015

@laanwj Indeed, my fault... but should be warned when banning 127.0.0.1 or do you assume people know what they are banning?

@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch 2 times, most recently from cc60893 to adf76c8 Compare June 26, 2015 19:49
@jonasschnelli
Copy link
Contributor Author

Followed and implemented @laanwj s proposal.
Added CBanEntry as ban metadata container which uses enum BanReason (BanReasonNodeMisbehaving, BanReasonManuallyAdded). Also added nCreateTime within the new metadata class, to allow to backtrack when a ban was added/created.
Using now everywhere the typedef std::map<CSubNet, CBanEntry> banmap_t.

Now the banlist.dat file is extendable over the possibility to detect different versions (CBanEntry->nVersion) of entries and allow possible migrations during deserialization.

@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch from adf76c8 to 0386ba4 Compare June 26, 2015 19:59
UniValue rec(UniValue::VOBJ);
rec.push_back(Pair("address", (*it).first.ToString()));
rec.push_back(Pair("banned_untill", (*it).second));
rec.push_back(Pair("banned_untill", banEntry.nBanUntil));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/untill/until/ I suppose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/untill/until/ I suppose?

Fixed typo.

@laanwj
Copy link
Member

laanwj commented Jun 29, 2015

utACK

fResult = true;
}
}
return fResult;
}

void CNode::Ban(const CNetAddr& addr, int64_t bantimeoffset, bool sinceUnixEpoch) {
void CNode::Ban(const CNetAddr& addr, const BanReason &banReason, int64_t bantimeoffset, bool sinceUnixEpoch) {
CSubNet subNet(addr.ToString()+(addr.IsIPv4() ? "/32" : "/128"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hhmm at some point we should add a CSubNet constructor or factory function that makes a subnet with one unique CNetAddr (netmask all ones), and use that, instead of going through a string. Not a big deal, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hhmm at some point we should add a CSubNet constructor or factory function that makes a subnet with one unique CNetAddr (netmask all ones), and use that, instead of going through a string. Not a big deal, though.

Right. Added (see latest commit). If we don't add it now, it probably will never be added.

@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch from 0386ba4 to 67511db Compare June 29, 2015 18:04
@ajweiss
Copy link
Contributor

ajweiss commented Jun 29, 2015

I have an unsubmitted patch that also adds "remembering" of ban scores associated by IP address across disconnects over a time window (24h I think). That is, it prevents malicious nodes from carrying out sawtooth attacks where they can reset their ban score by disconnecting and reconnecting before being banned. There are pros (prevents malicious behavior) and cons (tightens up the rules and increases the likelihood of NAT exit hosts getting banned due to lots of small accumulations of ban points over time.) Is there any interest in this? I can open a new pull if so...

@laanwj
Copy link
Member

laanwj commented Jul 2, 2015

@ajweiss Not sure about that. At least make sure that the associated structure is size-limited. Especially with IPv6 it will be extremely easy to fill memory that way. But this needs to be discussed somewhere else as it is not related to this pull.

memset(netmask, 255, sizeof(netmask));

std::vector<CNetAddr> vIP;
if (!LookupHost(addr.ToString().c_str(), vIP, 1, fAllowLookup))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. But this seems a bit circuitous. Let's do just:

CSubNet::CSubNet(const CNetAddr &addr, bool fAllowLookup):
    valid(addr.IsValid())
{
    memset(netmask, 255, sizeof(netmask));
    network = addr;
}

Also the fAllowLookup parameter is unnecessary. It should never be needed to look up anything when going from an already binary address.

Edit: added initialization for valid()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Wasn't aware of the LookupHost() in the CNetAddr::CNetAddr() constructor.
Fixed and pushed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right! The isValid must be set explicit. Fixed.

- added a reason enum for a ban
- added creation time for a ban

Using CBanEntry as container will keep banlist.dat extenable.
@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch 3 times, most recently from a4ea221 to 9dde3bd Compare July 2, 2015 18:44
@jonasschnelli jonasschnelli force-pushed the 2015/06/store_ban_list branch from 9dde3bd to 177a0e4 Compare July 2, 2015 18:44
@laanwj laanwj merged commit 177a0e4 into bitcoin:master Jul 2, 2015
laanwj added a commit that referenced this pull request Jul 2, 2015
177a0e4 Adding CSubNet constructor over a single CNetAddr (Jonas Schnelli)
409bccf use CBanEntry as object container for banned nodes (Jonas Schnelli)
dfa174c CAddrDB/CBanDB: change filesize variables from int to uint64_t (Jonas Schnelli)
f581d3d banlist.dat: store banlist on disk (Jonas Schnelli)
zkbot added a commit to zcash/zcash that referenced this pull request Mar 6, 2020
banlist.dat: store banlist on disk

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6310
- bitcoin/bitcoin#6315
  - Only the new signal and net changes, no QT
- bitcoin/bitcoin#7350
- bitcoin/bitcoin#7458

Part of #2074.
@jnewbery jnewbery mentioned this pull request Aug 17, 2020
zkbot added a commit to zcash/zcash that referenced this pull request Nov 12, 2020
banlist.dat: store banlist on disk

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6310
- bitcoin/bitcoin#6315
  - Only the new signal and net changes, no QT.
- bitcoin/bitcoin#7350
- bitcoin/bitcoin#7458
- bitcoin/bitcoin#7696
- bitcoin/bitcoin#7959
- bitcoin/bitcoin#7906
  - Only the ban-related commits.
- bitcoin/bitcoin#8392
  - Only the second commit.
- bitcoin/bitcoin#8085
  - Only the second commit.
- bitcoin/bitcoin#10564

Part of #2074.
zkbot added a commit to zcash/zcash that referenced this pull request Feb 11, 2021
banlist.dat: store banlist on disk

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6310
- bitcoin/bitcoin#6315
  - Only the new signal and net changes, no QT.
- bitcoin/bitcoin#7350
- bitcoin/bitcoin#7458
- bitcoin/bitcoin#7696
- bitcoin/bitcoin#7959
- bitcoin/bitcoin#7906
  - Only the ban-related commits.
- bitcoin/bitcoin#8392
  - Only the second commit.
- bitcoin/bitcoin#8085
  - Only the second commit.
- bitcoin/bitcoin#10564

Part of #2074.
zkbot added a commit to zcash/zcash that referenced this pull request Feb 17, 2021
banlist.dat: store banlist on disk

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6310
- bitcoin/bitcoin#6315
  - Only the new signal and net changes, no QT.
- bitcoin/bitcoin#7350
- bitcoin/bitcoin#7458
- bitcoin/bitcoin#7696
- bitcoin/bitcoin#7959
- bitcoin/bitcoin#7906
  - Only the ban-related commits.
- bitcoin/bitcoin#8392
  - Only the second commit.
- bitcoin/bitcoin#8085
  - Only the second commit.
- bitcoin/bitcoin#10564
- bitcoin/bitcoin#13061
  - Wasn't needed when I first made this backport in 2017, but it is now!

Part of #2074.
zkbot added a commit to zcash/zcash that referenced this pull request Feb 17, 2021
banlist.dat: store banlist on disk

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6310
- bitcoin/bitcoin#6315
  - Only the new signal and net changes, no QT.
- bitcoin/bitcoin#7350
- bitcoin/bitcoin#7458
- bitcoin/bitcoin#7696
- bitcoin/bitcoin#7959
- bitcoin/bitcoin#7906
  - Only the ban-related commits.
- bitcoin/bitcoin#8392
  - Only the second commit.
- bitcoin/bitcoin#8085
  - Only the second commit.
- bitcoin/bitcoin#10564
- bitcoin/bitcoin#13061
  - Wasn't needed when I first made this backport in 2017, but it is now!

Part of #2074.
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants