-
Notifications
You must be signed in to change notification settings - Fork 37.7k
CAddrMan: stochastic address manager #787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
What needs testing with these changes? I assume:
Do you have a tool for comparing two addrman databases? Might be nice to verify that two copies of bitcoin started at about the same time end up with mostly different peer-sets. |
Some improvements and bugfixes done.
The main thing to be tested is whether the bucket count (256 and 64) and bucket size (64) are large enough, knowing that they do not get quickly filled completely. For example, a new-style addr.dat of a week old, will it easily find connections when ran using -noirc -nodnsseed -nolisten? |
Testing on my Mac: I get a core dump on exit, in: CDB::Write<std::string, CAddrMan> (this=0xb0594f20, key=@0xb0594ed0, value=@0x372560, fOverwrite=true) at db.h:106 Looks like a use-after-free problem: (gdb) p _this |
Also: I tested running with your patch and then running an old bitcoind; works nicely (old bitcoind thinks the addr.dat is empty, which is OK; run the new bitcoind and it thinks it needs to rewrite addr.dat again, which is also OK). |
@luke-jr recently had a crash that seemed related to addrman; I am unable to reproduce the problem though. Stack traces or valgrind information from anyone who can reproduce it, is very welcome. |
Design goals: * Only keep a limited number of addresses around, so that addr.dat does not grow without bound. * Keep the address tables in-memory, and occasionally write the table to addr.dat. * Make sure no (localized) attacker can fill the entire table with his nodes/addresses. See comments in addrman.h for more detailed information.
fix tab text truncation
if (nLastTry && nLastTry >= nNow-60) // never remove things tried the last minute | ||
return false; | ||
|
||
if (nTime > nNow + 10*60) // came in a flying DeLorean |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
always makes me laugh this line does :) thanks :)
#define ADDRMAN_MIN_FAIL_DAYS 7 | ||
|
||
// the maximum percentage of nodes to return in a getaddr call | ||
#define ADDRMAN_GETADDR_MAX_PCT 23 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why 23%?
e69dc56983 Use additional system macros to figure out endianness 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. git-subtree-dir: src/secp256k1 git-subtree-split: e69dc569834e070c1d7418d5946fbe120183562e
5e5fb28 Use additional system macros to figure out endianness 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. git-subtree-dir: src/secp256k1 git-subtree-split: 5e5fb28
2ebf20f30b f update test vectors 79da28a01d f use evenness as tie breaker for R fc8ffb8550 f add 0 to nonce function tag according to updated spec d623ff03f0 valgrind_ctime_test: Add schnorrsig_sign cc998b5422 schnorrsig: Add taproot test case c7d8a81e1b schnorrsig: Add benchmark for sign and verify dea212b86b schnorrsig: Add BIP-340 compatible signing and verification f977c8602b schnorrsig: Add BIP-340 nonce function bb48fe50cb schnorrsig: Init empty experimental module eb93995ca8 Allow initializing tagged sha256 2e809e0eaa extrakeys: Add keypair_xonly_tweak_add 5b4c32fcee extrakeys: Add keypair struct with create, pub and pub_xonly b60d714e41 Separate helper functions for pubkey_create and seckey_tweak_add 3824ff5107 extrakeys: Add xonly_pubkey_tweak_add & xonly_pubkey_tweak_add_test 4c0e791f7f Separate helper function for ec_pubkey_tweak_add 88cca701ea extrakeys: Add xonly_pubkey with serialize, parse and from_pubkey 47e6618 extrakeys: Init empty experimental module 3e08b02 Make the secp256k1_declassify argument constant 670cdd3 Merge bitcoin#798: Check assumptions on integer implementation at compile time 7c06899 Compile-time check assumptions on integer types 02b6c87 Add support for (signed) __int128 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. git-subtree-dir: src/secp256k1 git-subtree-split: 2ebf20f30b733a940e3aacf2e6bf756f1ffc74d9
fb442254ef valgrind_ctime_test: Add schnorrsig_sign e2d6f21b1c schnorrsig: Add taproot test case 00779cd965 schnorrsig: Add benchmark for sign and verify 53b0417865 schnorrsig: Add BIP-340 compatible signing and verification 929daeda9c schnorrsig: Add BIP-340 nonce function bb48fe50cb schnorrsig: Init empty experimental module eb93995ca8 Allow initializing tagged sha256 2e809e0eaa extrakeys: Add keypair_xonly_tweak_add 5b4c32fcee extrakeys: Add keypair struct with create, pub and pub_xonly b60d714e41 Separate helper functions for pubkey_create and seckey_tweak_add 3824ff5107 extrakeys: Add xonly_pubkey_tweak_add & xonly_pubkey_tweak_add_test 4c0e791f7f Separate helper function for ec_pubkey_tweak_add 88cca701ea extrakeys: Add xonly_pubkey with serialize, parse and from_pubkey 47e6618 extrakeys: Init empty experimental module 3e08b02 Make the secp256k1_declassify argument constant 670cdd3 Merge bitcoin#798: Check assumptions on integer implementation at compile time 7c06899 Compile-time check assumptions on integer types 02b6c87 Add support for (signed) __int128 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. git-subtree-dir: src/secp256k1 git-subtree-split: fb442254ef88d6bfcfe76cf25236ebda02bad795
79785f0f56 f add test for signing with an invalid keypair b6bc83107a f improve doc of secp256k1_keypair_xonly_tweak_add 307dfbf665 f clarify that you want to use a different secret if keypair_create fails ab1b9b0616 f replace mention of 32-byte public key with 32-byte sequence fb442254ef valgrind_ctime_test: Add schnorrsig_sign e2d6f21b1c schnorrsig: Add taproot test case 00779cd965 schnorrsig: Add benchmark for sign and verify 53b0417865 schnorrsig: Add BIP-340 compatible signing and verification 929daeda9c schnorrsig: Add BIP-340 nonce function bb48fe50cb schnorrsig: Init empty experimental module eb93995ca8 Allow initializing tagged sha256 2e809e0eaa extrakeys: Add keypair_xonly_tweak_add 5b4c32fcee extrakeys: Add keypair struct with create, pub and pub_xonly b60d714e41 Separate helper functions for pubkey_create and seckey_tweak_add 3824ff5107 extrakeys: Add xonly_pubkey_tweak_add & xonly_pubkey_tweak_add_test 4c0e791f7f Separate helper function for ec_pubkey_tweak_add 88cca701ea extrakeys: Add xonly_pubkey with serialize, parse and from_pubkey 47e6618 extrakeys: Init empty experimental module 3e08b02 Make the secp256k1_declassify argument constant 670cdd3 Merge bitcoin#798: Check assumptions on integer implementation at compile time 7c06899 Compile-time check assumptions on integer types 02b6c87 Add support for (signed) __int128 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. git-subtree-dir: src/secp256k1 git-subtree-split: 79785f0f56d2d1ba8c7c746c489eb270eeb31bb1
8ab24e8 Merge bitcoin#558: Add schnorrsig module which implements BIP-340 compliant signatures f3733c5 Merge bitcoin#797: Fix Jacobi benchmarks and other benchmark improvements cb5524a Add benchmark for secp256k1_ge_set_gej_var 5c6af60 Make jacobi benchmarks vary inputs d0fdd5f Randomize the Z coordinates in bench_internal c7a3424 Rename bench_internal variables 875d68b Merge bitcoin#699: Initialize field elements when resulting in infinity 54caf2e Merge bitcoin#799: Add fallback LE/BE for architectures with known endianness + SHA256 selftest f431b3f valgrind_ctime_test: Add schnorrsig_sign 16ffa9d schnorrsig: Add taproot test case 8dfd53e schnorrsig: Add benchmark for sign and verify 4e43520 schnorrsig: Add BIP-340 compatible signing and verification 7332d2d schnorrsig: Add BIP-340 nonce function 7a703fd schnorrsig: Init empty experimental module eabd9bc Allow initializing tagged sha256 6fcb5b8 extrakeys: Add keypair_xonly_tweak_add 5825446 extrakeys: Add keypair struct with create, pub and pub_xonly f001034 Separate helper functions for pubkey_create and seckey_tweak_add 910d9c2 extrakeys: Add xonly_pubkey_tweak_add & xonly_pubkey_tweak_add_test 176bfb1 Separate helper function for ec_pubkey_tweak_add 4cd2ee4 extrakeys: Add xonly_pubkey with serialize, parse and from_pubkey f49c989 Merge bitcoin#806: Trivial: Add test logs to gitignore aabf00c Merge bitcoin#648: Prevent ints from wrapping around in scratch space functions f5adab1 Merge bitcoin#805: Remove the extremely outdated TODO file. bceefd6 Add test logs to gitignore 1c32519 Remove the extremely outdated TODO file. 47e6618 extrakeys: Init empty experimental module 3e08b02 Make the secp256k1_declassify argument constant 8bc6aef Add SHA256 selftest 670cdd3 Merge bitcoin#798: Check assumptions on integer implementation at compile time 5e5fb28 Use additional system macros to figure out endianness 7c06899 Compile-time check assumptions on integer types 02b6c87 Add support for (signed) __int128 979961c Merge bitcoin#787: Use preprocessor macros instead of autoconf to detect endianness 887bd1f Merge bitcoin#793: Make scalar/field choice depend on C-detected __int128 availability 0dccf98 Use preprocessor macros instead of autoconf to detect endianness b2c8c42 Merge bitcoin#795: Avoid linking libcrypto in the valgrind ct test. 57d3a3c Avoid linking libcrypto in the valgrind ct test. 79f1f7a Autodetect __int128 availability on the C side 0d7727f Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field 805082d Merge bitcoin#696: Run a Travis test on s390x (big endian) 3929536 Test travis s390x (big endian) 6034a04 Merge bitcoin#778: secp256k1_gej_double_nonzero supports infinity f609159 Merge bitcoin#779: travis: Fix argument quoting for ./configure 9e49a9b travis: Fix argument quoting for ./configure 18d3632 secp256k1_gej_double_nonzero supports infinity 214cb3c Merge bitcoin#772: Improve constant-timeness on PowerPC 40412b1 Merge bitcoin#774: tests: Abort if malloc() fails during context cloning tests 2e1b9e0 tests: Abort if malloc() fails during context cloning tests 67a429f Suppress a harmless variable-time optimization by clang in _int_cmov 5b19633 Remove redundant "? 1 : 0" after comparisons in scalar code 3e5cfc5 Merge bitcoin#741: Remove unnecessary sign variable from wnaf_const 66bb932 Merge bitcoin#773: Fix some compile problems on weird/old compilers. 1309c03 Fix some compile problems on weird/old compilers. 2309c7d Merge bitcoin#769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 22e578b Undef HAVE___INT128 in basic-config.h to fix gen_context compilation 3f4a5a1 Merge bitcoin#765: remove dead store in ecdsa_signature_parse_der_lax f00d657 remove dead store in ecdsa_signature_parse_der_lax dbd41db Merge bitcoin#759: Fix uninitialized variables in ecmult_multi test 2e7fc5b Fix uninitialized variables in ecmult_multi test 37dba32 Remove unnecessary sign variable from wnaf_const 6bb0b77 Fix test_constant_wnaf for -1 and add a test for it. 47a7b83 Clear field elements when writing infinity 61d1ecb Added test with additions resulting in infinity 60f7f2d Don't assume that ALIGNMENT > 1 in tests ada6361 Use ROUND_TO_ALIGN in scratch_create 8ecc6ce Add check preventing rounding to alignment from wrapping around in scratch_alloc 4edaf06 Add check preventing integer multiplication wrapping around in scratch_max_allocation git-subtree-dir: src/secp256k1 git-subtree-split: 8ab24e8
Design goals:
See comments in addrman.h for more detailed information.