0.15: Backports #11592

maflcko · 2017-11-01T14:16:26Z

No description provided.

ryanofsky

utACK 6666d9524148931d261aed1be6700562876e8116

laanwj · 2017-11-01T14:35:59Z

utACK, though wouldn't we in the long run be better off backporting the function name change to the 0.15 branch wholesale?

TheBlueMatt · 2017-11-01T14:36:49Z

Given we want to use the 0.15 branch for segwit-wallet, that seems like a wise decision to me.

…t cases Github-Pull: bitcoin#10853 Rebased-From: 5864e9c

…xception Github-Pull: bitcoin#10853 Rebased-From: 677d893

-BEGIN VERIFY SCRIPT- sed -i 's/assert_raises_jsonrpc/assert_raises_rpc_error/g' test/functional/*py test/functional/test_framework/*py -END VERIFY SCRIPT- Github-Pull: bitcoin#10853 Rebased-From: 47ba8cf

maflcko · 2017-11-01T14:50:27Z

Right, #11376 (0.15.1) needs it as well.

ryanofsky · 2017-11-01T14:54:40Z

Seems fine, but if you wanted a more minimal change you could just add:

assert_raises_rpc_error = assert_raises_jsonrpc

to the bottom of util.py to add an alias for the new function.

Previous behaviour was to destroy the wallet (to zero-length) Github-Pull: bitcoin#11376 Rebased-From: 5d465e3

Now using a std::unique_ptr, the Db instance is correctly released when CDB initialization fails. The internal CDB state and mapFileUseCount are only mutated when the CDB initialization succeeds. Github-Pull: bitcoin#11492 Rebased-From: 7104de8

Make sure wallet databases have unique fileids. If they don't, throw an error. BDB caches do not work properly when more than one open database has the same fileid, because values written to one database may show up in reads to other databases. Bitcoin will never create different databases with the same fileid, but users can create them by manually copying database files. BDB caching bug was reported by Chris Moore <dooglus@gmail.com> bitcoin#11429 Fixes bitcoin#11429 Github-Pull: bitcoin#11476 Rebased-From: 478a89c

This should fix issues with the multiwallet test and symlinks when the tmpdir is a relative path. Rather than fixing os.symlink to work with paths relative to a directory descriptor, which does not work on Windows, normalize the path instead. Github-Pull: bitcoin#11472 Rebased-From: fa9de37

Github-Pull: bitcoin#11472 Rebased-From: fafa003

Github-Pull: bitcoin#10357 Rebased-From: 0311836

Nodes don't consider themselves out of "initial block download" until their active chain has more work than nMinimumChainWork. While in initial block download, nodes won't relay blocks to their peers, so test that this parameter functions as intended by verifying that block relay only succeeds past a given node once its nMinimumChainWork has been exceeded. Github-Pull: bitcoin#10357 Rebased-From: eac64bb

A peer could try to waste our resources by sending us unrequested blocks with low work, eg to fill up our disk. Since e265200 we no longer request blocks until we know we're on a chain with more than nMinimumChainWork (our anti-DoS threshold), but we would still process unrequested blocks that had more work than our tip. This commit fixes that behavior. Github-Pull: bitcoin#11458 Rebased-From: ce8cd7a

Github-Pull: bitcoin#11458 Rebased-From: 08fd822

maflcko · 2017-11-02T17:27:44Z

@morcos missing #11326 😛

* This removes block-size-limiting code in favor of GBT clients doing the limiting themselves (if at all). * -blockmaxsize is deprecated and only used to calculate an implied blockmaxweight, addressing confusion from multiple users. * getmininginfo's currentblocksize return value was returning garbage values, and has been removed, also removing a GetSerializeSize call in some block generation inner loops and potentially addressing some performance edge cases. Github-Pull: bitcoin#11100 Rebased-From: ba206d2

No sensible user will ever keep the default settings here, so not having sensible defaults only serves to screw users who are paying less attention, which makes for terrible defaults. Github-Pull: bitcoin#11100 Rebased-From: 3dc263c

Github-Pull: bitcoin#11100 Rebased-From: 6f703e9

Github-Pull: bitcoin#11560 Rebased-From: 2d4327d

Github-Pull: bitcoin#11560 Rebased-From: db32a65

If our tip hasn't updated in a while, that may be because our peers are not relaying blocks to us that we would consider valid. Allow connection to an additional outbound peer in that circumstance. Also, periodically check to see if we are exceeding our target number of outbound peers, and disconnect the one which has least recently announced a new block to us (choosing the newest such peer in the case of tie). Github-Pull: bitcoin#11560 Rebased-From: ac7b37c

Github-Pull: bitcoin#11560 Rebased-From: 83df257

Github-Pull: bitcoin#11560 Rebased-From: 6262915

A rare race condition may trigger while awaiting the body of a message, see upsteam commit 5ff8eb26371c4dc56f384b2de35bea2d87814779 for details. This may fix some reported rpc hangs/crashes. Github-Pull: bitcoin#11593 Rebased-From: 6b58360

The bug was introduced in 2.1.6-beta, versions before that don't need the workaround. Github-Pull: bitcoin#11593 Rebased-From: 97932cd

morcos

utACK 8195cb0
Did not thoroughly vet each change, but backported changes myself and verified conflict resolution.

ryanofsky

utACK 8195cb0. I verified that the cherry-pick commits were tagged with the right PR numbers and were previously merged to master, and I confirmed that all the merge conflict resolutions and diffs after cherry-picking looked good.

Used a little bash script to help with this

BRANCH=remotes/origin/0.15
BACKPORT=11592

git reset --hard
git checkout $BRANCH
for commit in $(git rev-list --reverse --topo-order $BRANCH..remotes/origin/pull/$BACKPORT/head); do
    pr=$(git log -n1 --pretty=format:%B "$commit" | sed -n 's/Github-Pull: #//p')
    orig=$(git log -n1 --pretty=format:%B "$commit" | sed -n 's/Rebased-From: //p')
    echo "- Commit $commit orig $orig pr $pr"
    if [ -z "$pr" ]; then
        # Commit is not from a pr, just cherry pick and add REVIEWME tag
        git cherry-pick "$commit" || exit 1
        GIT_EDITOR="sed -i '1 s/.*/REVIEWME\n\n&/'" git commit --amend
        continue
    fi
    if ! git -c rerere.enabled=false cherry-pick $orig > /dev/null; then
        EDITOR=true git commit -a
    fi
    if ! git diff --quiet "HEAD..$commit"; then
        git read-tree "$commit"
        git commit -m "REVIEWME from $commit orig $orig pr $pr"
        while ! git reset --hard; do sleep 1; done
    fi
    if ! grep -q "$orig" <(git rev-list $BRANCH..remotes/origin/pull/$pr/head); then
        echo "Error: Commit $orig not found in pr $pr" >&2
        exit 1
    fi
    if [ "$(git merge-base $orig origin/master)" != $orig ]; then
        echo "Error: Commit $orig from pr $pr not found in master" >&2
        exit 1
    fi
done

git log --grep REVIEWME -p $BRANCH..HEAD

theuni · 2017-11-02T21:23:27Z

utACK 8195cb0, assuming Travis is happy with it.

luke-jr · 2017-11-02T22:29:15Z

NACK #11100. Removing functionality has no place being snuck into a backport, and benefits nobody. It is only ever a reason for NOT upgrading.

gmaxwell · 2017-11-02T22:37:49Z

@luke-jr your opposition to #11100 was noted on the PR itself.

sdaftuar · 2017-11-03T00:41:46Z

I manually verified the backports here; conflict resolution looks correct to me as well.

ACK 8195cb0

meshcollider · 2017-11-03T02:25:59Z

utACK 8195cb0
Thanks for the script @ryanofsky :) That should perhaps be put into contrib/devtools/

sipa · 2017-11-03T22:31:41Z

utACK 8195cb0, looked over the list of backported PRs, and verified using @ryanofsky's script.

8195cb0 rpc: further constrain the libevent workaround (Cory Fields) 34153a7 rpc: work-around an upstream libevent bug (Cory Fields) fc308a6 Add unit test for stale tip checking (Suhas Daftuar) 2ed0647 Add CConnmanTest to mutate g_connman in tests (João Barbosa) a607a95 Connect to an extra outbound peer if our tip is stale (Suhas Daftuar) 459f2db Track tip update time and last new block announcement from each peer (Suhas Daftuar) 49bf090 net: Allow connecting to extra outbound peers (Suhas Daftuar) bb83fe1 Add release notes describing blockmaxweight deprecation (Matt Corallo) 4c82cea Use a sensible default for blockmaxweight (Matt Corallo) 7871a7d Deprecate confusing blockmaxsize, fix getmininginfo output (Matt Corallo) 6baa317 Fix minchainwork test for 0.15 backport (Suhas Daftuar) 55b7abf Make p2p-acceptablock not an extended test (Matt Corallo) 5bec774 [qa] test that invalid blocks on an invalid chain get a disconnect (Matt Corallo) 92d6105 Reject headers building on invalid chains by tracking invalidity (Matt Corallo) 51001d6 Accept unrequested blocks with work equal to our tip (Matt Corallo) c6e4d0c Stop always storing blocks from whitelisted peers (Matt Corallo) e976c36 Rewrite p2p-acceptblock in preparation for slight behavior changes (Matt Corallo) ec8dedf net: Add missing lock in ProcessHeadersMessage(...) (practicalswift) 59b210d Disconnect outbound peers relaying invalid headers (Suhas Daftuar) fc966bb moveonly: factor out headers processing into separate function (Suhas Daftuar) e327224 Add unit test for outbound peer eviction (Suhas Daftuar) 9961abf Permit disconnection of outbound peers on bad/slow chains (Suhas Daftuar) bf191a7 Disconnecting from bad outbound peers in IBD (Suhas Daftuar) d570aa4 Fix uninitialized g_connman crash in Shutdown() (MeshCollider) 0a5477c net: stop both net/net_processing before destroying them (Cory Fields) b4136f2 net: drop unused connman param (Cory Fields) dc897e5 net: use an interface class rather than signals for message processing (Cory Fields) 8aee55a net: pass CConnman via pointer rather than reference (Cory Fields) 6f27965 Rename fAddnode to a more-descriptive "manual_connection" (Matt Corallo) ffb6ea4 Add comment explaining forced processing of compact blocks (Suhas Daftuar) 2df65ee qa: add test for minchainwork use in acceptblock (Suhas Daftuar) 3acec38 Don't process unrequested, low-work blocks (Suhas Daftuar) 0e9d04b [qa] Test nMinimumChainWork (Suhas Daftuar) da4908c Allow setting nMinimumChainWork on command line (Suhas Daftuar) 4108879 qa: Remove never used return value of sync_with_ping (MarcoFalke) f3457d0 qa: Make tmpdir option an absolute path (MarcoFalke) 9c8006d Avoid opening copied wallet databases simultaneously (Russell Yanofsky) de7053f [wallet] Fix leak in CDB constructor (João Barbosa) fd79ed6 Ensure backupwallet fails when attempting to backup to source file (Tomas van der Wansem) d94fc33 scripted-diff: rename assert_raises_jsonrpc to assert_raises_rpc error (John Newbery) 623de0a [tests] do not allow assert_raises_message to be called with JSONRPCException (John Newbery) 5b728c8 [tests] remove direct testing on JSONRPCException from individual test cases (John Newbery) Pull request description: Tree-SHA512: 9fdb5c47844a899271023d8d445f7fc728e3ad71916490cd9783464684967594b07cda05dd644b722bfcea9fade74d06cfc501e1a68abf118d6d03fbbf7d7707

Backport same PRs as done in bitcoin#11592

fanquake added this to the 0.15.0.2 milestone Nov 1, 2017

fanquake added the Tests label Nov 1, 2017

ryanofsky reviewed Nov 1, 2017

View reviewed changes

jnewbery added 3 commits November 1, 2017 10:40

[tests] remove direct testing on JSONRPCException from individual tes…

5b728c8

…t cases Github-Pull: bitcoin#10853 Rebased-From: 5864e9c

[tests] do not allow assert_raises_message to be called with JSONRPCE…

623de0a

…xception Github-Pull: bitcoin#10853 Rebased-From: 677d893

scripted-diff: rename assert_raises_jsonrpc to assert_raises_rpc error

d94fc33

-BEGIN VERIFY SCRIPT- sed -i 's/assert_raises_jsonrpc/assert_raises_rpc_error/g' test/functional/*py test/functional/test_framework/*py -END VERIFY SCRIPT- Github-Pull: bitcoin#10853 Rebased-From: 47ba8cf

maflcko force-pushed the Mf1711-qa01502 branch from 6666d95 to d94fc33 Compare November 1, 2017 14:42

maflcko changed the title ~~[qa] 0.15.0.2: Fix silent merge conflict (assert_raises_rpc_error)~~ [qa] 0.15: Backports (jnewbery) Nov 1, 2017

maflcko changed the title ~~[qa] 0.15: Backports (jnewbery)~~ 0.15: Backports Nov 1, 2017

maflcko force-pushed the Mf1711-qa01502 branch 2 times, most recently from 691c733 to 1aa6f88 Compare November 1, 2017 19:11

tomasvdw and others added 5 commits November 1, 2017 15:23

Ensure backupwallet fails when attempting to backup to source file

fd79ed6

Previous behaviour was to destroy the wallet (to zero-length) Github-Pull: bitcoin#11376 Rebased-From: 5d465e3

[wallet] Fix leak in CDB constructor

de7053f

Now using a std::unique_ptr, the Db instance is correctly released when CDB initialization fails. The internal CDB state and mapFileUseCount are only mutated when the CDB initialization succeeds. Github-Pull: bitcoin#11492 Rebased-From: 7104de8

qa: Remove never used return value of sync_with_ping

4108879

Github-Pull: bitcoin#11472 Rebased-From: fafa003

maflcko force-pushed the Mf1711-qa01502 branch 4 times, most recently from 269e6dd to 5ab0dd8 Compare November 1, 2017 21:38

fanquake added the Backport label Nov 1, 2017

maflcko changed the title ~~0.15: Backports~~ WIP 0.15: Backports Nov 2, 2017

sdaftuar and others added 4 commits November 2, 2017 13:08

Allow setting nMinimumChainWork on command line

da4908c

Github-Pull: bitcoin#10357 Rebased-From: 0311836

qa: add test for minchainwork use in acceptblock

2df65ee

Github-Pull: bitcoin#11458 Rebased-From: 08fd822

maflcko force-pushed the Mf1711-qa01502 branch from 238b4a2 to 55b7abf Compare November 2, 2017 17:23

maflcko force-pushed the Mf1711-qa01502 branch from 8c8e512 to 55b7abf Compare November 2, 2017 18:51

sdaftuar and others added 11 commits November 2, 2017 15:02

Fix minchainwork test for 0.15 backport

6baa317

Use a sensible default for blockmaxweight

4c82cea

No sensible user will ever keep the default settings here, so not having sensible defaults only serves to screw users who are paying less attention, which makes for terrible defaults. Github-Pull: bitcoin#11100 Rebased-From: 3dc263c

Add release notes describing blockmaxweight deprecation

bb83fe1

Github-Pull: bitcoin#11100 Rebased-From: 6f703e9

net: Allow connecting to extra outbound peers

49bf090

Github-Pull: bitcoin#11560 Rebased-From: 2d4327d

Track tip update time and last new block announcement from each peer

459f2db

Github-Pull: bitcoin#11560 Rebased-From: db32a65

Add CConnmanTest to mutate g_connman in tests

2ed0647

Github-Pull: bitcoin#11560 Rebased-From: 83df257

Add unit test for stale tip checking

fc308a6

Github-Pull: bitcoin#11560 Rebased-From: 6262915

rpc: work-around an upstream libevent bug

34153a7

A rare race condition may trigger while awaiting the body of a message, see upsteam commit 5ff8eb26371c4dc56f384b2de35bea2d87814779 for details. This may fix some reported rpc hangs/crashes. Github-Pull: bitcoin#11593 Rebased-From: 6b58360

rpc: further constrain the libevent workaround

8195cb0

The bug was introduced in 2.1.6-beta, versions before that don't need the workaround. Github-Pull: bitcoin#11593 Rebased-From: 97932cd

maflcko changed the title ~~WIP 0.15: Backports~~ 0.15: Backports Nov 2, 2017

morcos approved these changes Nov 2, 2017

View reviewed changes

ryanofsky reviewed Nov 2, 2017

View reviewed changes

sipa merged commit 8195cb0 into bitcoin:0.15 Nov 3, 2017

ajtowns mentioned this pull request Mar 7, 2018

blockmaxsize parameter has no effect #12640

Closed

maflcko deleted the Mf1711-qa01502 branch April 30, 2018 18:54

codablock mentioned this pull request Sep 26, 2019

Backport same PRs as done in bitcoin#11592 dashpay/dash#3113

Merged

codablock added a commit to dashpay/dash that referenced this pull request Sep 30, 2019

Merge pull request #3113 from codablock/pr_backports_v15_v16_5

b2c6b48

Backport same PRs as done in bitcoin#11592

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

0.15: Backports #11592

0.15: Backports #11592

Uh oh!

maflcko commented Nov 1, 2017

Uh oh!

ryanofsky left a comment

Uh oh!

laanwj commented Nov 1, 2017

Uh oh!

TheBlueMatt commented Nov 1, 2017

Uh oh!

maflcko commented Nov 1, 2017

Uh oh!

ryanofsky commented Nov 1, 2017

Uh oh!

maflcko commented Nov 2, 2017

Uh oh!

morcos left a comment

Uh oh!

ryanofsky left a comment

Uh oh!

theuni commented Nov 2, 2017

Uh oh!

luke-jr commented Nov 2, 2017

Uh oh!

gmaxwell commented Nov 2, 2017

Uh oh!

sdaftuar commented Nov 3, 2017

Uh oh!

meshcollider commented Nov 3, 2017 •

edited

Loading

Uh oh!

sipa commented Nov 3, 2017

Uh oh!

Uh oh!

0.15: Backports #11592

0.15: Backports #11592

Uh oh!

Conversation

maflcko commented Nov 1, 2017

Uh oh!

ryanofsky left a comment

Choose a reason for hiding this comment

Uh oh!

laanwj commented Nov 1, 2017

Uh oh!

TheBlueMatt commented Nov 1, 2017

Uh oh!

maflcko commented Nov 1, 2017

Uh oh!

ryanofsky commented Nov 1, 2017

Uh oh!

maflcko commented Nov 2, 2017

Uh oh!

morcos left a comment

Choose a reason for hiding this comment

Uh oh!

ryanofsky left a comment

Choose a reason for hiding this comment

Uh oh!

theuni commented Nov 2, 2017

Uh oh!

luke-jr commented Nov 2, 2017

Uh oh!

gmaxwell commented Nov 2, 2017

Uh oh!

sdaftuar commented Nov 3, 2017

Uh oh!

meshcollider commented Nov 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sipa commented Nov 3, 2017

Uh oh!

Uh oh!

meshcollider commented Nov 3, 2017 •

edited

Loading