Skip to content

Conversation

JeremyRubin
Copy link
Contributor

Fixes #22865. Required for #21702 tests.

The basic issue is that that the current exclude flag doesn't play nicely with flags like DISCOURAGE_UPGRADABLE_NOPS + SOME_OPCODE_VERIFY.

When you have a valid TX under SOME_OPCODE_VERIFY, then the tests will try disabling that flag and it will trip over DISCOURAGE_UPGRADABLE_NOPS.

When you exclude DISCOURAGE_UPGRADABLE_NOPS, then it gets re-included when the test checks all cominations of the excluded flags being re-enabled to show failure, but then it doesn't fail when SOME_OPCODE_VERIFY is set.

To address this, we add two optional fields: 1, a list of flags to always enable; 2, a bool of if to disable the one-by-one checker.

By then testing the same vector with:

...'DISCOURAGE_UPGRADABLE_NOPS', 'NONE', true]
...'NONE', 'SOME_OPCODE_VERIFY']

we get around the one flag at a time checker.

An alternative approach, that would provide better testing coverage, would allow a more abritrary relationship checker that can be specificed by the test writer (e.g., a map of flag to implied flag) to be used during the one-by-one checker.

We cannot simply apply these rules in FillFlags/TrimFlags because if we have a transaction that should fail normally with VERIFY_SOME_OPCODE and DISCOURAGE_UPGRADABLE_OPCODES we can't support both cases.

This gap arose when implementing test vectors for BIP-119. Along the way I also discovered that, e.g., CHECKSEQUENCEVERIFY's Upgradable NOP treatment is improperly asserted as per:

                   // To provide for future soft-fork extensibility, if the
                    // operand has the disabled lock-time flag set,
                    // CHECKSEQUENCEVERIFY behaves as a NOP.
                    if ((nSequence & CTxIn::SEQUENCE_LOCKTIME_DISABLE_FLAG) != 0)
                        break

which does not assert the proper DISCOURAGE_UPGRADABLE_NOP behavior. I beleive that lack of discouragement to be mildly worrying but not a major security concern. Fixing the testing harnesses would allow us to properly implement DISCOURAGE_UPGRADABLE_NOPS for CSV and add the corresponding test cases.

@fanquake fanquake added the Tests label Sep 3, 2021
@glozow
Copy link
Member

glozow commented Sep 3, 2021

This particular test failure is because SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS should only be applied for NOPs that aren't upgraded: See #21702 (comment).

If the test passes when adding another script verification flag, it probably means that it was unnecessary to exclude this flag, the test should be updated to be more specific, or this flag isn't a soft fork.

@JeremyRubin
Copy link
Contributor Author

After patch #22871 then CSV will also have a DISCOURAGE_UPGRADABLE_NOP path.

DISCOURAGE_UPGRADABLE can only ever be a soft fork since it's a non-consensus flag.

@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 12, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

@JeremyRubin
Copy link
Contributor Author

@MarcoFalke is their an automated tool to run to fix this PR?

@maflcko
Copy link
Member

maflcko commented Dec 13, 2021

@JeremyRubin Personally I use three-way-diff, which allows to resolve conflicts in a mechanical way (though, manual).

It might be possible to write your own program and set it as mergetool: https://git-scm.com/docs/git-config#Documentation/git-config.txt-mergetool (assuming the mergetool is also used for rebases?)

@bitcoin bitcoin deleted a comment from Dqddyvillager1722 Dec 13, 2021
@JeremyRubin
Copy link
Contributor Author

rebased!

@glozow
Copy link
Member

glozow commented Apr 22, 2022

Concept NACK, we either have the property of restrict-only script verification flags or we don't. Along the lines of the discussion in #22865 (comment), I'd encourage you to write an implementation of CTV that doesn't break this property and disable tests.

@maflcko
Copy link
Member

maflcko commented May 12, 2022

Can this be closed? Seems to be too controversial based on the previous NACK.

@JeremyRubin
Copy link
Contributor Author

No it can't be closed -- I didn't see the nack till now.

Gloria is incorrect that CTV "breaks" this property. Any opcode upgrade would break this property in the future if it is correctly implemented. @sipa has approach ACK'd this approach #22865 (comment) here.

@glozow
Copy link
Member

glozow commented May 12, 2022

Oh indeed I misunderstood that comment. It looks like gating the opcode under DISCOURAGE_UPGRADABLE_NOPS is needed for its usage to remain nonstandard before activation. I can't think of another way to do it either.

Perhaps breaking the restrict-only property is fine if we're repurposing a nop. However, I don't believe that this change is desirable on its own.

@JeremyRubin
Copy link
Contributor Author

JeremyRubin commented May 12, 2022

It's not just a NOP, this type of change, is also required for e.g. OP_SUCCESSX, or for new key types in Taproot as used in APO, or for doing anything with the Annex (edit: annex may be able to 'cheat' like taproot did, will need to think on that).

It'd be good to make the testing changes required in core now since ideally we'd have one approach that fits all these types of discouragable upgrades.

@bitcoin bitcoin locked and limited conversation to collaborators Dec 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Improve Transaction Tests Flags
5 participants