-
Notifications
You must be signed in to change notification settings - Fork 37.8k
Allow descriptor imports with importmulti #14491
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/wallet/rpcdump.cpp
Outdated
if (::IsMine(*pwallet, raw_pubkey_script) == ISMINE_SPENDABLE) { | ||
throw JSONRPCError(RPC_WALLET_ERROR, "The wallet already contains the private key for this public key"); | ||
} | ||
if (!pwallet->AddWatchOnly(raw_pubkey_script, timestamp)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the same reason as pointed out in the other PR, you can't do this; you're going to mark payments to individual multisig pubkeys as incoming payments.
You'll need a way to restrict this to P2PK, P2WPKH, and P2SH/P2WSH wrapped versions of those.
EDIT: I realize that when it's about private keys, the same effect applies too, and there it can't be avoided. Perhaps this stuff is just to scary, and we should wait until there's a way to actually specify what to treat as ours explicitly...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Restricting it from multisig(to avoid being tricked as you mention) would make this even more confusing to a user.
Unfortunate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mentioned on IRC, but what if we just never import public keys at all, and only either A) add the scriptPubKey as watch only or B) import the private key. If we have the private key then IMO it's less scary, because it's still "ours" and we can access the funds
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right; the same concern doesn't exist for private keys as you're obviously able to spend those coins anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would not importing the pubkeys affect the wallet's ability to construct PSBT inputs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only for things that have PKH/WPKH construction in them (GetPubKey is needed for those, which finds a pubkey based on pubkeyhash).
@meshcollider To be clear, we do have to import the pubkey for those; otherwise the result is not solvable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just some comments while reading the code. Will test.
src/wallet/rpcdump.cpp
Outdated
|
||
success = true; | ||
} | ||
if (data.exists("scriptPubKey")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (data.exists("scriptPubKey") && data.exists("descriptor")) {
// throw error because these should be exclusive?
}
and add test.
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Descriptor is invalid"); | ||
} | ||
if (!parsed_desc->IsRange() && data.exists("range")) { | ||
throw JSONRPCError(RPC_INVALID_PARAMETER, "Range should not be specified for an un-ranged descriptor"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test.
src/wallet/rpcdump.cpp
Outdated
if (!parsed_desc->IsRange() && data.exists("range")) { | ||
throw JSONRPCError(RPC_INVALID_PARAMETER, "Range should not be specified for an un-ranged descriptor"); | ||
} else if (parsed_desc->IsRange() && !data.exists("range")) { | ||
throw JSONRPCError(RPC_INVALID_PARAMETER, "Descriptor is ranged, please specify the range"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test.
} else if (data.exists("descriptor")) { | ||
ProcessImportDesc(pwallet, data, timestamp); | ||
} else { | ||
throw JSONRPCError(RPC_INVALID_PARAMETER, "Either a descriptor or scriptPubKey must be provided."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I said more tests were coming 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still coming?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks like it came: fbb5e93#diff-3dfbfa462305488434b8d8da81f99de7R613
After discussion with sipa, closing for now, will come back to this after #14454 is merged |
Since #14454 has been merged, this can be reopened? |
I believe a simpler approach is possible on top of #14565. |
Yep I'll reopen this when rebased on 14565 |
Rebased on #14565 Still planning on adding more tests + release notes, please don't nitpick the lack of tests yet |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK, lightly tested 6894bf5 and it's able to import a test watch-only wallet I used previously.
I didn't use the watchonly
while importing a descriptor with an xpub
, should that be allowed? Or should watchonly
argument not be allowed when using a descriptor?
Much smaller code change than I would have thought, nice!
It would be nice if range
had a default of [0, DEFAULT_KEYPOOL_SIZE]
or even using the -keypool
argument.
Can you add a usage example with a descriptor?
When I cherry-pick #14477 on top of this and inspect a used address with getaddressinfo
, the origin info in the descriptor seems both wrong and incomplete: "desc": "wpkh([224885f8]026...)",
where 224885f8
is not the master fingerprint I used, and derivation info is missing. There might be some reusable stuff in #14021.
If I use watchonly: true
, then getaddressinfo
doesn't show a descriptor and it says solvable: false
, which seems wrong (the latter also happens without cherry-pick).
@Sjors thanks for the feedback :)
It should only watch for the scriptPubKey and import no other information, watch-only is a different requirement than being solvable without private keys.
You're right, good point. I'll take a look at andrews PR
I think it should be allowed, because of the above reason. Using an xpub without watch only allows it to be solvable without being spendable |
@meshcollider @Sjors I was imagining that "watchonly" would be implicit when using descriptors ( The reason for "watchonly" is so that users need to be explicit about the fact they don't want solvability (generally, you should always want solvability, but if you can't, you can tell importmulti that it's fine without). |
@sipa I like your suggestion. In that case we should disallow usage of the Regarding getting origin information from the descriptors, @achow101 just rebased #14021 on top of this PR. It's a non trivial change. Perhaps for this PR it's best to not store origin information. Just make sure that if you do |
I don't see the problem if you want to add a ranged descriptor as watch only to import all the scriptPubKeys but not retain any more info than that? |
IIUC you would indicate that in the descriptor by wrapping the result in |
xpub byte prefix mismatch results in very generic error; would be nice to give something more meaningful |
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
In addition, you can not ignore the |
Now might be a good to time to rebase it, because testing this might reveal anything we've missed in the overhaul PR #14565. |
I've rebased this but haven't tested, it was a bit of a messy rebase and still a couple of things to address such as adding the warnings to the descriptor import function, plus more tests. I'll hopefully finish those things off in the next couple of days too. |
I've rebased and squashed as well as added some more tests for descriptor imports and fixed the public key issue now #15263 has gone in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK b985e9c
FlatSigningProvider out_keys; | ||
|
||
// Expand all descriptors to get public keys and scripts. | ||
// TODO: get private keys from descriptors too |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: add (TODO to add) expandRange
method to Descriptor instance
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK b985e9c |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK b985e9c. Changes since last review: rebase, more tests, changes to comments, documentation, and error checking.
// Check if this private key corresponds to a public key from the descriptor | ||
if (!pubkey_map.count(id)) { | ||
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Unused private key provided"); | ||
warnings.push_back("Ignoring irrelevant private key."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #14491 (comment)
These comments don't appear to be resolved, but I think that's ok.
// This does not take into account threshold multisigs which could be spendable without all keys. | ||
// Thus, threshold multisigs without all keys will be considered not spendable here, even if they are, | ||
// perhaps triggering a false warning message. This is consistent with the current wallet IsMine check. | ||
bool spendable = std::all_of(pubkey_map.begin(), pubkey_map.end(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "[wallet] Allow descriptor imports with importmulti" (9f48053)
It would be really nice to move this duplicated watchonly checking code out of ProcessLegacy
and ProcessImportDescriptor
either up into ProcessImport
or down to into a common helper function. The watchonly checks are weird enough to deal with once, much less twice in two functions with slightly different variable names and comments.
// This does not take into account threshold multisigs which could be spendable without all keys | ||
bool spendable = std::all_of(pubkey_map.begin(), pubkey_map.end(), [&](const std::pair<CKeyID, CPubKey>& used_key){ return privkey_map.count(used_key.first) > 0; }); | ||
if (!watch_only && !spendable) { | ||
warnings.push_back("Some private keys are missing, outputs will be considered watchonly. If this is intentional, specify the watchonly flag."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #14491 (comment)
s/specify the watchonly flag/specify the watchonly flag to suppress this warning/
Note: implementing this suggestion would make the warning text vary between ProcessImportLegacy
and ProcessImportDescriptor
.
src/wallet/rpcdump.cpp
Outdated
// Generate the script and destination for the scriptPubKey provided | ||
CScript script; | ||
CTxDestination dest; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import_data.import_scripts.emplace(x.second); | ||
} | ||
|
||
std::copy(out_keys.pubkeys.begin(), out_keys.pubkeys.end(), std::inserter(pubkey_map, pubkey_map.end())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: #14491 (comment)
For future reference, this was fixed by removing out.pubkeys.emplace
in #15263.
I think it would be helpful to have a comment here noting that this imports the public keys which occur inside P2PKH and P2WPKH descriptors, but not those inside multisig descriptors. It would also be great to include or link to sipa's rationale for this in #14491 (comment)
b985e9c Add release notes for importmulti descriptor support (MeshCollider) fbb5e93 Add test for importing via descriptor (MeshCollider) 9f48053 [wallet] Allow descriptor imports with importmulti (MeshCollider) d2b381c [wallet] Refactor ProcessImport() to call ProcessImportLegacy() (John Newbery) 4cac0dd [wallet] Add ProcessImportLegacy() (John Newbery) a1b25e1 [wallet] Refactor ProcessImport() (John Newbery) Pull request description: ~~Based on #14454 #14565, last two commits only are for review.~~ Best reviewed with `?w=1` Allows a descriptor to be imported into the wallet using `importmulti` RPC. Start and end of range can be specified for ranged descriptors. The descriptor is implicitly converted to old structures on import. Also adds a simple test of a P2SH-P2WPKH address being imported as a descriptor. More tests to come, as well as release notes. Tree-SHA512: 160eb6fd574c4ae5b70e0109f7e5ccc95d9309138603408a1114ceb3c558065409c0d7afb66926bc8e1743c365a3b300c5f944ff18b2451acc0514fbeca1f2b3
53b7de6 Add test for dumping the private key imported from descriptor (MeshCollider) 2857bc4 Extend importmulti descriptor tests (MeshCollider) 81a884b Import private keys from descriptor with importmulti if provided (MeshCollider) a4d1bd1 Add private key derivation functions to descriptors (MeshCollider) Pull request description: ~This is based on #14491, review the last 3 commits only.~ Currently, descriptors have an Expand() function which returns public keys and scripts for a specific index of a ranged descriptor. But the private key for a specific index is not given. This allows private keys for specific indices to be derived. This also allows those keys to be imported through the `importmulti` RPC rather than having to provide them separately. ACKs for commit 53b7de: achow101: ACK 53b7de6 Tree-SHA512: c060bc01358a1adc76d3d470fefc2bdd39c837027f452e9bc4bd2e726097e1ece4af9d5627efd942a5f8819271e15ba54f010b169b50a9435a1f0f40fd1cebf3
…escriptor 53b7de6 Add test for dumping the private key imported from descriptor (MeshCollider) 2857bc4 Extend importmulti descriptor tests (MeshCollider) 81a884b Import private keys from descriptor with importmulti if provided (MeshCollider) a4d1bd1 Add private key derivation functions to descriptors (MeshCollider) Pull request description: ~This is based on bitcoin#14491, review the last 3 commits only.~ Currently, descriptors have an Expand() function which returns public keys and scripts for a specific index of a ranged descriptor. But the private key for a specific index is not given. This allows private keys for specific indices to be derived. This also allows those keys to be imported through the `importmulti` RPC rather than having to provide them separately. ACKs for commit 53b7de: achow101: ACK 53b7de6 Tree-SHA512: c060bc01358a1adc76d3d470fefc2bdd39c837027f452e9bc4bd2e726097e1ece4af9d5627efd942a5f8819271e15ba54f010b169b50a9435a1f0f40fd1cebf3
Summary: This commit is move-only and doesn't make any functional changes. It simply moves code around within ProcessImport() in preparation for refactors in the next commits. This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@a1b25e1 Some loop variable names have been changed to make sure we don't have shaddowing going on. Test Plan: ninja all check-all Reviewers: #bitcoin_abc, jasonbcox Reviewed By: #bitcoin_abc, jasonbcox Differential Revision: https://reviews.bitcoinabc.org/D6141
Summary: This commit adds a ProcessImportLegacy() function which currently does nothing. It also unindents a block of code for a future move-only change. Reviewer hint: review with -w to ignore whitespace changes. This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@4cac0dd Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D6189
Summary: This is almost entirely a move-only commit. Reviewer hint: use --color-moved=zebra for review. This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@d2b381c Depends on D6189 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6190
Summary: This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@9f48053 Depends on D6190 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D6191
Summary: This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@fbb5e93 Depends on D6187 and D6191 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6192
Summary: This is a partial backport of Core [[bitcoin/bitcoin#14491 | PR14491]] : bitcoin/bitcoin@b985e9c Depends on D6192 Test Plan: read Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6193
Summary: This wasn't ported from [[bitcoin/bitcoin#14491 | PR14491]] because it used segwit, but this can be done without segwit. Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Subscribers: majcosta, PiRK Differential Revision: https://reviews.bitcoinabc.org/D8359
Based on #14454 #14565, last two commits only are for review.Best reviewed with
?w=1
Allows a descriptor to be imported into the wallet using
importmulti
RPC. Start and end of range can be specified for ranged descriptors. The descriptor is implicitly converted to old structures on import.Also adds a simple test of a P2SH-P2WPKH address being imported as a descriptor. More tests to come, as well as release notes.