Skip to content

Conversation

ryanofsky
Copy link
Contributor

Add type conversion hooks to allow UniValue objects, and objects that have CDataStream Serialize and Unserialize methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.


This PR is part of the process separation project.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 20, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK dergoegge, achow101

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

@ryanofsky ryanofsky changed the title multiprocess: Add basic type conversion hoois multiprocess: Add basic type conversion hooks Nov 20, 2023
@ryanofsky ryanofsky mentioned this pull request Nov 20, 2023
87 tasks
@ryanofsky
Copy link
Contributor Author

ryanofsky commented Nov 21, 2023

Updated 42a51f8 -> 118d60e (pr/ipcc.1 -> pr/ipcc.2, compare) fixing lint error and making minor cleanups
Updated 118d60e -> 33d7964 (pr/ipcc.2 -> pr/ipcc.3, compare) replacing CDataStream with DataStream as suggested and making minor cleanups
Updated 33d7964 -> a506148 (pr/ipcc.3 -> pr/ipcc.4, compare) to try to fix MSVC link error https://github.com/bitcoin/bitcoin/actions/runs/6948636857/job/18905029537?pr=28921

Comment on lines +50 to +64
//! Overload multiprocess library's CustomBuildField hook to allow any
//! serializable object to be stored in a capnproto Data field or passed to a
//! canproto interface. Use Priority<1> so this hook has medium priority, and
//! higher priority hooks could take precedence over this one.
template <typename LocalType, typename Value, typename Output>
void CustomBuildField(
TypeList<LocalType>, Priority<1>, InvokeContext& invoke_context, Value&& value, Output&& output,
// Enable if serializeable and if LocalType is not cv or reference
// qualified. If LocalType is cv or reference qualified, it is important to
// fall back to lower-priority Priority<0> implementation of this function
// that strips cv references, to prevent this CustomBuildField overload from
// taking precedence over more narrow overloads for specific LocalTypes.
std::enable_if_t<ipc::capnp::Serializable<LocalType>::value &&
std::is_same_v<LocalType, std::remove_cv_t<std::remove_reference_t<LocalType>>>>* enable = nullptr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm gonna try to summarize my understanding of this to make sure I actually got it right.

We want this specialization of CustomBuildField for (most) of our serializable types. So we can either define it manually for each type or we can make use of sfinae (like you do here) to automatically only create the specialization for each of the required serializable types.

We also want to be able to further overload CustomBuildField for serializable types that can't use this generic specialization (e.g. if we need to use TX_WITH_WITNESS), so we specify Priority<1> such that another overload with Priority<2> would take precedence.

What I don't understand is making this exclusive to non cv and reference qualified types.

If LocalType is cv or reference qualified, it is important to fall back to lower-priority Priority<0> ... to prevent this CustomBuildField overload from taking precedence

As I understand it, this comment implies that (without && std::is_same_v<LocalType, std::remove_cv_t<std::remove_reference_t<LocalType>>>) the following would not take precedence, even though it has Priority<2>?

template <typename LocalType, typename Output>
void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, const CTransaction& value, Output&& output)
{
    DataStream stream;
    stream << TX_WITH_WITNESS(value);
    auto result = output.init(stream.size());
    memcpy(result.begin(), stream.data(), stream.size());
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand it, this comment implies that (without && std::is_same_v<LocalType, std::remove_cv_t<std::remove_reference_t<LocalType>>>) the following would not take precedence, even though it has Priority<2>?

template <typename LocalType, typename Output>
void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, const CTransaction& value, Output&& output)

Yes that's right, but it would be more correct to say "would not reliably take precedence". For example, in the case where a const CTransaction argument was passed then the Priority<2> function actually would take precedence, but if a non-const CTransaction were passed the precedence would be ambiguous. This is because if you passed a non-const CTransaction, the types of the two functions after template type deduction would be:

void CustomBuildField(TypeList<LocalType>, Priority<1>, InvokeContext&, CTransaction&, Output&& output);
void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext&, const CTransaction&, Output&& output);

So with a non-const CTransaction argument, it would be ambiguous whether to call the Priority<2> function that requires a const conversion, or a Priority<1> function not requiring a const-conversion, and there would be a compile error about the overload being ambiguous.

Copy link
Member

@dergoegge dergoegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK a506148

CI failure looks spurious.

Add unit test to test IPC method calls and type conversion between bitcoin c++
types and capnproto messages.

Right now there are custom type hooks in bitcoin IPC code, so the test is
simple, but in upcoming commits, code will be added to convert bitcoin types to
capnproto messages, and the test will be expanded.
Allow any C++ object that has Serialize and Unserialize methods and can be
serialized to a bitcoin CDataStream to be converted to a capnproto Data field
and passed as arguments or return values to capnproto methods using the Data
type.

Extend IPC unit test to cover this and verify the serialization happens
correctly.
Extend IPC unit test to cover this and verify the serialization happens
correctly.
@ryanofsky
Copy link
Contributor Author

Rebased a506148 -> 6acec6b (pr/ipcc.4 -> pr/ipcc.5, compare) due to silent conflicts with #28922 and #28912

Copy link
Member

@dergoegge dergoegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reACK 6acec6b

@achow101
Copy link
Member

ACK 6acec6b

@achow101 achow101 merged commit 2f218c6 into bitcoin:master Jan 23, 2024
@bitcoin bitcoin locked and limited conversation to collaborators Jan 23, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants