Skip to content

How to rustify libmultiprocess ? #56

@ariard

Description

@ariard

I would like to enable libmultiprocess across different languages where ProxyClient is in a C++ process and ProxyServer is in a Rust process (or vice-versa), while still keeping the interface description reconciliation safe and straightforward.

After a bit of research, I think there is 2 different approaches

  • libmultiprocess.rs : rewriting libmultiprocess in rust, including a new rust-mpgen binary relying on Cap'n Proto rust compiler plugin
  • rust-libmultiprocess.rs : writing Rust bindings for libmultiprocess API (src/mp/proxy.h, src/mp/util.h) and a Rust FFI generator for the artifacts of mpgen

In the case of the first approach, I think API consistency across process would be guaranteed by consuming the data definition (*.capnp). If the calculator.rs/calculator.capnp doesn't match a error should happen at Rust process compilation. This approach comes also
with the benefit that if libmultiprocess features want to be used across Rust only client-server processes, the C++ code doesn't come as a dependency. That said a new dependency is added on the rust compiler plugin and it's a lot of development code and maintenance
to guarantee behavior consistency and feature compatibility of the both library.

For this reason, I think the second approach is wiser.

AFAICT, you have 6 files output by mpgen:

  • calculator.capnp.c++ (generated by the capnp proto compiler though called by mpgen)
  • calculator.capnp.h
  • calculator.proxy-client.c++
  • calculator.proxy-server.c++
  • calculator.proxy-types.c++
  • calculator.proxy.h

I think they're mostly stub code relying on code in include/mp/proxy-types.h and as such should be okay to generater FFI interface for them. I've a doubt if there is a need to cover the calculator.capnp.c++, as I'm not sure if will be consumed by the Rust code anyway.
W.r.t to libmultiprocess API, it's nice to have coverage for proxy.cpp/util.cpp and as such rebuild the process orchestration API available in Bitcoin Core's ipc/* on the Rust-side to have consistent process behavior.

Once you have *.rs output by a Rust FFI generator you can integrate them in your Rust build system. Of course, the C++ generated code must still be build by a C++ compiler, and the outcome of which linked in your Rust binary, otherwise you'll have missing symboles throw at you by the linker.

Here a graphical illustration of the second rust-libmultiprocess approach.



             ---|                   | - calculator.capnp.c++                 
        API     |                   | - calculator.capnp.h                   
                |-----> mpgen ----> | - calculator.capnp.proxy-client.c++   --------------------------> c++-compiler ----------------------------------------------------------------> cpp.o ----|
        data    |                   | - calculator.capnp.proxy-server.c++                                                                                                                        | 
             ---|                   | - calculator.capnp.proxy-types.c++                                                                                                                         |-----> linker ----> calculator.bin
                                    | - calculator.capnp.proxy.h            --------|                               | - calculator.capnp.rs                                                      |
                                                                                    |                               | - calculator.capnp.proxy-client.rs                                         |
                                                                                    |                               | - calculator.capnp.proxy-server.rs  ---------> rust-compiler --> rust.o ---|
                                                                                    |---------> rust-mpgen -------> | - calculator.capnp.proxy-types.rs                   ^
                                                                                                                    | - calculator.capnp.proxy.h.rs                       |
                                                                                                                                                                          |
                                                                                                                                                                          |
                                                                                                                                                proxy.rs -----------------|
                                                                                                                                                util.rs

Do you think the approach advocated is reasonable ? Feel free to raise all relevant points I'm missing.

This is in the context of the Altnet project where I would like to offer the choice between C++/Rust to the pluggable transports daemon writers. Starting by myself :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions