Skip to content

Conversation

mogery
Copy link
Contributor

@mogery mogery commented Feb 3, 2024

Fixes #1
/claim #1

This PR removes the rp shell script, and replaces it with an rp Rust crate. The new implementation does not spawn commands, instead it uses rosenpass-* for key generation and exchange (replacing calls to the rosenpass CLI), wireguard-keys to generate and derive WireGuard keys (replacing calls to wg genkey and wg pubkey), and a few netlink-related crates to create network links and set WireGuard attributes on them (replacing calls to ip link and wg set).

The new implementation is completely arg-compatible with the previous version, except for the explain arg, which is now unsupported. We should discuss if this is okay, or if we should emit fake commands to help users understand what rp is doing, even if it's not necessarily true.

The new implementation includes tests for arg parsing, and performing genkey and pubkey. Tests for exchange are not included due to high complexity (needs sudo, needs to terminate rosenpass event loop, platform-dependent) with little reward (basically doing the same thing as the rosenpass tests). exchange functionality was tested on two Ubuntu 22.04.3 VMs in a virtual network running on QEMU (arm64). My test was to do the getting started guide, which I successfully completed.

Screenshot 2024-02-03 at 23 06 18

One aspect of this replacement that I have not (yet?) been able to figure out is release and packaging, as I'm not familiar with Nix enough to be able to understand the flake, so I'm especially open to guidance in that area.

@mogery
Copy link
Contributor Author

mogery commented Mar 8, 2024

@wucke13 @koraa could you take a look please?

@cvengler cvengler self-assigned this Mar 9, 2024
@cvengler
Copy link
Member

cvengler commented Mar 9, 2024

@mogery Sorry for the long response on our end. I will have a look at this now. :)

Copy link
Member

@cvengler cvengler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed most of this except for the netlink part. Thank you a lot for working on this issue. 🙂

@cvengler
Copy link
Member

cvengler commented Mar 9, 2024

By the way, do we even need a crate for generating Wireguard keys?
After all, we just need three features:

  • Generate a random array of 32 bytes.
  • Support for Base64 encoding/decoding
  • Properly zeroize everything

@mogery
Copy link
Contributor Author

mogery commented Mar 10, 2024

Thanks for the reviews! Will be resolving these tomorrow. I've also been familiarizing myself with Nix so I can try and get that working as well.

@mogery
Copy link
Contributor Author

mogery commented Mar 10, 2024

By the way, do we even need a crate for generating Wireguard keys? After all, we just need three features:

  • Generate a random array of 32 bytes.
  • Support for Base64 encoding/decoding
  • Properly zeroize everything

We also needed something to derive the WireGuard public key. I took a peek at the original wireguard-keys source and I pulled in x25519-dalek to do the derivation work.

@mogery
Copy link
Contributor Author

mogery commented Mar 10, 2024

Added some code to the Nix flake in ff09b4c to compile rp as well and bundle it in the release-package.

@koraa
Copy link
Member

koraa commented Mar 10, 2024

Added some code to the Nix flake in ff09b4c to compile rp as well and bundle it in the release-package.

Great stuff. Thank you!

@koraa
Copy link
Member

koraa commented Mar 10, 2024

@emilengler Can you give it a test and then merge the code?

@mogery Would you be interested in working on integration tests as a follow-up task?

@mogery
Copy link
Contributor Author

mogery commented Mar 10, 2024

@mogery Would you be interested in working on integration tests as a follow-up task?

Not a 100% sure of how to approach that yet, but after some research on my end I'd be interested!

@koraa
Copy link
Member

koraa commented Mar 11, 2024

@mogery Splendid! Can you get in touch via #rosenpass-dev:matrix.org then?

@mogery
Copy link
Contributor Author

mogery commented Mar 11, 2024

@mogery Splendid! Can you get in touch via #rosenpass-dev:matrix.org then?

Yup, joined

Copy link
Member

@cvengler cvengler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Executing the following on the same machine leads to a crash on the server.

Server-side:

$ sudo ../target/debug/rp verbose exchange server.rosenpass-secret/ dev rosenpass0 listen [::1]:9999 peer client.rosenpass-public/ allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

Client-side:

$ sudo ../target/debug/rp verbose exchange client.rosenpass-secret/ dev rosenpass1 peer server.rosenpass-public/ endpoint "[::1]:9999" allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

The crash happens immediately after I start the client.

@mogery
Copy link
Contributor Author

mogery commented Mar 12, 2024

The crash happens immediately after I start the client.

Odd! Debugging this ASAP.

@koraa
Copy link
Member

koraa commented Mar 20, 2024

@mogery @emilengler Please ping me when this is ready for my final review :)

@emilengler Fantastic review, thank you!
@mogery You to, great job working on this PR :)

@mogery
Copy link
Contributor Author

mogery commented Mar 28, 2024

Executing the following on the same machine leads to a crash on the server.

Server-side:

$ sudo ../target/debug/rp verbose exchange server.rosenpass-secret/ dev rosenpass0 listen [::1]:9999 peer client.rosenpass-public/ allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

Client-side:

$ sudo ../target/debug/rp verbose exchange client.rosenpass-secret/ dev rosenpass1 peer server.rosenpass-public/ endpoint "[::1]:9999" allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

The crash happens immediately after I start the client.

This happens when you don't have wg installed. It's an error coming from rosenpass::AppServer::output_key, which doesn't have anything to do with this PR. That was one hell of a bug to track down :D

Anyways, after a nix shell nixpkgs#wireguard-tools, it all works just fine. I ran through the steps on rosenpass.eu to verify.

Sorry for the delay, some stuff went down IRL and I didn't have the time to work on this (or anything for that matter).

@koraa
Copy link
Member

koraa commented Mar 28, 2024

Executing the following on the same machine leads to a crash on the server.
Server-side:

$ sudo ../target/debug/rp verbose exchange server.rosenpass-secret/ dev rosenpass0 listen [::1]:9999 peer client.rosenpass-public/ allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

Client-side:

$ sudo ../target/debug/rp verbose exchange client.rosenpass-secret/ dev rosenpass1 peer server.rosenpass-public/ endpoint "[::1]:9999" allowed-ips fe80::/64
An error occurred: No such file or directory (os error 2)

The crash happens immediately after I start the client.

This happens when you don't have wg installed. It's an error coming from rosenpass::AppServer::output_key, which doesn't have anything to do with this PR. That was one hell of a bug to track down :D

Anyways, after a nix shell nixpkgs#wireguard-tools, it all works just fine. I ran through the steps on rosenpass.eu to verify.

Sorry for the delay, some stuff went down IRL and I didn't have the time to work on this (or anything for that matter).

Can we add a nice error message for that?

@mogery
Copy link
Contributor Author

mogery commented Mar 29, 2024

Can we add a nice error message for that?

Added it (on the rosenpass side) in fa32883. It now errors with:

$ sudo ../target/debug/rp verbose exchange server.rosenpass-secret/ dev rosenpass0 listen '[::1]:9999' peer client.rosenpass-public/ allowed-ips fe80::/64
An error occurred: Could not find wg command

@cvengler
Copy link
Member

cvengler commented Apr 1, 2024

Okay, sweet! Thank you for this. 🙂
I hope we can get this merged until Sunday. 👍

@mogery
Copy link
Contributor Author

mogery commented Apr 1, 2024

Build issues should be fixed now.

@mogery mogery requested a review from cvengler April 5, 2024 19:05
Copy link
Member

@cvengler cvengler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you a lot for the PR. It looks good to me, with a few tiny left overs to address. :-)

  1. Could we make it, that nix build also (tries to) build the rp utility? Right now I had to navigate to that directory manually and compile it for myself.
  2. When I Ctrl-C the application on a Rosenpass connection on the local host, I get An error occurred: Interrupted system call (os error 4). I think we should treat EINTR as a success and not an error. (This paradigm is pretty common among applications).
  3. Would either @koraa and/or @wucke13 test it as well? I've ran it on localhost and it worked like a charm, but I would enjoy some diversity in testing.
  4. @AliceOrunitia The Rosenpass website would have to be updated in order to remove mentions of rp explain as well as rp help. Alternatively, @mogery could add rp help again.
  5. I have only tested this on a Linux system. Has anyone interest in testing this on FreeBSD?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wucke13 Could you review the Nix parts of the pull requests? I do not feel qualified to do so.

flake.nix Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cvengler
Copy link
Member

cvengler commented Apr 7, 2024

This still has some merge conflicts. Could you rebase?

@mogery
Copy link
Contributor Author

mogery commented Apr 7, 2024

  1. Could we make it, that nix build also (tries to) build the rp utility? Right now I had to navigate to that directory manually and compile it for myself.

nix build .#rp

  1. When I Ctrl-C the application on a Rosenpass connection on the local host, I get An error occurred: Interrupted system call (os error 4). I think we should treat EINTR as a success and not an error. (This paradigm is pretty common among applications).

On it.

  1. @AliceOrunitia The Rosenpass website would have to be updated in order to remove mentions of rp explain as well as rp help. Alternatively, @mogery could add rp help again.

Sure.

@mogery
Copy link
Contributor Author

mogery commented Apr 7, 2024

I'm not sure what kind of horrible git sin I just committed but it... seems to be fine? Sorry for messing up timestamps.

@cvengler
Copy link
Member

cvengler commented Apr 7, 2024

No worries. We will most likely do a squash merge anyway :)

@koraa
Copy link
Member

koraa commented Apr 18, 2024

@mogery Please address both issues pointed out by @prabhpreet, fix the Cargo.lock and then you are done finally ping @AliceOrunitia to merge. This is ready!

@mogery mogery force-pushed the mog/rp-rust branch 2 times, most recently from 913c19f to 8a02dff Compare April 19, 2024 09:19
@mogery
Copy link
Contributor Author

mogery commented Apr 19, 2024

Fixed, squashed, rebased, all done! Pinging @AliceOrunitia

@AliceOrunitia
Copy link
Contributor

/tip $100 @prabhpreet

@AliceOrunitia
Copy link
Contributor

/tip $150 @mogery

Copy link

algora-pbc bot commented Apr 19, 2024

1 similar comment
Copy link

algora-pbc bot commented Apr 19, 2024

Copy link

algora-pbc bot commented Apr 19, 2024

🎉🎈 @mogery has been awarded $150! 🎈🎊

Copy link

algora-pbc bot commented Apr 19, 2024

🎉🎈 @prabhpreet has been awarded $100! 🎈🎊

@mogery
Copy link
Contributor Author

mogery commented Apr 19, 2024

Thank you!

@mogery
Copy link
Contributor Author

mogery commented Apr 19, 2024

Whoops, seems like i missed some of the changes in the merge. On it.

@prabhpreet
Copy link
Contributor

prabhpreet commented Apr 19, 2024

@mogery I recently introduced a change in master that allows to pass in test parameters for integration tests for AppServer in the rosenpass crate. You can just pass in "None" as an additional argument in exchange.rs and that should fix it (once your changes are in sync with master)

feat(rp-rust): implement cli arg parsing

feat(rp-rust): add tests

fix(rp-rust): misuse of rtnetlink caused exchange failure

fix(rp-rust): netlink misuse causing failing wg configuration

fix(rp-rust): interpret lack of response from netlink correctly

fix(rp-rust): fix filename typos

chore(rp-rust): lint

fix(rp-rust): fix bad WireguardOut params

feat(rp-rust): support verbosity on exchange

fix(rp-rust/exchange): use port + 1 for peer endpoints

fix(rp-rust): netlink misuse resulting in partial wg set + cleanup

chore(rp-rust): lint

fix(exchange): netlink misuse leading to ignored wg set requests

fix(exchange): re-add ctrl+c cleanup

chore(rp-rust): dependency restructure

fix(rp-rust): fix loud fail on exchange cleanup

fix(rp-rust): remove incorrect warning message in exchange

fix(rp-rust): remove explain arg from usage, add warn if used

chore(rp-rust): format

chore(rp-rust): fix clippy

fix(rp): don't use wireguard-keys crate

fix(rp/key): enforce proper private key directory permissions

fix(rp/key): don't overwrite generated private keys if they already exist

feat(rp/nix): compile in Nix flake and add to release

chore(docs): update readme

chore(rp): cargo fmt

fix(rp): store secrets with 0600 permissions

fix(rp/exchange): fail nicely if specified port is 65535

fix(rp/exchange): zeroize encoded wgsk

fix(rp/exchange): always zeroize wgsk

chore(rp): remove unused import

feat(rosenpass/app_server): return pretty error if wg command is missing

proper permission for secrets aka 0o600

When creating secret keys or use the out file feature, the material
shouldn't be readble to everyone by default.

Fix: rosenpass#260

Signed-off-by: Paul Spooren <mail@aparcar.org>

dependabot: add configuration

This checks daily for outdated cargo crates.

Signed-off-by: Paul Spooren <mail@aparcar.org>

build(deps): bump base64 from 0.21.5 to 0.21.7

Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.21.5 to 0.21.7.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](marshallpierce/rust-base64@v0.21.5...v0.21.7)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump anyhow from 1.0.75 to 1.0.81

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.75 to 1.0.81.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.75...1.0.81)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump env_logger from 0.10.1 to 0.10.2

Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.10.1 to 0.10.2.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](rust-cli/env_logger@v0.10.1...v0.10.2)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

chore: fmt

fix(rp): fix build on darwin

proper permission for secrets aka 0o600

When creating secret keys or use the out file feature, the material
shouldn't be readble to everyone by default.

Fix: rosenpass#260

Signed-off-by: Paul Spooren <mail@aparcar.org>

dependabot: add configuration

This checks daily for outdated cargo crates.

Signed-off-by: Paul Spooren <mail@aparcar.org>

build(deps): bump base64 from 0.21.5 to 0.21.7

Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.21.5 to 0.21.7.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](marshallpierce/rust-base64@v0.21.5...v0.21.7)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump anyhow from 1.0.75 to 1.0.81

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.75 to 1.0.81.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.75...1.0.81)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump env_logger from 0.10.1 to 0.10.2

Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.10.1 to 0.10.2.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](rust-cli/env_logger@v0.10.1...v0.10.2)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump serde from 1.0.193 to 1.0.197

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.193 to 1.0.197.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.193...v1.0.197)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump memoffset from 0.9.0 to 0.9.1

Bumps [memoffset](https://github.com/Gilnaa/memoffset) from 0.9.0 to 0.9.1.
- [Changelog](https://github.com/Gilnaa/memoffset/blob/master/CHANGELOG.md)
- [Commits](Gilnaa/memoffset@v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: memoffset
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

build(deps): bump log from 0.4.20 to 0.4.21

Bumps [log](https://github.com/rust-lang/log) from 0.4.20 to 0.4.21.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](rust-lang/log@0.4.20...0.4.21)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

chore: add dedicated nixpkgs input to flake

This ensures that `nix flake update` doesn't create surprises on
different systems.

chore: update flake.lock

Six months passed, time to get up-to-date again.

fix: .ci/gen-workflow-files.nu script

- Fix std log import (remove the asterisk)
- Add sort to dependencies field to make script output deterministic
- Remove whitespace error at EOF
- Add nushell to the default devShell, so that the script can be ran
  from the devShell

chore: apply .ci/gen-workflow-files.nu script

There is still hand-written stuff in the workflow file that we need to
get rid of, but now at least all autogenerated dependency fields are
sorted.

fix(app_server): use anyhow::bail!

feat(rp): implement clean help command

chore(rp/exchange): document wg_set

fix(rp/exchange): treat EINTR as success

chore(rp): fmt

feat(rp/exchange): use Secret for handling wgsk

fix(rp/cli): incorrect error messages

fix(rp/key): use store_secret for pqsk

chore(cargo): add rp to default-members

chore(cargo): fix Cargo.lock

fix(rp/exchange): get rid of unused import

fix(rp): fix clippy and other errors
@mogery
Copy link
Contributor Author

mogery commented Apr 19, 2024

Fixed and squashed. My bad, thanks for the patience!

Cause of the error: merging and checking on macOS (the failing code is cfg-bound to Linux and FreeBSD only lol)

@koraa
Copy link
Member

koraa commented Apr 19, 2024

@mogery Could you quickly explain why the changes to nix and ci where neccesary in the context of this project?

@koraa
Copy link
Member

koraa commented Apr 19, 2024

OK so the CI changes just build the rp command, right?

@mogery
Copy link
Contributor Author

mogery commented Apr 19, 2024

@mogery Could you quickly explain why the changes to nix and ci where neccesary in the context of this project?

Sorry, phone was dead. They just build rp.

@cvengler
Copy link
Member

I will merge this now. While I am still a bit unqualified in reviewing the NixOS part, it should not be a blocking issue here IMO.
Thank you a lot for making this possible. :-)

@cvengler cvengler merged commit cc7e8dc into rosenpass:main Apr 19, 2024
@ognevny
Copy link

ognevny commented Apr 24, 2024

if I understand right, wireguard-tools are not required after this change, right? I want to provide rosenpass for a msys2 distribution. I found that wireguard-tools doesn't support mingw compilers, so it would be hard to make it to compile there

@mogery
Copy link
Contributor Author

mogery commented Apr 24, 2024

if I understand right, wireguard-tools are not required after this change, right? I want to provide rosenpass for a msys2 distribution. I found that wireguard-tools doesn't support mingw compilers, so it would be hard to make it to compile there

It still does unfortunately, as rosenpass itself still invokes the wg command line tool. Maybe that should be an issue as well?

@ognevny
Copy link

ognevny commented Apr 24, 2024

Maybe that should be an issue as well?

I'm sorry, but didn't understand what you meant. can you explain it, please?

@prabhpreet
Copy link
Contributor

prabhpreet commented Apr 24, 2024

Maybe that should be an issue as well?

I'm sorry, but didn't understand what you meant. can you explain it, please?

I don't think this is an issue (or a new bug/enhancement request from what I understood from mogery's comment). Windows also uses "wg" as its CLI tool once installed.

However, there were linux specific flags in this crate that create a network adapter- I'm not sure how this would work with Windows wireguard client and the rp crate. Not sure if the bash script would have worked either.

@prabhpreet prabhpreet mentioned this pull request Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace rp shell script with a tool written in Rust
6 participants