-
Notifications
You must be signed in to change notification settings - Fork 722
update 3rd party libraries, part2 #3707
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ca31a3a
to
25a88f3
Compare
f7603a6
to
6481118
Compare
b254d90
to
2a93369
Compare
6481118
to
f39584d
Compare
2a93369
to
d79d2c9
Compare
a45865d
to
5bb9771
Compare
84e7715
to
1766a7b
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3707 +/- ##
=======================================
Coverage 89.33% 89.34%
=======================================
Files 260 260
Lines 14735 14738 +3
=======================================
+ Hits 13164 13167 +3
Misses 1571 1571 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
vcpkg.json
Outdated
@@ -1,5 +1,5 @@ | |||
{ | |||
"builtin-baseline": "ca7b1b15f548c25c766360593a2c732d56ed0133", | |||
"builtin-baseline": "c82f74667287d3dc386bce81e44964370c91a289", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
builtin-baseline
field defines the versions of the dependencies of our vcpkg packages. In this particular case, grpc needs this update to get his dependency versions right because of his own update. The value of builtin-baseline
field is simply the commit hash of the vcpkg
repo.
Sorry to bother you guys. Recently, multipass team has been dealing with rebasing the patches (code commits on the top of the release tag) of grpc after updating grpc to newer version. It has been a painful process due to the size of the patches and conflicts with the grpc refactors and changes. This really got us thinking what is the exact reason to keep these patches. Why multipass has a unique use case from the regular ones? By looking at commit messages and the code, my superficial understanding is that. Multipass wants to support skipping verifying server certificate. Because of that, the Any information would be appreciated in case you guys know about it, thanks a lot. |
Basically to support SSH-style authorization. Both sides generate their encryption keys in isolation, without a CA between them. All communication is encrypted, and the key fingerprints are what determines whether they want to talk to one other on a higher level, not whether the certificates are valid or not. |
Thanks for the reply. But why exactly does multipass have to use the SSH-style authorization? The standard TLS/mTLS always requires the server certificate and verifies it. As a result, grpc api only provides My wondering here is whether this orthodox way has some inadequacies we are not aware of or it somehow does not fit in the multipass use case. Any info will be appreciated, thanks. |
Hey @georgeliao, As @Saviq alluded to, it was done this way to avoid having to use a CA and generate valid SSL certificates. In the early days when this was implemented, we found it to be easier to just modify gRPC and carry the patches than to deal with a CA. If you all want to deal with a CA and generate valid SSL certs in order to use what gRPC already supports, then that's a technical decision the Multipass team will need to make. Maybe it's easier now than in 2018. 🤷♂️ Care will also need to be taken with the whole authorization plumbing, but I think that should still work. |
This was the only way to have clients other than the first/default one to get access to the daemon simply by providing a shared passphrase. If you go for the CA approach, you have to generate a key pair and a certificate signing request and ask someone with access to the CA (admin) to sign your CSR and provide you back with the signed certificate. There's also no path to accepting the first client like we do today on Linux and macOS, you'd need to somehow get through the above dance in the installation phase. Using a passphrase is just a more user-friendly way to achieving the same level of security, is all (one missing bit in the current scheme is managing a list of accepted daemon certificates, |
@townsend2010 |
816e265
to
114fe3f
Compare
114fe3f
to
6b9d382
Compare
The base branch was changed.
…ld time and lib size.
…, which is non-intrusiveserver address resolver from intrusive change to
Co-authored-by: Ricardo Abreu <6698114+ricab@users.noreply.github.com> Signed-off-by: George Liao <georgeliaojia@gmail.com>
fc97bbb
to
e795fce
Compare
@ricab , @xmkg @Sploder12 can any of you approve this again so we can start the merge queue? |
@Sploder12 Can you do some sanity testing on various platforms since this PR has been rebased several times before the approval? Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Windows with installer, Snap and not-Snap Ubuntu gave me no issues while testing!
I have tested on macos with the installer. Things are fine. |
close #3568
MULTI-1176
git submodule
1. vcpkg, updated to 2025.01.13.
vcpkg managed:
1. grpc, 1.52.1-> 1.68.2.
The vcpkg grpc build recipe (located in
<multipass_path>/3rd-party/vcpkg-ports/grpc
) particularlyportfile.cmake
, is based on the standard grpc build recipe found in the directory<multipass_path>/3rd-party/vcpkg/ports/grpc
. Therefore, an effective way to view the actual change is by comparing these two directories using tools like Beyond Compare ordiff
.Most changes are centered around one thing, that is eliminating unneeded library build. As a result, it significantly reduced the grpc build time. To achieve this,
remove_unneeded_lib_custom.patch
was introduced. To review the content of the patch, you can compare it with the original patch applied to grpc 1.52.Although maintaining this patch can be cumbersome, the build time savings justify the effort. The grpc build time was reduced from 9 mins 53 seconds to 4 mins 40 seconds on my 20 hardware threads machine.
Another deviation from the standard build recipe is the removal of openssl from the
3rd-party/vcpkg-ports/grpc/vcpkg.json
, this follows the setup of the old version. The purpose is to prevent vcpkg pulls openssl. Consequently, the cmake will find system installed openssl. As to whether we want to migrate to the vcpkg openssl from the system openssl, that can be discussed within the team and done in another PR.Additionally, there are some api calls changes on multipass side. Those are made to accomodate some grpc' code changes in this duration which broke the backward compatibility. The particular commit is this one. There is also another github issue which has relevant discussions about this. This change affects grpc version 1.57.0 and onwards. The fix we adopt is the one mentioned in the github issue, where they call
grpc.insecure_channel(srv_endpoint, options=(('grpc.default_authority', 'localhost'),))
on python client to overwrite the default authority, the C++ equivalent to this would be changinggrpc::CreateChannel
calls togrpc::CreateCustomChannel
calls with channel arguments, see below code snippet for detailsBe aware that this change affects all types of address resolvers, that probably is fine in the use cases of multipass.
snap:
One small note: the update of vcpkg changes the file structure in
scripts
folder, so you will need to re-run the bootstrap script (./bootstrap-vcpkg.sh
on linux and./bootstrap-vcpkg.bat
on windows) in<multipass_src_path>/3rd-party/vcpkg
directory.