Skip to content

Conversation

zaddach
Copy link
Contributor

@zaddach zaddach commented Mar 17, 2023

Fixes #20619
Unblocks #24894
Unblocks #24402

Some ports, such as clamav or librsvg require an installed Rust toolchain. I've taken inspiration from Lord-Kamina's work linked in #24402 to download static rust binaries.

I need help to figure out how to detect the system rust correctly, my current attempt with find_program isn't finding my system's installation in $HOME/.cargo.

@zaddach
Copy link
Contributor Author

zaddach commented Mar 17, 2023

@FrankXie05 I saw you posted on #24402. Could you help me getting this to work?

@Neumann-A
Copy link
Contributor

This should be moved into a script helper port instead of into scripts

@zaddach
Copy link
Contributor Author

zaddach commented Mar 17, 2023

@Neumann-A Do you have an example of a script helper port that I can use as scaffolding?

@Neumann-A
Copy link
Contributor

@Neumann-A Do you have an example of a script helper port that I can use as scaffolding?

vcpkg-cmake ?

@FrankXie05
Copy link
Contributor

@MonicaLiu0311 MonicaLiu0311 added the category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly label Mar 20, 2023
@dg0yt
Copy link
Contributor

dg0yt commented Mar 20, 2023

@MonicaLiu0311
Copy link
Contributor

@zaddach
Thanks for your contribution, is this PR still working?

@FrankXie05
Copy link
Contributor

https://github.com/microsoft/vcpkg/blob/master/scripts/cmake/vcpkg_find_acquire_program.cmake

Please don't touch this.

Maybe we can package rust into a macro.

@Neumann-A
Copy link
Contributor

Maybe we can package rust into a macro.

No the way to do this is to simply provide a vcpkg-rust port. (just like vcpkg-cmake). There shouldn't be any new functionality be added into scripts/cmake

@FrankXie05
Copy link
Contributor

I mean, as a port, like python. :)

@zaddach
Copy link
Contributor Author

zaddach commented Apr 3, 2023

No the way to do this is to simply provide a vcpkg-rust port. (just like vcpkg-cmake). There shouldn't be any new functionality be added into scripts/cmake

Okay, I'll give that a shot.

@zaddach
Copy link
Contributor Author

zaddach commented Apr 4, 2023

I've transformed the script into a helper port. It seems to work (on Windows), there are still a couple of open points:

  • A system rust installation isn't recognized correctly. I've tried to use find_program() here, but that doesn't find my system Rust in ~/.cargo/bin/cargo.exe, even though it's in my $env:PATH. I appreciate any help on how I can detect the system cargo binary in a better way.
  • Currently I've only added packages for Windows MSVC x86 and x64. I think those should work both for native and mingw installations.
  • Linux support would need figuring out if gnu libc or musl libc is needed.
  • MacOS, FreeBSD and NetBSD support should be straightforward, but I don't have the system to test (readily available).

Here is the ClamAV 1.0.0 package PR that uses this package for testing: #30660

@MonicaLiu0311
Copy link
Contributor

MonicaLiu0311 commented Apr 17, 2023

@BillyONeal Can you take a look at this PR?

@Neumann-A
Copy link
Contributor

but that doesn't find my system Rust in ~/.cargo/bin/cargo.exe

On windows the environment gets sanitized so you need to pass it explicitly via the triplet or forward PATH via the pass-through mechanism for env variables.

@zaddach
Copy link
Contributor Author

zaddach commented Apr 18, 2023

On windows the environment gets sanitized so you need to pass it explicitly via the triplet or forward PATH via the pass-through mechanism for env variables.

From what I read about passing through env variables, my understanding is that it's very much intended to not use the system's tools and prefer to install a portable tool(chain) in the interest of reproducibility. What's irking me here is that vcpkg finds my system Rust and uses it when it's present, which feels like going against this philosophy.

My takeaway is that it's okay to add the rust toolchain even on systems that have already (a system) one installed.

@zaddach
Copy link
Contributor Author

zaddach commented Apr 19, 2023

I've updated the rust packages and limited architectures and platforms to windows, linux, x86, x64. It's easy to add more architectures or platforms (as long as they're compiled by Rust), but I'd only add them when the need arises (and somebody can assist me in debugging on that platform).

I also have a script for generating the rust package section in vcpkg_rust_install.cmake. Is this something that should go into vcpkg as a helper script, or not?

@MonicaLiu0311
Copy link
Contributor

@BillyONeal User wants to add Rust cargo to vcpkg in such a way, could you please take a look at it?

@zaddach zaddach mentioned this pull request May 9, 2023
6 tasks
@MonicaLiu0311
Copy link
Contributor

I also have a script for generating the rust package section in vcpkg_rust_install.cmake. Is this something that should go into vcpkg as a helper script, or not?

Thanks for your contribution, the script is nice, but it might not be accepted:

  • SHA512 can be manually specified in vcpkg_rust_install.cmake;
  • There are currently no packages that need to be imported in vcpkg;
  • We have sent an email about this PR, and will continue to pay attention to it.

@MonicaLiu0311
Copy link
Contributor

Note: I will be converting your PR to draft status. When you respond, please revert to "ready for review". That way, I can be aware that you've responded since you can't modify the tags.

@MonicaLiu0311 MonicaLiu0311 changed the title Rust support [vcpkg-rust] Rust support May 17, 2023
@MonicaLiu0311 MonicaLiu0311 marked this pull request as draft May 22, 2023 09:51
@sakiodre
Copy link
Contributor

Any chance of merging yet?

@MonicaLiu0311
Copy link
Contributor

Any chance of merging yet?

Yes, reasons for hold see: #31748

endif()

# If the package doesn't match the host triplet, bail here
if(NOT HOST_TRIPLET MATCHES "^${arg_ARCH}-${arg_PLATFORM}.*$")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to test against something else then the triplet name! In general -> if native: check against VCPKG_CMAKE_SYSTEM_NAME otherwise test against CMAKE_HOST_WIN32 or something similar. Triplet names can be unrelated to the host/target

@talregev
Copy link
Contributor

talregev commented Jul 13, 2025

@zaddach
Do you wish to continue work on this port?

@zaddach
Copy link
Contributor Author

zaddach commented Jul 21, 2025

Hi, @talregev. My understanding was that there's not much interest in integrating with Rust in vcpkg at the time when I opened the PR. I haven't followed the discussion about integrating with Rust recently, if the stance has changed I'd be happy to bring the PR up to speed and have it reviewed. Without having Rust as a tool available there's no point in trying to integrate packages that rely on Rust (like ClamAV, in my case).

@talregev
Copy link
Contributor

talregev commented Jul 21, 2025

Hi, @talregev. My understanding was that there's not much interest in integrating with Rust in vcpkg at the time when I opened the PR. I haven't followed the discussion about integrating with Rust recently, if the stance has changed I'd be happy to bring the PR up to speed and have it reviewed. Without having Rust as a tool available there's no point in trying to integrate packages that rely on Rust (like ClamAV, in my case).

I am not the maintainer, I just contribute to vcpkg. I am interesting in rust because I want to update the librsvg, and not just me:
#24402
https://github.com/microsoft/vcpkg/blob/master/ports/librsvg/portfile.cmake#L1

You mean you are willing to work on the PR if there is ok from maintainers to do so?

@dg0yt
Copy link
Contributor

dg0yt commented Jul 21, 2025

There is not a lack of interest. IMO the main barrier is cargo. Downloading crates as neeeded with cargo is convenient ... unless you have to go through a different download, caching and dependency tracking mechanism.

@zaddach
Copy link
Contributor Author

zaddach commented Jul 22, 2025

@MonicaLiu0311 Since tool ports aren't a thing any more, and the reason for hold should be gone, will you merge the PR if I bring it up to speed?

@talregev
Copy link
Contributor

@BillyONeal Can you give your opinion?

@gerard-ryan-immersaview
Copy link
Contributor

There is not a lack of interest. IMO the main barrier is cargo. Downloading crates as neeeded with cargo is convenient ... unless you have to go through a different download, caching and dependency tracking mechanism.

Just trying to get a better scope of the problem, but after skimming the crates.io documentation, it looks like we could emulate the registry on disk, which could give us an opportunity to integrate with vcpkg caching, then I imagine we could configure cargo to use the disk registry. Is this the type of thing we are looking to solve, or did I miss something important?

@dg0yt
Copy link
Contributor

dg0yt commented Aug 23, 2025

I don't fully understand how the ports would interact with this registry, but it might be more complicated.

Literally, "dependency resolution is a global problem, not a local problem".

I always wonder how I would collect all the licenses and attribution and sources I would need to pass or provide alongside my app. If I can't answer that for some package, I cannot use that package.

@gerard-ryan-immersaview
Copy link
Contributor

I see. So the complicated issue is that we need all dependency managers to agree on compatible dependency resolutions.

Well, I'll assume there is no good reason to change how vcpkg resolves dependencies to continue the discussion, which, from my understanding, involves essentially one solved dependency graph for each release, allowing for some tweaks for end users. And then we override environment variables, etc, so tools like pkgconfig use the vcpkg dependencies. So the logical path for a solution would be to coerce cargo to follow the same restrictions.

Which sounds theoretically possible by replacing the registry.
The consequences of that path would probably make many crates incompatible, as cargo isn't so strict with dependency versions.
But that sounds like a solvable problem to me; it'd just be more work for port writers to make the crates compatible, hopefully upstream.
The other related consequence is how these incompatibilities are discovered through vcpkg. I haven't looked myself, but I'd imagine the error would be similar to something like meson if you overrode a vcpkg dependency version to an incompatible version.

I agree it's a complicated direction, but if I understand this problem correctly, vcpkg and cargo take two different approaches to the same dependency problem, so either these ecosystems work closely to align better, or we find something that will work, albeit with some complications and compromises.
But if there is enough interest...

@dg0yt
Copy link
Contributor

dg0yt commented Sep 2, 2025

... in particular for a libxml2 wrapper, https://github.com/KWARC/rust-libxml.
Noticed via #47155, somehow highlighting why you don't want premature integration across package managers ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:new-port The issue is requesting a new library to be added; consider making a PR!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support cargo (Rust) libraries
10 participants