Skip to content

lib/attrsets: make getFirstOutput support strings #410779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

pbsds
Copy link
Member

@pbsds pbsds commented May 25, 2025

EDIT: title was "lib/attrsets: make getBin and co respect outputBin if set"

As a motivating example I added a commit that sets sdl2-compat.meta.mainProgram (to match SDL_compat). It installs its binary to $dev/bin/ due to outoutBin = "dev", since the binary is typically only used during configurePhase. This change makes lib.getExe sdl2-compat point at the right path.

  • lib/attrsets: make getFirstOutput support strings
  • buildRustCrate: change outputDev from listOf str to str
  • lib/attrsets: make getBin and co respect outputBin if set
  • doc/stdenv/multiple-output: document $outputInclude
  • sdl2-compat: set meta.mainProgram

EDIT: dropped two commits, since this path is not the way to go, but the fixes along the way still apply. I'll open a treewide pr for the alternative approach

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: lib The Nixpkgs function library 8.has: documentation This PR adds or changes documentation labels May 25, 2025
@pbsds pbsds requested a review from roberth May 25, 2025 12:29
@pbsds pbsds force-pushed the feat-getbin-sdl-1747653458 branch from 2651d7f to 2f03be6 Compare May 25, 2025 13:01
@github-actions github-actions bot added the 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. label May 25, 2025
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to more input, but based on the provided info, I'd recommend against this for cost/benefit reasons.

The outputDev etc variables have been internal to setup.sh.
It complicates getBin, which I believe should be simple, so that it is easy to understand how an output is found.

We don't seem to have needed this, and SDL compat doesn't seem like a good motivating example to me.
The sdl-compat program seems to be a relic of the pre-pkg-config past. Configure scripts should probably use pkg-config instead. That also more straightforward for cross compilation.

Do we really need this? It seems that we'd impose the cost of this questionable solution on all Nixpkgs users, by imposing the complexity of this solution on all packages.

@github-actions github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels May 25, 2025
@pbsds
Copy link
Member Author

pbsds commented May 25, 2025

A rg 'outputBin = "dev' detects 45 unique nix files, including pkgs/development/libraries/gtk/4.x.nix.

I agree that the eval overhead is not pretty. Would you prefer we switch to pkg.${pkg.outputBin} or pkg.bin or ... chains to avoid all the list allocations, or would you prefer we passthru the dev output as bin in each package instead?

@pbsds pbsds force-pushed the feat-getbin-sdl-1747653458 branch 2 times, most recently from 531438d to 4630d5d Compare May 25, 2025 13:20
@pbsds pbsds marked this pull request as ready for review May 25, 2025 13:29
@pbsbot
Copy link

pbsbot commented May 25, 2025

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 410779


x86_64-linux

⏩ 1 package marked as broken and skipped:
  • lua54Packages.luaevent
⏩ 2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
✅ 31 packages built:
  • gwe
  • libreoffice-qt (libreoffice-qt-still)
  • libreoffice-qt-fresh
  • lua51Packages.luaevent
  • lua51Packages.luaexpat
  • luaPackages.luaevent (lua52Packages.luaevent)
  • luaPackages.luaexpat (lua52Packages.luaexpat)
  • lua53Packages.luaevent
  • lua53Packages.luaexpat
  • lua54Packages.luaexpat
  • luajitPackages.luaevent
  • luajitPackages.luaexpat
  • nixpkgs-manual
  • prosody
  • python312Packages.gftools
  • python312Packages.gftools.dist
  • python312Packages.notobuilder
  • python312Packages.notobuilder.dist
  • python313Packages.gftools
  • python313Packages.gftools.dist
  • python313Packages.notobuilder
  • python313Packages.notobuilder.dist
  • sile
  • sile.dev
  • sile.doc
  • sile.man
  • tests.buildRustCrate.test
  • tuxclocker
  • tuxclocker-nvidia-plugin
  • tuxclocker-plugins-with-unfree
  • vinegar

@roberth
Copy link
Member

roberth commented May 25, 2025

This works fine:

nix-repl> :b stdenv.mkDerivation { name = "test"; stictDeps = true; buildInputs = [ sdl2-compat ]; dontUnpack = true; configurePhase = "sdl2-config --help"; }     
...
       > Usage: /nix/store/3r1cplqpqfxc3j3my2ma0l311rs932qy-sdl2-compat-2.32.54-dev/bin/sdl2-config [--prefix[=DIR]] [--exec-prefix[=DIR]] [--version] [--cflags] [--libs]
...

What's the use case for getBin?

@roberth
Copy link
Member

roberth commented May 25, 2025

If convenience for the getExe use case is the goal (a narrow goal, but alas), it'd be hard to beat passthru.exe = "${getDev finalAttrs.finalPackage}/bin/sdl2-config";.

That could then even be made compatible with getExe using the lib part of https://github.com/NixOS/nixpkgs/pull/173675/files

@pbsds pbsds force-pushed the feat-getbin-sdl-1747653458 branch from 4630d5d to 86d1558 Compare May 25, 2025 15:09
@pbsds
Copy link
Member Author

pbsds commented May 25, 2025

  • lib/attrsets: make getBin and co respect outputBin if set
  • sdl2-compat: set meta.mainProgram

I dropped the two commits named above, but kept the rest since the fixes along the way still apply. I'll open a treewide pr for the alternative approach

refs for posterity:

@pbsds pbsds changed the title lib/attrsets: make getBin and co respect outputBin if set lib/attrsets: make getFirstOutput support strings May 25, 2025
@github-actions github-actions bot added 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. and removed 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels May 25, 2025
@pbsds pbsds mentioned this pull request May 25, 2025
13 tasks
@emilazy
Copy link
Member

emilazy commented May 26, 2025

This works fine:

nix-repl> :b stdenv.mkDerivation { name = "test"; stictDeps = true; buildInputs = [ sdl2-compat ]; dontUnpack = true; configurePhase = "sdl2-config --help"; }     
...
       > Usage: /nix/store/3r1cplqpqfxc3j3my2ma0l311rs932qy-sdl2-compat-2.32.54-dev/bin/sdl2-config [--prefix[=DIR]] [--exec-prefix[=DIR]] [--version] [--cflags] [--libs]
...

What's the use case for getBin?

It does not work if strictDeps is spelled correctly.

@roberth
Copy link
Member

roberth commented May 26, 2025

It does not work if strictDeps is spelled correctly.

🤦
Wish I could finish those mkDerivation refactors yesterday, but we're ways away from having a checked variant of it.

nativeBuildInputs works, but that would get the wrong splice.
I suppose dev should use input propagation to add itself as a native build input? @Ericson2314

@pbsds
Copy link
Member Author

pbsds commented May 30, 2025

is this amputated PR mergeable or is there reason to maybe want to reintroduce the dropped commits?

@roberth
Copy link
Member

roberth commented Jun 20, 2025

Yeah it's close.

The lib.getFirstOutput commit should come with a test case and a documentation update.
Normally we/lib maintainers would recommend a separate PR for lib changes, but the other commits aren't any more urgent so it may be ok to keep it here.
(Unfortunately getFirstOutput has no direct tests, so we're overdue for creating some. Can be added to lib/tests/misc.nix)

The buildRustCrate commit seems fine, but a little explanation in the commit message would be good for other maintainers to understand the change.

The outputInclude docs look ok to me.

@pbsds pbsds force-pushed the feat-getbin-sdl-1747653458 branch from 86d1558 to 379f2c6 Compare June 25, 2025 01:44
pbsds added 5 commits June 25, 2025 03:44
`outputDev` is not expect any `listOf str`, only `str`.
This is the only instance of `outputDev` being set to a list in nixpkgs.
This error was likely masked by `toString` behavior for lists.
@pbsds pbsds force-pushed the feat-getbin-sdl-1747653458 branch from 379f2c6 to d36a1ed Compare June 25, 2025 01:44
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: lib The Nixpkgs function library 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants