pkgs/top-level/all-packages.nix: add cc & ccChooser #409851

RossComputerGuy · 2025-05-22T18:39:59Z

Things done

Introduce ccCross and ccCrossChooser since #365057 will make it easier to switch elements of the toolchain out. By localizing it to a specific mechanism similar to libcCross and libcCrossChooser, it simplifies the addition of new CC's.

This is necessary for working on the new crossStdenvs which will be added in to replace nixpkgs internally calling variants/sub nixpkgs instances.

Add a 👍 reaction to pull requests you find important.

emilazy · 2025-05-22T20:35:37Z

Shouldn’t something like buildPackages.stdenv.cc work for this?

RossComputerGuy · 2025-05-22T20:37:14Z

I'm not sure what you mean.

Ericson2314 · 2025-05-23T03:24:01Z

I'm very skeptical of anything with "cross" in the name. I would need to read more, but that feels to me like a similar problem as variants.

RossComputerGuy · 2025-05-23T03:28:37Z

The difference with this is it is only the stdenv, there's a lot of packages (like Steam) which need some sort of cross environment to invoke. I'm currently working on the machinery for crossStdenv, currently testing the replacement for pkgsCross.arm-embedded.stdenv which is crossStdenv.libc.newlib.configs.arm-none-eabi. As you can see by that name, it follows what was brought up at #400351 (comment).

RossComputerGuy · 2025-05-26T02:01:10Z

I'm very skeptical of anything with "cross" in the name.

Cross is no longer in the name, this is now adding cc and ccChooser. This is aligning more with toolchain attrs and the stdenv redesign design doc.

Ericson2314

Oh I realized we shouldn't do this as is, because cc has the wrong host/target platform cross wise to be a tool in this package set. We should find some idiom to distinguish the tools for a package set from the packages in the package set .

RossComputerGuy · 2025-05-26T05:07:27Z

What makes the host/target platforms wrong for it to be used? I see it just needing to be pulled in like stdenv. We could have a cc and ccTarget. cc itself is meant to go from build -> target while ccTarget runs on the target.

Ericson2314 · 2025-05-26T05:13:41Z

stdenv.cc is already wrong in this regard, but stdenv is at least grandfathered in as a weird thing.

If I see cc, I assume we have something like cc = clang or cc = gcc, but in fact what we have is (after simplifying) cc = buildPackages.clang or cc = buildPackages.gcc. That is unlike, for example, how bintools is an alias.

RossComputerGuy · 2025-05-26T05:24:01Z

Ok, that's fixed. It was actually quite simple to do that. You can verify this by looking at what pkgsCross.mingwW64.cc is versus cc versus stdenv.cc in the cross and host.

Ericson2314 · 2025-05-26T05:48:13Z

OK now it is consistent, but we have another problem, which is that there is not supposed to be just a single next stage. We want as few things to be "target" sensitive" as possible: so it doesn't really make sense to choose cc in the previous stage.

We do in fact want to choose it in the next stage, but not have it be part of that stage.

RossComputerGuy · 2025-05-26T05:51:05Z

I'm not sure what you mean or how to fix that.

Ericson2314 · 2025-05-26T20:08:14Z

https://github.com/NixOS/nixpkgs/pull/21268/files#diff-7f30042ab97e5e7b73fd0e1f2fc383f9d373f70f48e4fa52e9e348eb7db9e2fe this is not quite the same, but basically I was thinking that each bootstrapping could be not a single package set, but a pair of package sets:

Choosing tools/libraries from the previous stage, and applying wrappers to put them all together
The packages built with those wrappers

The ideal goal would be for every proper package set (2) to be target-agnostic, and the only target-specific bits would be the wrappers that are in (1). At that point, we could get rid of stdenv.targetPlatform, since for (1) we are operating from the perspective of the "next" package set and can use stdenv.{build,host}Platform (which would be the host and target platforms of the wrappers themselves.

We won't get there quite a way, so we will have to keep stdenv.targetPlatform around for GCC and other single-target compilers, but we should be using it as little as possible. For things which are multi-target, (1) should be the only time targets are chosen.

If we do this, I believe your idea of making more cross stdenvs boils down to making a bunch of (1) on top of the last stage without a corresponding (2). That might be possible, though building libraries without a full (2) is a challenge.

As a practical matter, we might call (1) a __tools makeScope / sub-package set within the main stage for now, with the obscure name making clear that these are things with morally belong in the previous stage. splicedPackages would have to be updated so __tools is properly spliced (e.g. pkgsHostTarget.__tools overrides pkgsBuildHost for depsBuildHost).

pwaller · 2025-06-01T07:16:39Z

pkgs/top-level/all-packages.nix

@@ -8227,6 +8227,43 @@ with pkgs;
    else
      null;

+  # We can choose:


Comment looks potentially incomplete? Or at least I don't understand what it is trying to convey beyond the rest of the code.

Not incomplete, copies what libcCrossChooser says:

nixpkgs/pkgs/top-level/all-packages.nix

Lines 8224 to 8226 in 36f6e42

# We can choose:

libcCrossChooser =

name:

pwaller · 2025-06-01T07:16:41Z

pkgs/top-level/all-packages.nix

+      # NOTE: we shouldn't imply GCC, ideally we should either fallback, null, or error.
+      pkgs.gcc or fallback;
+
+  # The C compiler to use


"The C compiler to use" -- under what circumstances exactly? Is it a statement about the present or aspirational?

Do I understand that this takes precedence over what's in the standard stdenv? Does it actually do that on linux? I still see for example this (below), did it get missed?

nixpkgs/pkgs/stdenv/linux/default.nix

Line 753 in 0ef2968

cc = prevStage.gcc;

We will be dropping CC from the stdenv in the future so this is one of those steps. All future use of stdenv.cc should be replaced with cc. It is a part of the stdenv and CC wrapper redesign which has been discussed on the stdenv team Matrix.

👍, I understand all that prior to making my comment and have read through the plan and the matrix chat - and I'm generally in favour.

My review comment follows from my belief that ideally, the code and its comments should reflect "the present" -- how things are at the point it is merged. It is confusing if someone is trying to understand how nixpkgs is wired, and they find a comment like this, but it does not currently reflect reality.

I understand the work has to be staged, but I hold a concern that if the work gets interrupted after the merge of this, the commentary in the code remain wrong/incomplete/misleading potentially for a long time. So the commit which makes the statement in the comment true should be the one adding the comment. Before that time, the comment should try to reflect what is true 'in the present', if that can be clarified.

Ok, so what should be changed then?

I suggest either removing the comments (I don't think they're adding much as it is?) or elaborate them so that they precisely reflect the intent of the behaviour as it is currently on the master branch once this lands.

You could even include a reference to the design doc so that the intended development trajectory is clear - that would also lead to it being discoverable by those reading the sources, leading to increased collaboration/support/testing on the project.

I've updated the comment.

axelkar · 2025-08-05T16:48:32Z

pkgs/top-level/all-packages.nix

@@ -8197,6 +8197,44 @@ with pkgs;
      null;

  # We can choose:
+  ccChooser =
+    platform: pkgs: fallback:


Rename to make this more self-explanatory.

Suggested change

platform: pkgs: fallback:

targetPlatform: hostPkgs: fallback:

pkgs.ccChooser (pkgs.stdenv.targetPlatform) pkgs null (where pkgs is what the returned CC targets)

or

Suggested change

platform: pkgs: fallback:

hostPlatform: buildPkgs: fallback:

pkgs.ccChooser (pkgs.stdenv.hostPlatform) pkgs.buildPackages null (where pkgs is what the returned CC will be running on)

Is all-packages.nix really the place for this? Could there be a "Nixpkgs-specific lib"? This function doesn't reference outside variables but takes in pkgs so it's not really a fit for lib.

Or don't take pkgs as a parameter and do what libcCrossChooser does by referencing the with pkgs; at the top of the file.

libcCrossChooser could be simpler, here's how I picture it:

Currently: hello to buildPackages.clang to buildPackages.libcCross to buildPackages.targetPackages.bionic to buildPackages.targetPackages.stdenvNoCC

What it could be: hello to buildPackages.clang to buildPackages.targetPackages.libc to buildPackages.targetPackages.bionic to buildPackages.targetPackages.stdenvNoCC

Can someone explain this?

Edit: I was looking at a two-month old Nixpkgs clone.. But asking on the stdenv team's Matrix channel.

github-actions bot added the 6.topic: stdenv Standard environment label May 22, 2025

github-project-automation bot added this to Stdenv May 22, 2025

nix-owners bot requested review from Ericson2314, philiptaron, emilazy, reckenrode and Artturin May 22, 2025 18:41

RossComputerGuy force-pushed the feat/cc-cross branch 3 times, most recently from 316a0bb to 3e8311b Compare May 22, 2025 18:54

github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels May 22, 2025

RossComputerGuy mentioned this pull request May 22, 2025

lib.systems: introduce toolchain, cc, and bintools attributes #365057

Open

13 tasks

emilazy requested a review from alyssais May 22, 2025 20:35

RossComputerGuy mentioned this pull request May 25, 2025

pkgs/top-level/stage.nix: add crossStdenv #410056

Open

13 tasks

RossComputerGuy force-pushed the feat/cc-cross branch from 3e8311b to aa68505 Compare May 26, 2025 01:53

github-actions bot added the 8.has: documentation This PR adds or changes documentation label May 26, 2025

RossComputerGuy force-pushed the feat/cc-cross branch from aa68505 to 6844021 Compare May 26, 2025 01:54

RossComputerGuy changed the title ~~pkgs/top-level/all-packages.nix: add ccCross & ccCrossChooser~~ pkgs/top-level/all-packages.nix: add cc & ccChooser May 26, 2025

RossComputerGuy force-pushed the feat/cc-cross branch from 8d0fe78 to 2258f1b Compare May 26, 2025 03:31

nix-owners bot requested a review from toonn May 26, 2025 03:33

RossComputerGuy force-pushed the feat/cc-cross branch from 2258f1b to 52ac439 Compare May 26, 2025 04:37

Ericson2314 requested changes May 26, 2025

View reviewed changes

github-project-automation bot moved this to In Progress in Stdenv May 26, 2025

RossComputerGuy force-pushed the feat/cc-cross branch from 52ac439 to 5721a6f Compare May 26, 2025 05:24

RossComputerGuy force-pushed the feat/cc-cross branch from 5721a6f to 8afa4cc Compare May 27, 2025 02:16

RossComputerGuy requested a review from Ericson2314 May 31, 2025 00:09

pwaller reviewed Jun 1, 2025

View reviewed changes

RossComputerGuy force-pushed the feat/cc-cross branch from 8afa4cc to 2441da3 Compare June 2, 2025 13:38

RossComputerGuy mentioned this pull request Jun 5, 2025

libcCross: Remove! #414321

Merged

13 tasks

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 9, 2025

RossComputerGuy added 4 commits June 9, 2025 19:40

pkgs/top-level/all-packages.nix: add cc & ccChooser

124de9e

doc/rl2511: mention cc and ccChooser

b6a1b6e

pkgs/stdenv/darwin: inherit cc to prevent infinite recursion

d33a89d

makeBinaryWrapper: cc no longer an alias

7ef4495

RossComputerGuy force-pushed the feat/cc-cross branch from 2441da3 to 7ef4495 Compare June 10, 2025 02:41

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 10, 2025

axelkar suggested changes Aug 5, 2025

View reviewed changes

	platform: pkgs: fallback:
	targetPlatform: hostPkgs: fallback:

Uh oh!

pkgs/top-level/all-packages.nix: add cc & ccChooser #409851

Are you sure you want to change the base?

pkgs/top-level/all-packages.nix: add cc & ccChooser #409851

Uh oh!

Conversation

RossComputerGuy commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

emilazy commented May 22, 2025

Uh oh!

RossComputerGuy commented May 22, 2025

Uh oh!

Ericson2314 commented May 23, 2025

Uh oh!

RossComputerGuy commented May 23, 2025

Uh oh!

RossComputerGuy commented May 26, 2025

Uh oh!

Ericson2314 left a comment

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented May 26, 2025

Uh oh!

Ericson2314 commented May 26, 2025

Uh oh!

RossComputerGuy commented May 26, 2025

Uh oh!

Ericson2314 commented May 26, 2025

Uh oh!

RossComputerGuy commented May 26, 2025

Uh oh!

Ericson2314 commented May 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pwaller Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

axelkar Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RossComputerGuy commented May 22, 2025 •

edited

Loading

pwaller Jun 1, 2025 •

edited

Loading

axelkar Aug 5, 2025 •

edited

Loading