-
-
Notifications
You must be signed in to change notification settings - Fork 16.7k
various: avoid throwing for invalid platforms combinations in CI #426645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
various: avoid throwing for invalid platforms combinations in CI #426645
Conversation
These asserts should only trigger when a user tries to use that combination. In CI, they should just not create an attrpath to evaluate.
Same reasoning as commit before.
Same reasoning as in the commit before.
This is a bit awkward, and I'm concerned it's a bit fragile of a solution. Nothing is to stop future contributions to reintroduce similar problems and I'm not sure how we can expect people to know about this concern. Speaking only for the BEAM ecosystem in this PR, we just need a way to properly represent the matrix of compatibility between components and expose those in a user friendly way (e.g. not using the context-less We've long had an accepted PR that seems a better solution all around, https://github.com/NixOS/rfcs/blob/master/rfcs/0127-issues-warnings.md Perhaps this is a good motivator to finally implement it? |
Once #426629 is merged, CI would tell you that there is a problem if you introduced such a pattern again. |
That's great and does resolve the reintroduction concern, thanks. :) I'm curious about what the message will look like and how it'll guide contributors to fix the problem, but not directly relevant to this PR. |
While I agree that Let's put it that way: There is currently no infrastructure to report these problems. Using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hopefully we get can better infrastructure for this in the future (e.g. problems?) but for now this should be fine.
When I revert the
So if you reintroduce such a throw, eval will... show you exactly that throw. Which is exactly what we want. There's not much more that we can do on top, because the error messages can't be caught and rethrown :/ (Edit: added more of the stacktrace) |
That should be clear enough, thanks! |
# Don't throw without aliases to not break CI. | ||
null | ||
else | ||
assert assertMsg bothAssert compatibilityMsg; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the description of the PR, I think you just need to switch between fail-fast and fail-safe. Correct me if I'm wrong! So, can we define an assert
function that can switch between the fail-fast and fail-safe to make sure the condition of if
is exactly the same as the condition of assertion? It will also help you avoid redundant if-else all over this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, can we define an
assert
function that can switch between the fail-fast and fail-safe to make sure the condition ofif
is exactly the same as the condition of assertion?
If you look closer, these patterns are not the same in every case. The assert
is not always the first thing in else
directly. Sometimes it's much deeper down. So the helper can't extract the assert itself.
The only thing that's similar is if !config.allowAliases && !condition then null else
. But I can't use config.allowAliases
in lib/
.. and now the benefit of writing up a helper in lib/
diminishes a lot..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
they are not the same! that's my point! why are they different?
your other PR today makes more sense, since you moved the assertion so they will be run lazily, but the changes here feels wrong!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume you mean #426642. There is a difference for that: These packages have an unfree license and are filtered out by CI that way. Thus, the lazy throw will never happen.
To achieve the same here, we'd need to mark these packages as meta.broken
. But that defeats the point of having these throws in the first place: They are a mechanism to work around the fact that meta.broken
is boolean only and doesn't allow to communicate a reason for this brokenness.
The solution to this is meta.problems
as mentioned above.
So no, the lazy throw approach will not work here.
they are not the same! that's my point! why are they different?
I looked into this a bit. At first glance it seems possible to move the throw
in androidndk-pkgs
to the top, right after the else, too. But it becomes more complicated: There are two different error messages thrown, depending on the specific case. We can't unify these without losing information. I played this through: These multiple throws won't work well with a helper function like this (which I assume you roughly have in mind):
lib.assertIf config.allowAliases
bothAssert compatibilityMsg
stdenv.mkDerivation {
...
At this stage, I'd rather defer the implementation of helper functions to later. That's because #426629 lists many similar problems for other platforms - and I assume there are other special cases to deal with. So it would probably be better to implement this ad-hoc if-else pattern a bit more often and then see which other cases come up, before simplifying. Although, it's arguably also a good idea to implement meta.problems
first, before spending more time on this - maybe a lot of these cases already go away that way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for explanation. The assertIf
is exactly what I was referring to, but in the end it makes sense. Even though it would be better if we had a trace of this deferring implementation in code directly by commenting a TODO, but I approved this PR.
pkgs/top-level/all-packages.nix
Outdated
if !config.allowAliases && stdenv.targetPlatform == stdenv.hostPlatform then | ||
# Don't throw without aliases to not break CI. | ||
null | ||
else |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a problem with returning null
here. gccWithoutTargetLibc
is an input to gccCrossLibcStdenv
. By returning null
the build of the latter is not blocked anymore. This is why we get "1 rebuild" here - a "new" package gccCrossLibcStdenv
. where targetPlatform == hostPlatform
.
Ofc, this doesn't make sense to build...
The other packages in this PR are fine, in principle, but just by accident - because they are not dependencies of other attributes listed for CI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed this case to start overriding meta.badPlatforms
for gccWithoutTargetLibc
. This is not super-nice, because we're losing the better stack-trace of the assert, but I don't see a better way. meta.problems
would help here, too, being able to provide a better message about it...
I also tried to just do the same null
handling for gccCrossLibcStdenv
- but this just makes the problem worse, because then this is passed around as null
and causes further eval and build failures.
98eb56d
to
5dc979c
Compare
An assert can't be caught by CI for generation of attrpaths during Eval. An error reported via meta.badPlatforms can be. The alternative of returning null in the CI case, similar to the commits before, is not helpful, because it only causes failed builds or eval downstream - when this dependency is passed on and used as `null`.
5dc979c
to
6de5573
Compare
TLDR: The "I want better error messages for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing to add
Some attributes are not available on all platforms. In most cases, these are just marked via
meta.platforms
and then become "unsupported on this system". In these cases, explicit asserts/throws were added, though.When they throw, they are similar to aliases - but when they don't, they should be available regularly, ofc. By returning null, these will be ignored by CI if unsupported, without the currently hacky solution of ignoring all asserts/throws.
There are probably a lot more cases for non-x86_64-linux systems - this PR only fixes these for that platform. I'm open for discussion about whether any common helper functions could be extracted for this pattern, but wasn't sure how.
lib
doesn't haveconfig.allowAliases
access, so we wouldn't really save much. Maybe inpkgs-lib
? Not sure.Part of #426629, ultimately targeting #397184.
Things done
Add a 👍 reaction to pull requests you find important.