Skip to content

Conversation

emilazy
Copy link
Member

@emilazy emilazy commented Aug 3, 2025

Meson really doesn’t want us to get useful information from test failures! There’s no way to disable this limit entirely, so we just set it to something ridiculous.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

Meson really doesn’t want us to get useful information from test
failures! There’s no way to disable this limit entirely, so we just
set it to something ridiculous.
@emilazy emilazy requested a review from alyssais August 3, 2025 21:40
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Aug 3, 2025
@alyssais
Copy link
Member

alyssais commented Aug 4, 2025

There’s no way to disable this limit entirely, so we just set it to something ridiculous.

@eli-schwartz just mentioning you in case you're interested in us having run into this limitation :)

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Aug 4, 2025
@eli-schwartz
Copy link

eli-schwartz commented Aug 4, 2025

Actually I didn't realize that option existed. :D

All the information should be stored in the testlog file inside the meson-logs directory. Is that suitable for your use case?

Btw:

Meson really doesn’t want us to get useful information from test failures!

Eh, this is... very much a matter of opinion. And the test frameworks you use. The cause of a failure is quite likely to appear somewhere in the last hundred lines of an individual test output -- but multi-test protocols (subtests) throw that off and meson should really refrain from printing successful subtests at all, when printing a subtest failure. And then it can print the last hundred lines of a single failing subtest. ;)

Tests which in the normal course of events log a thousand lines of output before failing are unlikely to be a fruitful thing to print to a user's interactive console. A valid point could be made about exporting that information to the console log of a buildbot, but then again, a valid point could be made that in such an event you really want structured results such as the logfile I mentioned, so...

Meson does need good defaults for manual interactive use. And my understanding is that's what the current design was based off of (though admittedly many such decisions occurred years before I became involved with the project, so I could be wrong here).

The option that was added feels awkwardly designed. It's neither one thing nor the other. There isn't much explanation in the PR that added it, for why it's designed the way it is...

mesonbuild/meson#13285 in case you're interested.

@emilazy
Copy link
Member Author

emilazy commented Aug 4, 2025

Ah, sorry @eli-schwartz – that was very much a sentence written for the context of Nixpkgs maintenance rather than Meson maintenance :) In our case, we have the unstructured build log, and the final outputs if the build is successful, and nothing else. Keeping the build directory around after failure is possible, but requires interactive intervention and isn’t an option on our public CI like ofborg and Hydra (they would be too big to keep around, anyway).

More structured build artefacts separate from outputs would of course be nice, but this is what we have to work with. The result is that a failed build where Meson doesn’t print any error logs (before we added --print-errorlogs) does not give us very helpful information and requires annoying manual to work to even start diagnosing the problem. Discovering that GoogleTest in at least some configurations doesn’t deign to repeat the logs from failed tests at the end of the output is how I ended up finding --max-lines and making this PR.

A lot of this is of course our fault and the legacy of a design from before modern CI artefact and structured tracing systems, but it means that we would generally like build tools to print all the context they have in the case of failure. I can understand why --print-errorlogs would truncate – although, if it’s truncating for the convenience of interactive use, then it seems like it’d be good to turn it on by default, too – but in our case we want a --really-print-errorlogs that just dumps the whole thing in case of failure.

We could adjust our Meson test hook to detect the failure case and cat the log, but it seems like it would be better for Meson – which already has the logic to decide when test logs ought to be printed, and already keeps track of where they are – to be able to do it for us, which it can… as long as there’s a cap on how many lines you’ll ever need. I agree that --max-lines doesn’t seem that useful other than for a hack like this and would not complain if it was replaced with something that more directly achieves our goal here. But this works, at a pinch.

@emilazy emilazy merged commit 9c42dc6 into NixOS:staging Aug 10, 2025
32 of 34 checks passed
@emilazy emilazy deleted the push-rrtwkuvqsols branch August 10, 2025 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 12.approvals: 1 This PR was reviewed and approved by one person.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants