Is this the kind of thing you're looking for? Then all I do is make sure the output is correct when running from ulab, and after eyeballing and finding no discrepancies then we're good to go?

Yes.

Unfortunately, the tests will fail now, given that master has changed since this PR.

@Derfies The bitwise tests fail, because the output is truncated. You can set the output format so: https://micropython-ulab.readthedocs.io/en/latest/ulab-ndarray.html#customising-array-printouts

@Derfies These tests still fail: https://github.com/v923z/micropython-ulab/actions/runs/5263105317/jobs/9513625274?pr=625.

I have the feeling that you're trying to compare numpy's output to ulab's. The usual way of doing the tests is to create a test script, run it with ulab, manually compare the results to numpy's, and if they are OK, then you can copy the expected file to the test folder: https://github.com/v923z/micropython-ulab#testing

If you do that, then you don't have the issue with the printout, either.

Sorry @v923z I think I'm fundamentally misunderstanding the testing procedure here :(

The scripts I've edited - these are meant to be executed with micropython / ulab and results saved to the identically named *.exp file alongside the test. Afterwards both these files are committed.

The build system runs all test scripts and compares the output to the *.exp file and if there's a discrepancy, the build fails. Makes sense.

But somewhere in between these steps the script should be executed with numpy and the result manually eyeballed to see whether the ulab result differs from the numpy result for parity's sake. Is this correct? And if this is the case, would we not want set_printoptions called in both scenarios and the output to be exactly the same so that eyeballing the results is as simple as checking whether we have a clean diff?

But somewhere in between these steps the script should be executed with numpy and the result manually eyeballed to see whether the ulab result differs from the numpy result for parity's sake. Is this correct?

Yes, that's correct.

And if this is the case, would we not want set_printoptions called in both scenarios and the output to be exactly the same so that eyeballing the results is as simple as checking whether we have a clean diff?

In your last commit, the results were OK, the problem was that ulab inserts commas between the entries of a tensor, while numpy doesn't. Even if we fixed that, that still wouldn't help, because numpy does a much better job at formatting the output than ulab. That could definitely be improved upon, but first, no-one has complained about it so far, second, this is a cosmetic detail that does not effect the results, and I just didn't think that it's worth the effort, speed, and flash space.

If you ask the question, what is more valuable, nice-looking displays of 3-by-3 matrices, or a function that does something useful for someone, then my vote would always be for the second. Given that I have a limited amount of free time, I've got to set priorities here.

Thanks for taking the time to clarify. Build seems to pass locally now after making some changes, with the exception of a single test for left_shift where a sign seems to have flipped.

If you think it's at all useful, I could add some printing functionality to go the other way, ie make numpy's output more similar to the default ulab string representation of an array, which could possibly remove the need to check against the numpy results manually and add that instead to the build process.

@jepler This fails with the circuitpython tests: https://github.com/v923z/micropython-ulab/actions/runs/5273259777/jobs/9570362763 Do we miss something here in the build script?

@dhalbert what's the new incantation?

@jepler Is it OK, if I merge this with the circuitpython error?

@Derfies Could you, please, close this PR, and open a new one against master? We've fixed the glitches, and the bitwise operators are already in master, only your tests are missing.