TLDR

I'm using AVA 4.0.0-rc.1 and have a few tests marked as ".failing" since they are known test failures. When looking at the test output, I expect that I can tell which tests are known failures, but known-failing tests look exactly like passing tests in some situations.

Details

AVA's default output doesn't clearly show which of the tests are known failures because it only distinguishes between test successes and test failures using color. When you only use color to distinguish between different things in an interface, you are making those things inaccessible to some people.

Here's some AVA test results on my terminal. You can see that both known failures and successes have the ✔ checkmark prepended and they only differ based on their color:

This is how those same AVA test results appear on GitHub's CI, which is similar to how red-green color-blind would see it. There's no way to tell which tests are known failures:

We need to re-think how these test results are displayed so they are accessible to everyone, including:

• developers who are red-green colorblind
• developers using screen readers
• developers who don't have a color-enabled terminal
• developers reading a plain text log file (like in a CI)

Possible solutions

As a reference, here's a screenshot of skipped, failing, passing and known failing tests:

Maybe known failures should be in "skipped" yellow with a ✖ cross instead of the checkmark. Known failures are more like skipped tests than anything else.

Also, right now failing tests only have a red ✖ with normal-colored descriptive text. Can we make the descriptive text of failing tests all in red? And maybe bold too? The bold isn't necessary since failures are printed again at the end of the test result list, but it helps the failures visually stick out from the rest of the list.

As for people who are using screen readers, I think we might need to add text to describe what -, ✖, ✔ mean. Especially if we use ✖ for known and unknown fails. Maybe this?:

  ✔ PASS: Example › should be an example test
  - SKIP: Example › should be an example test
  ✖ FAIL: Example › should be an example test
  ✖ KNOWN FAIL: Example › should be an example test (or maybe SKIP FAIL or SKIPPED FAIL)

Here's a mockup with color and bold added: