Skip to content

TestContext.Progress shows up as Warning in TFS/Azure Dev Ops #1037

@mharwig

Description

@mharwig

Version information

Issue

TestContext.Progress written during a test shows up in TFS/Azure DevOps as warning when running a test via a VSTest task with NUnit Adapter 4.3.1.0. ConsoleOut is enabled via runsettings.

Code used to write progress in test:
TestContext.Progress.WriteLine("Message via TestContext.Progress");
Expected result white (informational) log line in TFS. Instead the message is shows as a warning (orange) warning in TFS:

ProgressMessageAsWarningInTFS

Possible cause

My theory regarding the cause of the issue based on analysis of the code (did not run/debug the code):
The text written via TestContext.Progress.WriteLine results in a TestOutput event in EventListenerTextWriter:

private bool TrySendToListener(string text) { context.Listener.TestOutput(new TestOutput(text, _streamName, context.CurrentTest?.Id, context.CurrentTest?.FullName));

The TestOutput event is in handled in nunit3-vs-testadapter in class NUnitEventListener OnTestEvent:

public void OnTestEvent(string report) { . . . case NUnitTestEventHeader.EventType.TestOutput: TestOutput(new NUnitTestEventTestOutput(node)); break;

The handled TestOutput is always forwarded as warning in NUnitEventListener TestOutput:

public void TestOutput(INUnitTestEventTestOutput outputNodeEvent) { . . . recorder.SendMessage(TestMessageLevel.Warning, text); }

Possible solution:
recorder.SendMessage(outputNodeEvent.IsProgressStream ? TestMessageLevel.Informational : TestMessageLevel.Warning, text);
or
recorder.SendMessage(outputNodeEvent.IsErrorStream ? TestMessageLevel.Warning : TestMessageLevel.Informational, text);
Further more one could argue that if settings.ConsoleOut == 1 the message level should also be TestMessageLevel.Informational, similar to the message sent in NUnitEventListener TestFinished.

Apologies if the issue turns out to be caused by something else entirely (e.g. configuration on my side).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions