Skip to content

Discuss whether TestMethodAttribute.Execute should return a single TestResult object #5309

@Youssef1313

Description

@Youssef1313

TestMethodAttribute.Execute currently returns TestResult[] but all known implementations return a single result.

This seems to currently be an API flaw that has been present only for historical reasons where previously parameterized tests where handled in DataTestMethodAttribute:

public override TestResult[] Execute(ITestMethod testMethod)
{
ITestDataSource[] dataSources = testMethod.GetAttributes<Attribute>(true)?.Where(a => a is ITestDataSource).OfType<ITestDataSource>().ToArray();
if (dataSources == null || dataSources.Length == 0)
{
return new TestResult[] { new TestResult() { Outcome = UnitTestOutcome.Failed, TestFailureException = new Exception(FrameworkMessages.NoDataRow) } };
}
return RunDataDrivenTest(testMethod, dataSources);
}

So, DataTestMethodAttribute had to return multiple test results. The design is now very different and the whole DataTestMethodAttribute isn't actually required now. So I feel like the API should change here and return a single TestResult for simplicity.

Still, we need to be very careful with changes here, in case there are valid use cases for returning multiple results (but I don't see that right now).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: MSTestIssues with MSTest that are not specific to more refined area (e.g. analyzers or assertions)Breaking ‼️

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions