Skip to content

17.3.0 Fails in CI run #3939

@nohwnd

Description

@nohwnd

TestPlatform 17.3.0 was reported to fail in CI run by #3937 and #3938, the issue seems to b

The issue seems to be that after 17.3.0 introduced the ability to run multiple target frameworks at the same time, we are trying to run .NET Standard libraries which in 17.2.0 would have been excluded.

The workaround (and best practice) is to filter your test dlls down to what you actually want to run.

❌ The default VSTest task pattern is tests.dll, which includes all assemblies with test in name, potentially including resources, and some TestPlatform assemblies

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'

✔️ Instead you should use a pattern specific to your test assembly naming convention. Typically *Tests.dll, or *.Tests.dll name is used for a dll containing tests. In that case it is sufficient to simply pick that dll from the bin folder:

**\bin\**\*Tests.dll
**\bin\**\*Test.dll

✔️ You can also amend the original pattern to exclude more dlls, hopefully including only your desired test dll:

**\*test*.dll
!**\obj\**
!**\*.resources.dll
!**\*TestAdapter.dll
!**\*Microsoft.*TestPlatform*.dll
!**\*testhost*.dll
!**\testcentric.engine.metadata.dll

✔️ Set VSTEST_DISABLE_MULTI_TFM_RUN=1 to revert to the previous behavior (or close to previous).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions