Skip to content

sbt.testing: Run the tests from the suites nested in the explicitly selected one #2357

@dubinsky

Description

@dubinsky

With test code

import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.Suites

class Nesting extends Suites(
  new Nested
)

class Nested extends AnyFlatSpec {
  "success" should "pass" in {}
}

On JVM, when feeding the following TaskDef to ScalaTest via the sbt.testing interface

fullyQualifiedName=Nesting
selectors=[SuiteSelector]
explicitlySpecified=true

org.scalatest.Filter calculated from this TaskDef has:

  • tagsToInclude=Some(Set(org.scalatest.Selected)) and
  • dynaTags.suiteTags that maps Nesting to Set(org.scalatest.Selected)

and with this filter, org.scalatest.Suite.runTests() filters all tests from the Nested suite out, resulting in:

    Nested:
    success

    ScalaTest summary:
    Run completed in 52 milliseconds.
    Total number of tests run: 0
    Suites: completed 2, aborted 0
    Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
    No tests were executed.

We see that although tests from the Nested suite are listed (in green!), none of them are executed.

On the other hand, on Scala.js, feeding the same TaskDef to ScalaTest via the sbt.testing interface results in

    Nested:
    success
    - should pass

    ScalaTest summary:
    Run completed in 48 milliseconds.
    Total number of tests run: 1
    Suites: completed 2, aborted 0
    Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
    All tests passed.

We see that tests from the Nested suite are executed!

Scala.js Gradle plugin integrates with Gradle and IntelliJ Idea, and allows running tests using test frameworks that implement sbt.testing.Framework (including Scalatest) on both JVM and Scala.js. Running Nesting class from IntelliJ Idea translates to running Gradle with --tests "Nesting", which translates to the above TaskDef being fed to ScalaTest. Since this scenario unfolds when the IDE (or CLI) user explicitly runs the Nesting suite, I find the behaviour of ScalaTest on Scala.js - run the tests from the suites nested in the explicitly selected one - more intuitive (less surprising) than on its behaviour on JVM.

Could this inconsistency in the way ScalaTest interprets such TaskDefs on JVM and Scala.js be removed - in favour of the Scala.js approach? ;)

Thank you!!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions