Skip to content

Increase Fidelity of the sbt.testing.Framework Implementation #1107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 13, 2025
Merged

Increase Fidelity of the sbt.testing.Framework Implementation #1107

merged 1 commit into from
Aug 13, 2025
+119 −7

Conversation

dubinsky
Copy link
Contributor

@dubinsky dubinsky commented Apr 4, 2025 •
edited
Loading

  • when Selectors supplied include only TestSelectors and TestWildcardSelectors, filter properties to run by matching their names against the Selectors;
  • added a test that actually runs ScalaCheck via the SBT Test Interface and demonstrates the (now correct) treatment of the Selectors;
  • test also demonstrates two unfixable infidelities in the treatment of the nested properties;
  • thankfully, ScalaCheck's implementation of sbt.testing.Framework is, unlike in some other test frameworks, shared between the platforms (JVM, Scala.js, Scala Native), so the fixes do not have to be replicated for each platform, but:
  • the test needs to supply a testClassLoader: ClassLoader parameter when calling sbt.testing.Framework.runner(); on platforms other than JVM, getClass.getClassLoader is not available, so Platform.getClassLoader: ClassLoader method was added to every Platform; on platforms other than JVM, it returns null - which is fine, since on those platforms sbt.testing.Framework.runner() ignores the testClassLoader parameter anyway.

fixes #1105

This was referenced Mar 16, 2025
Closed
Closed
@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 05bd678 to 7ce3e5f Compare April 4, 2025 20:24
@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 7ce3e5f to da9cb92 Compare April 28, 2025 05:04
@dubinsky
Copy link
Contributor Author

@satorg any thoughts?

@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from da9cb92 to 4a468a9 Compare April 28, 2025 08:44
@dubinsky
Copy link
Contributor Author

@satorg I am starting to suspect that the plan here is to let this fix wither on the wine like the previous contribution in this area (#1031); if this is true, and my contribution is for some reason not wanted, please say so; otherwise - what do I need to do to get this merged?

Thank you!

@satorg
Copy link
Contributor

satorg commented Jun 15, 2025

@dubinsky thank you for the PR and sorry for the long reply!

Personally, I'd love to get both PRs (this and #1031) merged ultimately. I just don't feel very confident about this ScalaCheckFramework implementation, because I personally don't use it and mostly rely on other test framework runners.

Do you think your PR complements #1031 or supersedes it completely?

@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 4a468a9 to 2e33740 Compare June 16, 2025 05:00
@dubinsky
Copy link
Contributor Author

Personally, I'd love to get both PRs (this and #1031) merged ultimately. I just don't feel very confident about this ScalaCheckFramework implementation, because I personally don't use it and mostly rely on other test framework runners.

Code changes have comments, tests are added that verify the change, and issue #1105 has detailed explanations. Is there anything else I can do? Are you going to involve other contributors/maintainers that are more familiar with this area or do you have specific questions?

Do you think your PR complements #1031 or supersedes it completely?

My pull request completely supersedes #1031: it handles both TestSelector and TestWildcardSelector, whereas #1031 handles just TestSelector.

Thank you!

@dubinsky dubinsky mentioned this pull request Jul 27, 2025
Open
@dubinsky dubinsky mentioned this pull request Aug 5, 2025
Open
@satorg
Copy link
Contributor

satorg commented Aug 12, 2025

@dubinsky , thank you for your work on this issue. I found some time to wrap my head around SBT testing API, so that now I think understand what is going on there to some extent. If you are still interested, I guess we can get your PR merged.

However, I wouldn't like to dismiss the effort made in #1031 too, since it was kinda preceding yours. How about if I merge the former PR first, then you'll update yours overwriting conflicting parts and we'll merge your PR right after? Thereby everyone will get involved. @Duhemm, in a case you are still around, let us know if you are OK with this plan please. Thanks everyone!

@satorg satorg mentioned this pull request Aug 12, 2025
Merged
@dubinsky
Copy link
Contributor Author

@dubinsky , thank you for your work on this issue.

My pleasure!

If you are still interested, I guess we can get your PR merged.

Yes.

However, I wouldn't like to dismiss the effort made in #1031 too, since it was kinda preceding yours. How about if I merge the former PR first, then you'll update yours overwriting conflicting parts and we'll merge your PR right after?

Of course! Thank you!

@satorg
Copy link
Contributor

satorg commented Aug 12, 2025

@dubinsky , do you have any clue what serializeTask/deserializeTask methods of ScalaCheckRunner could be for? They look like they are supposed to be called via reflection, but I didn't manage to figure out when and how exactly.

@dubinsky
Copy link
Contributor Author

@dubinsky , do you have any clue what serializeTask/deserializeTask methods of ScalaCheckRunner could be for? They look like they are supposed to be called via reflection, but I didn't manage to figure out when and how exactly.

They are called by the Test Interface/Test Bridge on Scala.js and Scala Native to transport the task information from JVM to Node.js/Native code.

@satorg
Copy link
Contributor

satorg commented Aug 13, 2025

Hi @dubinsky , I've merged #1031.
Feel free to update this one and let's get it merged too.
Thanks!

@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 2e33740 to 1638679 Compare August 13, 2025 05:59
@dubinsky
Increase Fidelity of the sbt.testing.Framework Implementation
0e49970 
- when `Selector`s supplied include only `TestSelector`s and `TestWildcardSelector`s, filter properties to run by matching their names against the `Selector`s;
- added a test that actually runs ScalaCheck via the `SBT Test Interface` and demonstrates the (now correct) treatment of the `Selector`s;
- test also demonstrates two unfixable infidelities in the treatment of the nested properties;
- thankfully, ScalaCheck's implementation of `sbt.testing.Framework` is, unlike in some other test frameworks, shared between the platforms (JVM, Scala.js, Scala Native), so the fixes do not have to be replicated for each platform, but:
- the test needs to supply a `testClassLoader: ClassLoader` parameter when calling `sbt.testing.Framework.runner()`; on platforms other than JVM, `getClass.getClassLoader` is not available, so `Platform.getClassLoader: ClassLoader` method was added to every `Platform`; on platforms other than the JVM, it returns `null`, which is fine since on those platforms `sbt.testing.Framework.runner()` ignores the `testClassLoader` parameter anyway.

fixes typelevel#1105
@dubinsky dubinsky force-pushed the increase-fidelity-of-the-sbt-testing-framework-implementation branch from 1638679 to 0e49970 Compare August 13, 2025 06:02
@dubinsky
Copy link
Contributor Author

update this one and let's get it merged too. Thanks!

Done. Thank you!

satorg
satorg approved these changes Aug 13, 2025
View reviewed changes
Copy link
Contributor

@satorg satorg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@satorg satorg merged commit 0e49970 into typelevel:main Aug 13, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@satorg satorg satorg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

sbt ScalaCheckRunner: loss of test selection fidelity
2 participants
@dubinsky @satorg