Skip to content

Conversation

erikhuizinga
Copy link
Contributor

@erikhuizinga erikhuizinga commented Sep 17, 2024

This PR originally aimed to fix #820. What it now does is:

  1. Add benchmarks (results below) for clearAllMocks() and unmockkAll(), to demonstrate which of the two is more performant.
  2. Make the JUnit 5 MockKExtension behave like the JUnit 4 MockKRule, making it call unmockkAll after each unit test function. This PR makes MockKExtension respect the JUnit 5 test instance lifecycle (either per method (default) or per class).
  3. Update the Gradle wrapper to the latest version, because I noticed that the checked in wrapper jar and shell scripts were outdated.
  4. Update the setup-gradle action to v4, which performs Gradle wrapper validation. This is recommendable in open source repos that accept public contributions. E.g. see https://github.com/mockk/mockk/actions/runs/10904085959/job/30259684320?pr=1297#step:4:33.
  5. Update the benchmark dependencies used in this PR.

Benchmark results (on my local machine)

Input:

$ ./gradlew benchmark

Output:

main summary:
Benchmark                                     Mode  Cnt         Score           Error  Units
JmhTest.clearAllMocksAfterNoMockOrStub       thrpt    3  23288167.041 ± 443565082.712  ops/s
JmhTest.clearAllMocksAfterSimpleMock         thrpt    3       227.068 ±       988.568  ops/s
JmhTest.clearAllMocksAfterSimpleMockAndStub  thrpt    3       205.985 ±      1302.086  ops/s
JmhTest.noMockOrStub                         thrpt    3  79739137.777 ±   8111406.644  ops/s
JmhTest.simpleMock                           thrpt    3     62802.584 ±   1707748.668  ops/s
JmhTest.simpleMockAndStub                    thrpt    3     79292.319 ±     64931.152  ops/s
JmhTest.unmockkAllAfterNoMockOrStub          thrpt    3  39488828.944 ±  16722809.880  ops/s
JmhTest.unmockkAllAfterSimpleMock            thrpt    3     36727.737 ±   1064465.208  ops/s
JmhTest.unmockkAllAfterSimpleMockAndStub     thrpt    3     76428.656 ±     19136.524  ops/s

Conclusion

Contrary to what I originally claimed in #820, unmockkAll performs much faster (more operations per second) in the benchmarks in this PR, except when there's nothing to unmock (then clearAllMocks also is fast). This means that it's probably a good idea to unmock all after each unit test. This is what MockKRule does if applied as a JUnit 4 @Rule. However, before this PR the JUnit 5 MockKExtension doesn't call unmockkAll after each test. This PR fixes that.

In this way, #820 can now also be closed (after accepting these changes), since it is outdated / incorrect.

By running Gradle task `wrapper --gradle-version 8.9`
By running Gradle task `wrapper --gradle-version 8.10.1`
This also happens after each JUnit 4 test
@erikhuizinga erikhuizinga force-pushed the erikhuizinga-bench-junit-api branch from ec167dd to 92d1d0d Compare September 17, 2024 13:20
Note that gradle/actions/setup-gradle@v4 automatically does Gradle
Wrapper validation, which is recommendable for open source repositories
that accept contributions
@erikhuizinga erikhuizinga force-pushed the erikhuizinga-bench-junit-api branch from a87c418 to 7314083 Compare September 17, 2024 13:28
@erikhuizinga erikhuizinga marked this pull request as ready for review September 17, 2024 14:48
@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.9-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.1-bin.zip
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -4,11 +4,11 @@ import kotlinx.benchmark.gradle.benchmark

plugins {
buildsrc.convention.`kotlin-jvm`
id("org.jetbrains.kotlinx.benchmark") version "0.4.5"
id("org.jetbrains.kotlinx.benchmark") version "0.4.12"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -26,7 +26,7 @@ benchmark {
targets {
register("main") {
this as JvmBenchmarkTarget
jmhVersion = "1.35"
jmhVersion = "1.37"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Raibaz
Copy link
Collaborator

Raibaz commented Sep 19, 2024

This is awesome, thanks a lot!

@Raibaz Raibaz merged commit edf9f95 into mockk:master Sep 19, 2024
19 checks passed
@erikhuizinga erikhuizinga deleted the erikhuizinga-bench-junit-api branch September 20, 2024 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Do not unmock all mocks after every test function, but after the entire test class only. Clear mocks after every test function.
2 participants