:kotest-assertions:kotest-assertions-core:jvmTest

• com.sksamuel.kotest.assertions.timing.ContinuallyTest[jvm] (macos-latest)

  continually should[jvm]
      Test fail tests start off as passing then fail within the period[jvm] FAILED (3s)
  
      java.lang.AssertionError: Expected exception kotlin.Throwable but no exception was thrown.
  

  In progress: Use test TimeSource in continually tests #4109

• com.sksamuel.kotest.matchers.concurrent.suspension.ConcurrentTest[jvm] (windows-latest)

  Test should fail when given lambda pass before the given time range[jvm] FAILED (1.2s)
  
  java.lang.AssertionError: Expected exception java.lang.AssertionError but no exception was thrown.
  

  In progress: commonize shouldCompleteWithin/shouldCompleteBetween/shouldTimeout #4115

• io.kotest.assertions.nondeterministic.UntilTest[jvm] (macos-latest)

  Test until with config[jvm] FAILED (2.2s)
  
  java.lang.AssertionError: Block failed after 2s; attempted 36 time(s)
  The first error was caused by: expected:<true> but was:<false>
  

  In progress: Stabilize tests on slow runners #4132

• io.kotest.assertions.nondeterministic.EventuallyTest[jvm]

    Test handle java assertion errors[jvm] FAILED
  
    java.lang.AssertionError: Block failed after 157ms; attempted 1 time(s)
    The first error was caused by: boom
    java.lang.AssertionError: boom
  

  In progress: Use test TimeSource in eventually tests #4114

:kotest-assertions:kotest-assertions-shared:jvmTest

• com.sksamuel.kotest.assertions.until.UntilTest[jvm] (macos-latest)

  Test until with boolean predicate and interval[jvm] FAILED (2s)
  
  java.lang.AssertionError: Until block failed after 2.049s; attempted 36 time(s); FixedInterval(duration=10ms) delay between attempts
  Test until with patience config[jvm] FAILED (2s)
  
  java.lang.AssertionError: Until block failed after 2.016s; attempted 35 time(s); FixedInterval(duration=10ms) delay between attempts
  Test until with predicate and interval[jvm] FAILED
  
  java.lang.AssertionError: 115ms should be < 100ms
  

  In progress: Stabilize tests on slow runners #4132

• io.kotest.assertions.CluesTests[jvm]

  Test concurrent withClue invocations should be isolated from each other[jvm] FAILED (1.4s)
  
  java.lang.AssertionError: 46 should be >= 50
  

  In progress: Stabilize tests on slow runners #4132

• com.sksamuel.kotest.assertions.async.TimeoutTest[jvm]

  Test shouldTimeout should fail if a coroutine does not timeout[jvm] FAILED
  java.lang.AssertionError: Expected exception java.lang.AssertionError but no exception was thrown.
  

  In progress: Stabilize tests on slow runners #4132

:kotest-framework:kotest-framework-concurrency:jvmTest

• io.kotest.framework.concurrency.UntilSpec[jvm] (macos-latest, windows-latest)

  Test until with boolean predicate and interval[jvm] FAILED (2.1s)
  
  java.lang.AssertionError: Eventually block failed after 2000ms; attempted 40 time(s); FixedInterval(duration=10) delay between attempts
  The provided predicate failed 40 times
  
  Test until with predicate and interval[jvm] FAILED
  
  java.lang.AssertionError: 174 should be < 100
  

  In progress: Stabilize tests on slow runners #4132

• io.kotest.framework.concurrency.EventuallySpec[jvm]

  Test eventually handles java assertion errors[jvm] FAILED
  
  java.lang.AssertionError: Eventually block failed after 100ms; attempted 1 time(s); FixedInterval(duration=25) delay between attempts
  The first error was caused by: boom
  java.lang.AssertionError: boom
  

  In progress: Fix flaky tests ContinuallySpec, EventuallySpec #4142

  • also: Use available virtual time automatically, harmonize time measurement #4149

• io.kotest.framework.concurrency.ContinuallySpec[jvm]

  Test continually fails tests that start off as passing then fail before the time is up[jvm] FAILED (1.1s)
  400.milliseconds
  400.milliseconds  java.lang.AssertionError: Throwable should match regex: Test failed after d+(.d*)?ms; expected to pass for 12h; attempted 10 times
  400.milliseconds  Underlying failure was: expected:<true> but was:<false>
  400.milliseconds  Actual was:
  400.milliseconds  "Test failed after 1.101072583s; expected to pass for 12h; attempted 10 times
  400.milliseconds  Underlying failure was: expected:<true> but was:<false>"
  

  In progress: Fix flaky tests ContinuallySpec, EventuallySpec #4142

:kotest-framework:kotest-framework-engine:jvmTest

• com.sksamuel.kotest.engine.test.timeout.EngineTimeoutTest[jvm] (windows-latest)

  Test timeouts should be applied by the engine to suspend delays[jvm] FAILED (3.9s)
  
  io.kotest.assertions.AssertionFailedError: expected:<"Test 'a' did not complete within 1ms"> but was:<"Timed out waiting for 1 ms">
  

  In progress: Stabilize GlobalTimeoutTest and EngineTimeoutTest #4124

  • superseded by: Use available virtual time automatically, harmonize time measurement #4149

• com.sksamuel.kotest.engine.test.timeout.GlobalTimeoutTest[jvm]

  Test global timeouts should apply if no other timeout is set[jvm] FAILED (2.7s)
  
  java.lang.AssertionError: Expected true but actual was null
  
  Caused by: java.lang.AssertionError: Expected true but actual was null
  

  https://github.com/kotest/kotest/actions/runs/9555635058/job/26339325763?pr=4114#step:7:3157

  In progress: Stabilize GlobalTimeoutTest and EngineTimeoutTest #4124

  • superseded by: Use available virtual time automatically, harmonize time measurement #4149

• com.sksamuel.kotest.engine.test.timeout.MultipleTestTimeoutTest[jvm] (macos-latest)

  Test Test 5[jvm] FAILED
  
  kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 40 ms
  
  Test Test 7[jvm] FAILED
  
  kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 40 ms
  

  In progress: Stabilize tests on slow runners #4132

• com.sksamuel.kotest.engine.test.timeout.SpecInvocationTimeoutShouldNotApplyToContainersTest[jvm] (macos-latest, windows-latest)

  invocation timeouts at the spec level should not be applied to containers[jvm]
  
      Test suspending inner test[jvm] FAILED
  
      io.kotest.engine.test.interceptors.TestTimeoutException: Test 'suspending inner test' did not complete within 50ms
  
      Test blocking inner test[jvm] FAILED
  
      io.kotest.engine.test.interceptors.TestTimeoutException: Test 'blocking inner test' did not complete within 50ms
  

  In progress: Stabilize tests on slow runners #4132

  • also: Use available virtual time automatically, harmonize time measurement #4149

• io.kotest.engine.concurrency.TimeoutTest[jvm]

  Test detection with blocking job[jvm] FAILED
  java.lang.AssertionError: Expected exception kotlinx.coroutines.TimeoutCancellationException but no exception was thrown.
  

  In progress: Stabilize tests on slow runners #4132

  • also: Use available virtual time automatically, harmonize time measurement #4149

:kotest-property:jvmTest

• com.sksamuel.kotest.property.PropTestConfigConstraintsTest[jvm] (windows-latest)

  Test PropTestConfig constraints should support durations[jvm] FAILED
  
  java.lang.AssertionError: 420 should be < 400
  

  In progress: Stabilize tests on slow runners #4132

:kotest-runner:kotest-runner-junit5:jvmTest

• com.sksamuel.kotest.runner.junit5.PerformanceTest[jvm]

  Test performance of multiple tests[jvm] FAILED (20.8s)
  
  kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 20000 ms
  

  In progress: Stabilize tests on slow runners #4132

:kotest-property:jvmTest Test Shrinks all components to minimum value[jvm] FAILED

• https://github.com/kotest/kotest/actions/runs/9864899341/job/27240897776?pr=4141#step:7:976

> Task :kotest-property:jvmTest
com.***.kotest.property.arbitrary.BindShrinkTest[jvm]

  Test Shrinks all components to minimum value[jvm] FAILED

  java.lang.AssertionError: Test failed after 0s; attempted 5 times; underlying cause was "Property test failed for inputs
  
  0) MaximumComponents(a=1000, b=1000, c=1000, d=1000, e=1000, f=1000, g=1000, h=1000, i=1000, j=1000, k=1000, l=1000, m=1000, n=1000)
  
  Caused by java.lang.AssertionError: 1000 should be < 100 at
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invokeSuspend(BindShrinkTest.kt:54)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	io.kotest.property.internal.ProptestKt$proptest$3$2.invokeSuspend(proptest.kt:49)
  
  " should include substring "Shrink result (after 45 shrinks) => MaximumComponents(a=0, b=0, c=0, d=0, e=0, f=0, g=0, h=0, i=0, j=0, k=0, l=0, m=100, n=0)"

  Caused by: java.lang.AssertionError: Test failed after 0s; attempted 5 times; underlying cause was "Property test failed for inputs
  
  0) MaximumComponents(a=1000, b=1000, c=1000, d=1000, e=1000, f=1000, g=1000, h=1000, i=1000, j=1000, k=1000, l=1000, m=1000, n=1000)
  
  Caused by java.lang.AssertionError: 1000 should be < 100 at
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invokeSuspend(BindShrinkTest.kt:54)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	io.kotest.property.internal.ProptestKt$proptest$3$2.invokeSuspend(proptest.kt:49)
  
  " should include substring "Shrink result (after 45 shrinks) => MaximumComponents(a=0, b=0, c=0, d=0, e=0, f=0, g=0, h=0, i=0, j=0, k=0, l=0, m=100, n=0)"

  Caused by: java.lang.AssertionError: "Property test failed for inputs
  
  0) MaximumComponents(a=[1000](https://github.com/kotest/kotest/actions/runs/9864899341/job/27240897776?pr=4141#step:7:1001), b=1000, c=1000, d=1000, e=1000, f=1000, g=1000, h=1000, i=1000, j=1000, k=1000, l=1000, m=1000, n=1000)
  
  Caused by java.lang.AssertionError: 1000 should be < 100 at
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invokeSuspend(BindShrinkTest.kt:54)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	com.***.kotest.property.arbitrary.BindShrinkTest$1$2$2$stdout$1$1$1.invoke(BindShrinkTest.kt)
  	io.kotest.property.internal.ProptestKt$proptest$3$2.invokeSuspend(proptest.kt:49)
  
  " should include substring "Shrink result (after 45 shrinks) => MaximumComponents(a=0, b=0, c=0, d=0, e=0, f=0, g=0, h=0, i=0, j=0, k=0, l=0, m=100, n=0)"

In progress: #4167

DiscoveryTestWithoutSelectors - OutOfMemoryError: Java heap space

https://github.com/kotest/kotest/actions/runs/9998565996/job/27637892318?pr=4156#step:7:3384

https://scans.gradle.com/s/thygkik33v3r2

> Task :kotest-runner:kotest-runner-junit5:jvmTest
com.***.kotest.runner.junit5.DiscoveryTestWithoutSelectors[jvm]
  Test kotest should recognize suffixed class name pattern filters[jvm] FAILED (1.8s)
  io.kotest.engine.config.ConfigurationException: io.kotest.engine.config.ConfigurationException: io.github.classgraph.ClassGraphException: Uncaught exception during scan
  Caused by: io.kotest.engine.config.ConfigurationException: io.github.classgraph.ClassGraphException: Uncaught exception during scan
  Caused by: io.github.classgraph.ClassGraphException: Uncaught exception during scan
  Caused by: java.lang.OutOfMemoryError: Java heap space

FAILURE: Executed 106 tests in 4m 40s (1 failed, 1 skipped)

com.sksamuel.kotest.engine.spec.examples.DescribeSpecExample[jvm]

https://github.com/kotest/kotest/actions/runs/10133804332/job/28019385046?pr=4218#step:8:2652

> Task :kotest-framework:kotest-framework-engine:jvmMaxJdkTest
com.sksamuel.kotest.engine.spec.examples.DescribeSpecExample[jvm]
  a describe with config[jvm]
    with some describe[jvm]
      with some context[jvm]
        Test executionError[jvm] FAILED
        kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 234 ms
        Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 234 ms

FAILURE: Executed 1921 tests in 13m 16s (1 failed)
1921 tests completed, 1 failed

• Use virtual time to test DescribeSpecExample #4232

com.sksamuel.kotest.engine.test.TestCaseSourceRefTest[jvm]

https://github.com/kotest/kotest/actions/runs/10138373663/job/28033374831?pr=4221#step:8:2174

https://scans.gradle.com/s/c3ygboydkzmis

com.sksamuel.kotest.engine.test.TestCaseSourceRefTest[jvm]
  Test source ref should be performant[jvm] FAILED
  java.lang.AssertionError: 2m 3.600324900s should be < 2m
  Caused by: java.lang.AssertionError: 2m 3.600324900s should be < 2m

• Up for grabs