Update GitHub release workflows #4076

aSemy · 2024-06-08T14:18:16Z

Publish all artifacts to Maven Central from a single machine (prevents split-repos due to parallel publishing to Maven Central)
Update run-gradle.yml so it can access secrets
Pass secrets into run-gradle.yml from master.yml by setting secrets: inherit
remove unnecessary fetch-depth: 0 from all workflows (a depth of 0 fetches everything - the default only fetches the latest commit)
Tidy up release_multiplatform_plugin_gradle.yml

Of course, because GitHub Workflows are impossible to test locally, I can't verify if these workflows will work first time! Further work might be necessary.

Hopefully fixes #4067...

- Publish all artifacts to Maven Central from a single machine (prevents split-repos due to parallel publishing to Maven Central) - Update `run-gradle.yml` so it can access secrets - Pass secrets into `run-gradle.yml` from `master.yml` by setting `secrets: inherit` - remove unnecessary `fetch-depth: 0` from all workflows (a depth of 0 fetches everything - the default only fetches the latest commit) - Tidy up `release_multiplatform_plugin_gradle.yml`

.github/workflows/master.yml

Co-authored-by: Emil Kantis <e.kantis@gmail.com>

Kantis · 2024-06-08T21:24:28Z

.github/workflows/master.yml

+            include:
+               -  os: macos-latest
+               -  os: ubuntu-latest
+               -  os: windows-latest


I think this will make us run jvmTest on all platforms. Currently we have one or two flaky tests that sometimes fails builds.. Will become important to fix that

It'd be better to fix the flaky tests, or maybe enable retries for them?

We could also add --exclude-task jvmTest to 2 of the OSes, so Gradle will skip them, but of course fixing the flakey tests would be better!

I'm ok with merging as-is. Let's try to reduce flakiness of master if it becomes an issue

aSemy · 2024-06-10T10:21:47Z

Hmmm I really don't understand why PR-Test / test_linux (jvmTest) / run-tests is failing so regularly. The only change to the PR-Test workflow is to change the checkout depth.

I downloaded the test report, and it shows an exception which doesn't make sense to me.

org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 7.
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:64)
	at java.base@17.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@17.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base@17.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@17.0.11/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: org.junit.platform.commons.JUnitException: TestEngine with ID 'kotest' failed to discover tests
	at app//org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)
	at app//org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:134)
	at app//org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:108)
	at app//org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:80)
	at app//org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:110)
	at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at app//org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:119)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:94)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:89)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
	... 18 more
Caused by: java.lang.NoSuchMethodError: 'void io.kotest.mpp.Logger.logsimple(kotlin.jvm.functions.Function0)'
	at io.kotest.runner.junit.platform.KotestJunitPlatformTestEngine.discover(KotestJunitPlatformTestEngine.kt:101)
	at io.kotest.runner.junit.platform.KotestJunitPlatformTestEngine.discover(KotestJunitPlatformTestEngine.kt:35)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152)
	... 28 more

Why can't JUnit find the io.kotest.mpp.Logger.logsimple function in this PR?

kotest/kotest-common/src/commonMain/kotlin/io/kotest/mpp/logger.kt

Lines 31 to 33 in 1713eb7

    
           @OverloadResolutionByLambdaReturnType 
        
           @JvmName("logsimple") 
        
           fun log(f: () -> String): Unit = log { Pair(null, f()) }

Kantis · 2024-06-10T21:23:49Z

@aSemy honestly no clue why that happens. ☹️ Suspected some cache issue or similar, but I cleaned the GH action cache between one of the runs and nothing changed

aSemy · 2024-06-11T08:18:26Z

Maybe it's something to do with the jvmOnly flag?

kotest/buildSrc/src/main/kotlin/Ci.kt

Line 28 in 19ce4d3

const val JVM_ONLY = "jvmOnly"

jvmOnly doesn't seem to be set anywhere though.
Maybe JUnit gets confused by the @JvmName annotation?

There's some atypical Gradle config in the Kotest Gradle Plugin. Maybe that interferes with compilation?

kotest/kotest-framework/kotest-framework-multiplatform-plugin-gradle/build.gradle.kts

Lines 37 to 55 in ca5c340

    
           if (!project.hasProperty(Ci.JVM_ONLY)) { 
        
              // Build these libraries ahead of time so that the test project doesn't try to build them itself (if it tries to build them while we are as well, this can lead to conflicts) 
        
              setOf( 
        
                 projects.kotestAssertions.kotestAssertionsCore, 
        
                 projects.kotestFramework.kotestFrameworkApi, 
        
                 projects.kotestFramework.kotestFrameworkEngine, 
        
              ).map { project -> 
        
                 project.dependencyProject.path 
        
              }.forEach { projectPath -> 
        
                 setOf( 
        
                    "jvmJar", 
        
                    "compileKotlinLinuxX64", 
        
                    "compileKotlinMacosX64", 
        
                    "compileKotlinMacosArm64", 
        
                    "compileKotlinMingwX64", 
        
                 ).forEach { task -> 
        
                    dependsOn("$projectPath:$task") 
        
                 } 
        
              }

Sometimes Maven Local can get 'poisoned' if Maven downloads and caches the JVM variants of KMP artifacts, which means the Gradle metadata will be missing, and so Gradle gets confused. But that should trigger a Gradle dependency resolution error, not this type of error.
Maybe there's a problem with the dependencies not being declared correctly, e.g. a dependency should be declared as api() not implementation().
~~Maybe Kotlin is compiling both logger.kt file and class Logger to LoggerKt.class, and because the file names clash only one gets resolved.~~ No, logger.kt compiles to LoggerKt.class and class Logger compiles to Logger.class
Maybe @OverloadResolutionByLambdaReturnType and @JvmName don't work together?
Maybe it's something to do with the Java version? Kotest build config doesn't use toolchains - maybe it should, using this approach to maintain backwards compatibility with Java 8.

OliverO2 · 2024-06-11T11:55:33Z

It seems that the latest test failure is related to kotest-extensions-http, which is a deprecated Kotest extension with no known public usages. However, when checking out the PR locally, all tests run fine on my Ubuntu machine.

The test log says at line 701:

> Task :kotest-extensions:kotest-extensions-http:jvmTest FAILED


FAILURE: Executed 0 tests in 1.6s

And at line 836:

Execution failed for task ':kotest-extensions:kotest-extensions-http:jvmTest'.
> There were failing tests. See the report at: file:///home/runner/work/kotest/kotest/kotest-extensions/kotest-extensions-http/build/reports/tests/jvmTest/index.html

How could I view the cited HTML report file?

Kantis · 2024-06-11T12:33:06Z

@OliverO2 there should be a zip archive attached to the build summary

OliverO2 · 2024-06-11T12:53:51Z

I scrolled to the top of test_linux (jvmTest) / run-tests, clicked the cogwheel icon, and selected "Download log archive" in the dropdown menu. I got logs_24746409730.zip which contains only .txt files, but not a single HTML report. Also looked at the Gradle build scan, to no avail.

Without it, I can only guess what is going on here. With org.gradle.parallel=true I could suspect that Gradle might be messing up the classpath, trying to distribute classes across multiple processes analogous to what is described #3973 (comment), but that is just another guess.

If we cannot access the HTML reports, should we try running with --info or --debug?

Kantis · 2024-06-11T13:51:07Z

@OliverO2 I think that just downloads the full log of the build. We also upload some gradle reports available here, which shows more details:

OliverO2 · 2024-06-11T16:20:05Z

@Kantis Got it, thanks!

@aSemy

There's some atypical Gradle config in the Kotest Gradle Plugin. Maybe that interferes with compilation?

As I understand it, the chances of getting in trouble with explicit task dependencies are quite high. If possible, we should get rid of them completely. I don't see why we should use task wiring instead of letting Gradle figure out the proper processing order via task input/output dependencies. Maybe that is just some legacy workaround that is no longer needed and now interferes with parallel builds and caching. Unfortunately, this piece of task wiring is part of the rather large commit 8101fdd.

Kantis · 2024-06-11T17:31:02Z

There's some atypical Gradle config in the Kotest Gradle Plugin. Maybe that interferes with compilation?

kotest/kotest-framework/kotest-framework-multiplatform-plugin-gradle/build.gradle.kts

Lines 37 to 55 in ca5c340

if (!project.hasProperty(Ci.JVM_ONLY)) {

// Build these libraries ahead of time so that the test project doesn't try to build them itself (if it tries to build them while we are as well, this can lead to conflicts)

setOf(

projects.kotestAssertions.kotestAssertionsCore,

projects.kotestFramework.kotestFrameworkApi,

projects.kotestFramework.kotestFrameworkEngine,

).map { project ->

project.dependencyProject.path

}.forEach { projectPath ->

setOf(

"jvmJar",

"compileKotlinLinuxX64",

"compileKotlinMacosX64",

"compileKotlinMacosArm64",

"compileKotlinMingwX64",

).forEach { task ->

dependsOn("$projectPath:$task")

}

}

I think the purpose of that was to solve some problems when using the test-project in the multiplatform-gradle-plugin module. Not sure exactly why, but I think we could just try removing that and check if the test-project still works.

OliverO2 · 2024-06-11T17:53:47Z

I think the purpose of that was to solve some problems when using the test-project in the multiplatform-gradle-plugin module. Not sure exactly why, but I think we could just try removing that and check if the test-project still works.

How I'm reading it: The dependsOn invocations happen as part of each test task configuration, but then configure dependencies between unrelated tasks, and do so repeatedly if there are multiple test tasks.

I've set up an own branch, but need to re-authorize IntelliJ for Kotest after handling the reported security issue. Now waiting for repository owner approval. 🙃

aSemy · 2024-06-11T20:46:33Z

I made a plugin that should help with testing the Gradle plugin - https://github.com/adamko-dev/dev-publish-plugin. Kotest Gradle Plugin can depend on the other subprojects, and then each project will publish to a project-local directory. I'll make a new PR, and we can discuss it in there.

UPDATE: Adding dev-publish might take a little bit longer than I thought, I didn't anticipate a project depending on itself and now there's a circular reference.

Kantis · 2024-06-11T21:14:41Z

I still think this is related to caches. The fact that it states "FROM-CACHE" on compileKotlinJvm means it's loading it, not actually performing the step, no?

I took another shot at really cleaning the GH actions cache. I think it got completely emptied this time. Sadly GH actions has some service disruption ATM so I can't confirm if it did the trick.

Task :kotest-extensions:kotest-extensions-http:compileKotlinJvm FROM-CACHE

Build failed after, so this was still not the issue.

Kantis · 2024-06-12T06:01:00Z

Looking in the logs, it seems that for some reason Kotest 5.7.2 is being downloaded. That version of kotest-common does not have logsimple, so it would explain the missing function. But I don't get why it's fetching another kotest version at all?

Resource missing. [HTTP GET: https://dl.google.com/dl/android/maven2/io/kotest/kotest-common/5.7.2/kotest-common-5.7.2.pom]
Downloading https://repo.maven.apache.org/maven2/io/kotest/kotest-common/5.7.2/kotest-common-5.7.2.pom to /home/runner/.gradle/.tmp/gradle_download3973713156886687871bin

aSemy · 2024-06-12T06:06:43Z

Good catch! You can see it in the build scan too https://scans.gradle.com/s/2h744qoyqz6oa/dependencies?dependencies=Kotest-common&expandAll&focusedDependency=WzEwLDYsOTg2LFs5LDEsWzQwXV1d

aSemy · 2024-06-12T06:10:39Z

So the dependency on the Kotest Mockserver extension is probably related. Although I really don't understand why it's only failing on CI but not locally???

kotest/kotest-extensions/kotest-extensions-http/build.gradle.kts

Line 26 in 1f10a54

implementation(libs.kotest.extensions.mockserver)

Kantis · 2024-06-12T06:37:08Z

kotest-extensions-mockserver uses Kotest 4.6.1, so I don't think it's that either? Could it be fudging up the dependency resolution process anyhow?

OliverO2 · 2024-06-12T08:56:37Z

kotest-extensions-mockserver uses Kotest 4.6.1, so I don't think it's that either?

https://github.com/kotest/kotest-extensions-mockserver/blob/6fcb676b60ed6a28a86380914b65778a3602cd54/gradle/libs.versions.toml#L2

kotest = "5.7.2"

Kantis · 2024-06-12T08:58:13Z

D'oh, my local clone was severely out-of-date.. 😅

OliverO2 · 2024-06-12T08:59:34Z

This resolves to project :kotest-framework:kotest-framework-api locally:

\--- io.kotest.extensions:kotest-extensions-mockserver:1.3.0
     +--- io.kotest:kotest-framework-api:5.7.2 -> project :kotest-framework:kotest-framework-api (*)

Why doesn't it resolve that way on CI?

aSemy · 2024-06-12T09:04:00Z

There was also a KGP bug relating to inconsistent dependency resolution in KMP projects - I'm trying to find out more info. Although I don't see why it would be different on CI vs local.

Maybe it's because Kotest-CI builds & runs using Java 17, and I use Java 11 locally, and kotest-mockserver is built with Java 8? That's the only significant difference I can think of.

OliverO2 · 2024-06-12T09:12:01Z

I'm building and running on Java 17 locally if that line in ./gradlew build --info can be trusted:

[KOTLIN] Kotlin compilation 'jdkHome' argument: /home/oliver/.jdks/corretto-17.0.11

(Can Gradle report the JDK version used in an easier way?)

aSemy · 2024-06-12T09:12:27Z

@antohaby found it!

Ok, I reproduced it.
RELEASE_VERSION
is empty but not null in CI.

This set the version to an empty string "" and gradle don't know what to do with it. but it is silent as well. And thus published version wins in resolution.

.github/workflows/PR.yml

aSemy · 2024-06-12T09:25:49Z

I think it would help if we had a single re-usable workflow for running all tests. I don't think this blank-release-version problem was caused by this PR, so it must have snuck in with another PR, but it wasn't caught because PRs and master are tested with different workflows.

Kantis · 2024-06-12T09:29:21Z

@aSemy but wouldn't that PR also have failed by the PR build in that case? I'm not sure how master builds come into play?

kotest-extensions/kotest-extensions-http/build.gradle.kts

Kantis · 2024-06-12T09:39:03Z

.github/workflows/master.yml

+            include:
+               -  os: macos-latest
+               -  os: ubuntu-latest
+               -  os: windows-latest


I'm ok with merging as-is. Let's try to reduce flakiness of master if it becomes an issue

.github/workflows/PR.yml

aSemy · 2024-06-14T10:16:17Z

@aSemy but wouldn't that PR also have failed by the PR build in that case? I'm not sure how master builds come into play?

So, I see it as a problem that when this PR was merged to master the release.yaml actions failed.

I want these problems to be identified and fixed before merging to master. That can only happen if the same workflow that runs the tests is run against PRs and also against master.

aSemy mentioned this pull request Jun 8, 2024

Update release GitHub workflows #4067

Closed

Merge branch 'master' into feat/update-release-workflows

6282b65

Kantis reviewed Jun 8, 2024

View reviewed changes

.github/workflows/master.yml Outdated Show resolved Hide resolved

.github/workflows/master.yml Outdated Show resolved Hide resolved

.github/workflows/master.yml Show resolved Hide resolved

fix step names

25224a6

Co-authored-by: Emil Kantis <e.kantis@gmail.com>

Kantis approved these changes Jun 8, 2024

View reviewed changes

aSemy and others added 4 commits June 9, 2024 20:26

Merge branch 'master' into feat/update-release-workflows

a400efa

Merge branch 'master' into feat/update-release-workflows

24556ed

Merge branch 'master' into feat/update-release-workflows

52b03fa

Merge branch 'master' into feat/update-release-workflows

4fade64

enable Gradle parallel, remove nested gradle.properties

50ee64c

aSemy added 2 commits June 11, 2024 17:54

log info for PR tests

a608901

Merge branch 'master' into feat/update-release-workflows

8a77a2a

Try adding explicit dependency on junit5 runner

446aeff

Merge branch 'refs/heads/master' into feat/update-release-workflows

165d126

make releaseVersion null if blank

bdb033a

aSemy commented Jun 12, 2024

View reviewed changes

.github/workflows/PR.yml Outdated Show resolved Hide resolved

Kantis reviewed Jun 12, 2024

View reviewed changes

kotest-extensions/kotest-extensions-http/build.gradle.kts Outdated Show resolved Hide resolved

Kantis approved these changes Jun 12, 2024

View reviewed changes

aSemy commented Jun 12, 2024

View reviewed changes

.github/workflows/PR.yml Outdated Show resolved Hide resolved

aSemy commented Jun 12, 2024

View reviewed changes

.github/workflows/PR.yml Outdated Show resolved Hide resolved

tidy-up

e1cd1df

Kantis enabled auto-merge June 12, 2024 10:03

Kantis added this pull request to the merge queue Jun 12, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2024

Kantis added this pull request to the merge queue Jun 12, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2024

Kantis added this pull request to the merge queue Jun 12, 2024

Merged via the queue into kotest:master with commit e306738 Jun 12, 2024

aSemy deleted the feat/update-release-workflows branch June 14, 2024 10:16

This was referenced Jun 17, 2024

CI: Flaky tests on slow runners #4113

Open

CI: Run time-sensitive tests separate from parallel Gradle tasks #4120

Open

aSemy added the builds 🐘 PRs / Issues related to the CI/CD pipelines or Gradle builds label Jun 24, 2024

Update GitHub release workflows #4076

Update GitHub release workflows #4076

Uh oh!

Conversation

aSemy commented Jun 8, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kantis Jun 8, 2024

Choose a reason for hiding this comment

Uh oh!

aSemy Jun 9, 2024

Choose a reason for hiding this comment

Uh oh!

Kantis Jun 12, 2024

Choose a reason for hiding this comment

Uh oh!

aSemy commented Jun 10, 2024

Uh oh!

Kantis commented Jun 10, 2024

Uh oh!

aSemy commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

OliverO2 commented Jun 11, 2024

Uh oh!

Kantis commented Jun 11, 2024

Uh oh!

OliverO2 commented Jun 11, 2024

Uh oh!

Kantis commented Jun 11, 2024

Uh oh!

OliverO2 commented Jun 11, 2024

Uh oh!

Kantis commented Jun 11, 2024

Uh oh!

OliverO2 commented Jun 11, 2024

Uh oh!

aSemy commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kantis commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kantis commented Jun 12, 2024

Uh oh!

aSemy commented Jun 12, 2024

Uh oh!

aSemy commented Jun 12, 2024

Uh oh!

Kantis commented Jun 12, 2024

Uh oh!

OliverO2 commented Jun 12, 2024

Uh oh!

Kantis commented Jun 12, 2024

Uh oh!

OliverO2 commented Jun 12, 2024

Uh oh!

aSemy commented Jun 12, 2024

Uh oh!

OliverO2 commented Jun 12, 2024

Uh oh!

aSemy commented Jun 12, 2024

Uh oh!

Uh oh!

aSemy commented Jun 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kantis commented Jun 12, 2024

Uh oh!

Uh oh!

Kantis Jun 12, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aSemy commented Jun 14, 2024

aSemy commented Jun 11, 2024 •

edited

Loading

aSemy commented Jun 11, 2024 •

edited

Loading

Kantis commented Jun 11, 2024 •

edited

Loading

aSemy commented Jun 12, 2024 •

edited

Loading