DId you remove your discussion? Your link went to nothing.

@MemphisMeng see https://github.com/orgs/community/discussions/17245#discussioncomment-3421929

The forum was moved from github.community to here in summer, and most threads without a selected answer weren't migrated. That's probably why the link is broken. 😕

Sigh. Good work github, I documented a workaround and now that's gone.

Agreed.

The community engagement platform churn is nearly akin to Google’s messaging apps churn.

Thanks @mikz, this seems to be the workaround.
Thankfully we went from a bunch of files with just new line separators, so we were also able the parse all the files at once with

matrix="$(cat */matrix | jq -ncR '[inputs]')"

I wondered if a possible solution might be to be able to call another workflow from within a step,
this means you could also get around the problem by chaining multiple workflows together.
This might just be another workaround and not work for someone else.

Using artifacts seems like a possible workaround but there can be some edge cases where artifact get corrupted when multiple jobs write to same artifact at same time. There is a warning in upload action docs:

Warning: Be careful when uploading to the same artifact via multiple jobs as artifacts may become corrupted. When uploading a file with an identical name and path in multiple jobs, uploads may fail with 503 errors due to conflicting uploads happening at the same time. Ensure uploads to identical locations to not interfere with each other.

Probably very rare to happen but it may so watch for this

I'm implementing this workaround, if you want to prevent artifact corruption during concurrent executions of the same workflow. You can use the following:

concurrency:
  group: ${{ github.event.pull_request.number }}
  cancel-in-progress: true

Basically this will cancel previous executions and continue with the new one

@xriu There are two problems here:

1. concurrent execution of same workflow that updates artifact, (e.g. someone push twice to same PR almost at same time)
2. concurrent execution of matrix jobs that update same artifact within same workflow run

For first case your workaround will probably work. For second one it may get artifacts corrupted because matrix jobs will concurrently write results to same artifact. In second case you could set concurrency on job level but this would defeat whole purpose of matrix which is concurrent execution.

Ugly but probably working solution would be to use different artifacts for each matrix job. For example:

1. you have matrix like this:

   jobs:
     example_matrix:
       strategy:
         matrix:
           os: [ubuntu-22.04, ubuntu-20.04]
           version: [10, 12, 14]
   

2. each matrix job uploads artifact with special name like matrix-outputs-${{ github.run_id }}-${{ github.run_attempt }}-HASH_OF_MATRIX_OBJECT containing json file with outputs like:

   {
     "${{ matrix.os }}": {
       "${{ matrix.version }}": {
         "outputs": {....}
       }
     }
   }
   

3. job that need matrix output will:
   1. ask for artifacts using github API
   2. download
   3. unpack
   4. deep merge json data from unpacked artifacts into one object e.g. jq -s 'reduce .[] as $item ({}; . * $item)' *.json > outputs-combined.json

As you can see pretty ugly workaround but should be resistant to artifact corruption.

Note that actions/upload-artifact@v4.0.0 now disallows multiple uploads to the same artifact, so if you are using this workaround, you must ensure that each instance of the matrix job uses an unique artifact name. If you try to use the same artifact names for all instances, you will get an error like this:

Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

In my testing ${{strategy.job-index}} seems to be enough to provide distinction between multiple instances of the same matrix job.

Changing the job to use multiple artifacts may also cause another issue — actions/download-artifact still does not support a glob or any other filter for the artifact name (it can download either a single artifact specified by the exact name, or all artifacts for the workflow run), so it may download too much if the workflow also uses artifacts for other purposes (like passing the intermediate build results between jobs).

There is also a new limit on the number of artifacts:

Each job in a workflow run now has a limit of 10 artifacts.

However, looks like each instance of a matrix job is treated as a distinct job when this limit is applied (at least I tried a test with 25 jobs in the matrix, and it passed).

I've tested this (it was released in the v2.303.0 release and at least what in that issue is described we couldn't make it work (I've added a comment in the PR to see if I can get some response). Anyone has make it work? 🤔

It notes in the PR that:

Please note that: Server side changes are still in-progess

I guess there's changes in GitHub itself required to fully support the scenario.

yeap, make sense I guess, I don't really understand what they refer with that side note but I guess is it not finished then 🤷
Thanks for the comment 🙇

although it has been already asked in the pr comments but .. are there any updates?

Some time ago I create a PR to fix docs because those are quite misleading :/ Better to test all claims yourself before fully believing the docs. Hope my PR makes it more clear how this is handled actually.

This was my first time with GitHub Actions and it was a client's requirement but my god I'm never doing this again and will be actively advising any client against GA for anything beyond very simple pipelines.

I'm using GA for more than 3 years. It mostly works even with more complex workflows (workarounds sometimes needed) but as you noticed there are some edge cases or bad design decision made in the past that still need to be fixed. My impression however is that GH is more focused on enterprise features and security than normal day to day users problems. If you are sure about some other tools with similar features as GA, consider trying them out.

There are tons of essential features missing in github actions and they haven't been fixed in years. Github doesn't respond to these community discussions. We're shouting into the void here. You are better off using something else tbh.

If you are interested I have two more docs pull requests waiting for review. Here and here. Those are not related to this specific problem but may be of some help to people even if not merged.

upload/download artifacts is not a workaround for stricter security requirements

Can you help me understand this more?

Artifacts are the workaround. Unless you can hardcode your outputs uniquely for each matrix instance, you will need to generate uniquely-named artifacts in each matrix job, and then download them all in the downstream job that needs them.

Well this is one workaround to always use different output names in each matrix job, but this is not dynamic at all. You still need to remember to update output names each time you add or remove items from matrix. Also next jobs that use those outputs need to be changed. If it was possible to access those for example like this:

env:
  outpts_json: ${{ toJSON(needs.matrix_job.matrix.outputs.*.output_param) }}

it would be way easier to work with.

Your work around does not account for dynamically generated axes though.

A proper solution is still needed here.

@OscarVanL you are absolutely right. This could lead to completely wrong deployments. Especially in multi tenancy backend apps, where you perform configuration and migrations on several databases. It is sad to see that since 3 years now, nobody from the github side feels responsible or is answering. We are in discussion with ourselves :(

there are valid needs for this. for instance, my current usecase (which lead me here) is to spawn one runner per arch to build a docker image (i'm not using buildx for many reasons). each job runs independently and the image tag should be produced by the worker after publishing the image ; then another action should build the manifest based on those images. you can get around by pre-processing the image tags in an action before the matrix and it's ok like that, but i'd rather have this action doing and producing the artifact it should be producing.

why all the downvotes?

@mcascone The core of the issue is that we need the outputs from the matrix. The if condition you proposed will allow other steps in the workflow to run, but most of output(s) from the matrix will still be missing.

For example (...or at least this is my observed behavior), let's say I had a workflow that had a matrix to compile an app for Window, Mac and Linux, and that the output of the matrix was a SHA of the compiled binary. When the matrix runs, all three apps will be compiled, but if the Linux build completes first, the rest of the workflow will only receive the SHA for the Linux build, and the Windows and Mac SHAs will be omitted entirely.

Does anyone have a concrete scenario where artifacts won't work? Or is it a preference for outputs?

At my previous job (shameless plug that I'm looking for a gig), we had a build that generated short(ish) lived credentials (8hrs). Sticking those in an artifact was a no-go. The short term solution was to just use a loop instead of a matrix, but processing sizes were only going to grow, so a matrix of loops was going to be a requirement to keep build times reasonable (and developers happy).

preemptive edit: yes, keeping the creds out of the build log took some tinkering, although I don't recall what solution that was.

Thanks @dev-dull!. Technically outputs shouldn't be used for secrets and we even make it hard by dropping outputs that contain secrets. Nonetheless it's an interesting scenario and maybe we should support something like secret-outputs 🤔

Technically outputs shouldn't be used for secrets...

I'm suddenly glad those pipelines aren't my problem any longer.

FYI, I'm going to connect with you on LinkedIn to chat about this topic for reasons that'll make sense.

For me it is matter of convenience and consistency with normal jobs where you can easily access job outputs. With matrix it just not working even tho you can define and use outputs there. I can think of scenario where there is matrix that multiple jobs and then next job that depends on matrix and is run always after. This job could generate summary of all matrix job based on data from those outputs. Outputs can be easily accessed without additional steps and can be used directly or converted to json string then used by some scripts. Working with artifacts, even with merge option, is just inconvenient in this case. Allowing to use outputs with matrix job just generates bugs if people are not aware that they do not work as expected.