Skip to content

feat: Matrix execution for testing parameter combinations #990

@yottahmd

Description

@yottahmd

Add matrix execution to run steps with all combinations of multiple parameter sets (similar to GitHub Actions matrix). Related: #879

Motivation

When testing or deploying across multiple environments, versions, or configurations, users need to run the same workflow with different combinations:

# Currently requires manual duplication
steps:
  - name: test-ubuntu-go118
    command: docker run ubuntu:20.04 go1.18 test
  - name: test-ubuntu-go119
    command: docker run ubuntu:20.04 go1.19 test
  - name: test-alpine-go118
    command: docker run alpine:3.16 go1.18 test
  - name: test-alpine-go119
    command: docker run alpine:3.16 go1.19 test

Proposed Syntax

Basic matrix

steps:
  - name: test-all
    run: docker run ${os} go${version} test
    matrix:
      os: [ubuntu:20.04, alpine:3.16]
      version: [1.18, 1.19, 1.20]
    # Creates 2×3 = 6 executions

Include/Exclude

matrix:
  os: [ubuntu, macos, windows]
  version: [1.18, 1.19, 1.20]
  exclude:
    - os: windows
      version: 1.18  # Skip Windows + Go 1.18
  include:
    - os: ubuntu
      version: 1.21-rc
      experimental: true  # Add Ubuntu + Go 1.21 RC

Real-world Examples

Multi-cloud deployment

steps:
  - name: deploy
    run: terraform apply
    matrix:
      provider: [aws, gcp, azure]
      environment: [staging, prod]
      region: [us, eu]
    exclude:
      - environment: staging
        region: eu  # Only deploy staging to US
    env:
      - CLOUD: ${provider}
      - ENV: ${environment}
      - REGION: ${region}

Cross-platform testing

steps:
  - name: build-and-test
    run: workflows/test
    matrix:
      os: [ubuntu-latest, macos-latest, windows-latest]
      node: [16, 18, 20]
      arch: [x64, arm64]
    exclude:
      - os: windows-latest
        arch: arm64  # No Windows ARM64 runners
    maxConcurrent: 6

Key Differences from Parallel Execution

Parallel Matrix
Process list of items Test all combinations
parallel: ${ITEMS} matrix: {a: [...], b: [...]}
One dimension Multiple dimensions
${ITEM} variable ${a}, ${b} variables

Configuration Options

matrix:
  values:
    os: [linux, macos]
    arch: [amd64, arm64]
  maxConcurrent: 4   # Default: 10
  failFast: true     # Stop all if one fails

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions