success() returns false if dependent jobs are skipped #45058

suzuki-shunsuke · 2023-01-22T10:50:36Z

suzuki-shunsuke
Jan 22, 2023

Select Topic Area

Bug

Body

Please see the following workflow definition and result.

---
name: Test
on: pull_request
jobs:
  foo:
    runs-on: ubuntu-latest
    steps:
      - run: echo hello
  bar:
    runs-on: ubuntu-latest
    if: "false"
    steps:
      - run: echo bar
  zoo:
    runs-on: ubuntu-latest
    if: "success()"
    needs:
      - foo
      - bar
    steps:
      - run: echo hello

The job zoo depended on foo and bar and foo succeeded and bar was skipped, then zoo was skipped.

But I think zoo should have been run because none of the dependent jobs failed or was canceled.

https://docs.github.com/en/actions/learn-github-actions/expressions#success

Returns true when none of the previous steps have failed or been canceled.

https://docs.github.com/en/actions/learn-github-actions/contexts#needs-context

needs.<job_id>.result | string | The result of a job that the current job depends on. Possible values are success, failure, cancelled, or skipped.

skipped is different from cancelled.

I changed if from success() to ! failure(), then it worked as expected.

suzuki-shunsuke · 2023-01-28T14:39:19Z

suzuki-shunsuke
Jan 28, 2023
Author

I created a repository to reproduce this issue.

https://github.com/suzuki-shunsuke/bug-reproduction-github-actions-success

0 replies

suzuki-shunsuke · 2023-01-30T09:17:50Z

suzuki-shunsuke
Jan 30, 2023
Author

I created a GitHub Support ticket and got a reply today.

Thank you for writing to GitHub Support.

It appears the documented behavior is only being implemented if the success() function is used at the step level.
I have opened an issue with Engineering to seek further clarification. I will let you know as soon as I have an update from them.

I'm waiting for further reply from support.

2 replies

ChristopherHX Jan 30, 2023

Let's see if the GitHub Engineering alters the behavior of success() to include skipped jobs after more than 3 years of GitHub Actions formerly based on Azure Pipelines (which has this (unexpected) behavior documented https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#succeeded).

There is an very old issue about this behavior (from 2020): actions/runner#491 (comment)

You should use !failure() && !cancelled() as a workaround, unless you want that pressing cancel does nothing.

suzuki-shunsuke Jan 30, 2023
Author

Oh, thank you for sharing. I didn't know the issue.

suzuki-shunsuke · 2023-02-06T10:35:54Z

suzuki-shunsuke
Feb 6, 2023
Author

#45058 (comment)

I'm waiting for further reply from support.

I got a reply from support.

Hi Shunsuke,

Apologies for taking a while to get back to you on this.

The GitHub Actions team have placed this issue on the backlog. This is something that they are looking to fix but I do not have a timeline for when that will be done, sorry. I will open a follow up ticket to let you know as soon as the fix is deployed.

For now, as you noted, using the ! failure() workaround would achieve the equivalent behavior.

Let me know if you have any further questions.

0 replies

lyuhau · 2023-04-03T15:54:20Z

lyuhau
Apr 3, 2023

I'm having similar issues. A job with if-condition always() && !contains(needs.*.result, 'skipped') gets skipped even though that expression evaluates to true. always() && !contains(needs.*.result, 'failure') on the other hand works as expected. Here's the sample workflow file that I made to test:

name: ci

on:
  pull_request: null
  workflow_dispatch: null

jobs:
  runs:
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  skips:
    if: false
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  runs-with-skipped-upstream:
    needs: skips
    if: always()
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  always-and-no-skips:
    needs: [runs, runs-with-skipped-upstream]
    if: always() && !contains(needs.*.result, 'skipped')
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  always-and-no-failures:
    needs: [runs, runs-with-skipped-upstream]
    if: always() && !contains(needs.*.result, 'failure')
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  always-and-no-skips-or-failures:
    needs: [runs, runs-with-skipped-upstream]
    if: always() && !contains(needs.*.result, 'skipped') && !contains(needs.*.result, 'failure')
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.job }}

  debug-print:
    needs: [runs, runs-with-skipped-upstream]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - run: |
          echo "toJSON(needs) = ${{ toJSON(needs) }}"
          echo "toJSON(needs.*.result) = ${{ toJSON(needs.*.result) }}"
          echo "!contains(needs.*.result, 'skipped') = ${{ !contains(needs.*.result, 'skipped') }}"
          echo "!contains(needs.*.result, 'failure') = ${{ !contains(needs.*.result, 'failure') }}"

And the repository: https://github.com/lyuhau/github-actions-needs-if-bug

1 reply

lyuhau Apr 3, 2023

The console of the debug-print job:

amanfredi · 2023-05-12T04:37:15Z

amanfredi
May 12, 2023

Please change this bonkers behavior

0 replies

cundejo · 2023-05-18T15:43:23Z

cundejo
May 18, 2023

Please fix this, it doesn't make sense at all. Thank you!

0 replies

LudovicTOURMAN · 2023-11-03T10:43:37Z

LudovicTOURMAN
Nov 3, 2023

Since I struggled with some weird behavior (and not only with the success() function), I wanted to share a result table of the check functions.

flowchart LR;
    A[Job A]-->B1[Job B1];
    A-->B2[Job B2];
    B1-->D[" "];
    B2-->D;
    D-->E["always()"];
    D-->F["success()"];
    D-->G["failure()"];
    D-->H["cancelled()"];

Job A	Job B1 (needs A)	Job B2 (needs A)	always() (needs A + B1 + B2)	success() (needs A + B1 + B2)	failure() (needs A + B1 + B2)	cancelled() (needs A + B1 + B2)
${\color{green}\text{success}}$	${\color{green}\text{success}}$	${\color{green}\text{success}}$	${\color{green}\text{true}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$
${\color{green}\text{success}}$	${\color{grey}\text{skipped}}$	${\color{green}\text{success}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$
${\color{green}\text{success}}$	${\color{red}\text{failure}}$	${\color{green}\text{success}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$
${\color{green}\text{success}}$	${\color{grey}\text{skipped}}$	${\color{orange}\text{cancelled}}$ (concurrency)	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$
${\color{green}\text{success}}$	${\color{red}\text{failure}}$	${\color{orange}\text{cancelled}}$ (concurrency)	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$
${\color{green}\text{success}}$	${\color{grey}\text{skipped}}$	${\color{orange}\text{cancelled}}$ (manual)	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$
${\color{green}\text{success}}$	${\color{red}\text{failure}}$	${\color{orange}\text{cancelled}}$ (manual)	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$
${\color{red}\text{failure}}$	${\color{grey}\text{skipped}}$	${\color{grey}\text{skipped}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$
${\color{grey}\text{skipped}}$	${\color{green}\text{success}}$	${\color{green}\text{success}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$
${\color{grey}\text{skipped}}$	${\color{red}\text{failure}}$	${\color{green}\text{success}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$
${\color{orange}\text{cancelled}}$ (concurrency)	${\color{grey}\text{skipped}}$	${\color{grey}\text{skipped}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$
${\color{orange}\text{cancelled}}$ (manual)	${\color{orange}\text{cancelled}}$	${\color{orange}\text{cancelled}}$	${\color{green}\text{true}}$	${\color{red}\text{false}}$	${\color{red}\text{false}}$	${\color{green}\text{true}}$

⚠️ Notice how different are the results between a workflow cancelled manually and a workflow cancelled by concurrency mechanism (cancel-in-progress: true)

I hope that could help people dig into weird behaviors and make GitHub fixing/improving those weird condition behaviors (non-sense from a developer's pov)

1 reply

EcksDy Nov 5, 2023

Amazing, thank you for sharing!

Felixoid · 2024-02-14T18:05:58Z

Felixoid
Feb 14, 2024

Here's my way to check if all jobs in the needs have succeeded

if: ${{ ! (contains(needs.*.result, 'skipped') || contains(needs.*.result, 'failure')) }}

I must say, it's very ugly. Instead of needs_succeeded(), we need to have such a boilerplate. Connecting to the fact that we don't have anchors actions/runner#1182, it's just a shame.

for the reference Felixoid/actions-experiments#9

0 replies

success() returns false if dependent jobs are skipped #45058

Uh oh!

Uh oh!

Select Topic Area

Body

Replies: 8 comments · 4 replies

Uh oh!

suzuki-shunsuke Jan 28, 2023 Author

Uh oh!

Uh oh!

suzuki-shunsuke Jan 30, 2023 Author

Uh oh!

Uh oh!

suzuki-shunsuke Jan 30, 2023 Author

Uh oh!

suzuki-shunsuke Feb 6, 2023 Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 8 comments 4 replies

suzuki-shunsuke
Jan 28, 2023
Author

suzuki-shunsuke
Jan 30, 2023
Author

suzuki-shunsuke Jan 30, 2023
Author

suzuki-shunsuke
Feb 6, 2023
Author