Skip to content

Random segmentation faults since version: Ubuntu 22.04 - 20230109.1 #6931

@ItsReddi

Description

@ItsReddi

Description

Some of our workflows crashing randomly since 5 days.
More specific since version: 20230109.1

Sometimes the workflow will succeed but more than 90% of the runs are failing.

The workflow is not public and i was not able to reproduce it in a separate repository for now.
The workflow is a matrix container workflow setting up environment and running tests.

We all ready tried to get the core dump to track down the issue but the file is not generated.
Any hint to setup that correctly would help to investigate.

on:
  push:
    branches-ignore:
      - 'master'

jobs:
  codeception-tests:
    # Containers must run in Linux based operating systems
    runs-on: ubuntu-latest
    strategy:
      matrix:
        php-version: ["8.1"]
        codeception-command: ["cmd1", "cmd2", "cmd3"]
    container:
      image: vendor/php:${{ matrix.php-version }}-fpm-alpine-prod-stable
      options: --privileged --ulimit core=-1 --security-opt seccomp=unconfined
    name: Suite ${{ matrix.codeception-command }} on ${{ matrix.php-version }}
    services:
      sql:
        image: mysql:5.7.33

        ports:
          - 3306:3306

        env:
          MYSQL_ROOT_PASSWORD: test
          MYSQL_USER: test
          MYSQL_PASSWORD: test
          MYSQL_DATABASE: app_test
        options: --health-cmd "mysqladmin ping" --health-interval 10s --health-timeout 5s --health-retries 10
      redis:
        image: redis:6.0.8-alpine
        ports:
          - 6379:6379

    steps:
      - name: Check out repository code
        uses: actions/checkout@v3
      - name: Init application
        run: php setup --env=test --overwrite=All

      - name: Run db migrations
        run: php setup migrate-base/up

      # Performing a clean codecept build
      - name: Build application test helpers
        run: php vendor/bin/codecept build

      # Finally running application tests
      - name: Run application tests
        run: |
          ulimit -c unlimited
          echo '/tmp/core_dump' > /proc/sys/kernel/core_pattern
          php vendor/bin/codecept run ${{ matrix.codeception-command }}

      - uses: actions/upload-artifact@v2
        if: ${{ failure() }}
        with:
          name: core-dump
          path: |
            /tmp/core_dump

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Ubuntu 22.04
  • macOS 11
  • macOS 12
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

Current runner version: '2.300.2'
Operating System
Ubuntu
22.04.1
LTS
Runner Image
Image: ubuntu-22.04
Version: 20230109.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230109.1/images/linux/Ubuntu2204-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230109.1
Runner Image Provisioner
2.0.98.1

Is it regression?

20221212.1

Expected behavior

Workflow should pass.

Actual behavior

Fails Randomly
in: "Run application tests"
with:
grafik

Repro steps

Could not be reproduced right now but hints how to get and analyze the workflow - container - coredump would help.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions