Skip to content

Pytest Coverage Comment

Actions
Comments a pull request with the pytest code coverage badge and full report
v1.1.56
Latest
Star (222)

Pytest Coverage Comment

License Version Wakatime

A GitHub Action that adds pytest coverage reports as comments to your pull requests, helping you track and improve test coverage with visual feedback.

🎯 Features

  • 📊 Visual Coverage Reports - Automatically comments on PRs with detailed coverage tables
  • 🏷️ Coverage Badges - Dynamic badges showing coverage percentage with color coding
  • 📈 Test Statistics - Shows passed, failed, skipped tests with execution time
  • 🔗 Direct File Links - Click to view uncovered lines directly in your repository
  • 📁 Multiple Reports - Support for monorepo with multiple coverage reports
  • 🎨 Customizable - Flexible titles, badges, and display options
  • 📝 XML Support - Works with both text and XML coverage formats
  • 🚀 Smart Updates - Updates existing comments instead of creating duplicates

📋 Table of Contents

Click to expand

🚀 Quick Start

Add this action to your workflow:

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
📖 Complete workflow example
name: pytest-coverage-comment
on:
  pull_request:
    branches:
      - '*'

permissions:
  contents: write
  checks: write
  pull-requests: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: 3.11

      - name: Install dependencies
        run: |
          pip install pytest pytest-cov

      - name: Run tests with coverage
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=src tests/ | tee pytest-coverage.txt

      - name: Pytest coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml

⚙️ Configuration

Inputs

📝 Core Inputs
Name Required Default Description
github-token ${{github.token}} GitHub token for API access to create/update comments
pytest-coverage-path ./pytest-coverage.txt Path to pytest text coverage output (from --cov-report=term-missing)
pytest-xml-coverage-path Path to XML coverage report (from --cov-report=xml:coverage.xml)
junitxml-path Path to JUnit XML file for test statistics (passed/failed/skipped)
issue-number Pull request number to comment on (required for workflow_dispatch/workflow_run events)
🎨 Display Options
Name Default Description
title Coverage Report Main title for the coverage comment (useful for monorepo projects)
badge-title Coverage Text shown on the coverage percentage badge
junitxml-title Title for the test summary section from JUnit XML
hide-badge false Hide the coverage percentage badge from the comment
hide-report false Hide the detailed coverage table (show only summary and badge)
hide-comment false Skip creating PR comment entirely (useful for using outputs only)
report-only-changed-files false Show only files changed in the current pull request
xml-skip-covered false Hide files with 100% coverage from XML coverage reports
remove-link-from-badge false Remove hyperlink from coverage badge (badge becomes plain image)
remove-links-to-files false Remove file links from coverage table to reduce comment size
remove-links-to-lines false Remove line number links from coverage table to reduce comment size
🔧 Advanced Options
Name Default Description
create-new-comment false Create new comment on each run instead of updating existing comment
unique-id-for-comment Unique identifier for matrix builds to update separate comments (e.g., ${{ matrix.python-version }})
default-branch main Base branch name for file links in coverage report (e.g., main, master)
coverage-path-prefix Prefix to add to file paths in coverage report links
multiple-files Generate single comment with multiple coverage reports (useful for monorepos)

Outputs

📤 Available Outputs
Name Example Description
coverage 85% Coverage percentage from pytest report
color green Badge color based on coverage percentage (red/orange/yellow/green/brightgreen)
coverageHtml HTML string Full HTML coverage report with clickable links to uncovered lines
summaryReport Markdown string Test summary in markdown format with statistics (tests/skipped/failures/errors/time)
warnings 42 Number of coverage warnings from pytest-cov
tests 109 Total number of tests run (from JUnit XML)
skipped 2 Number of skipped tests (from JUnit XML)
failures 0 Number of failed tests (from JUnit XML)
errors 0 Number of test errors (from JUnit XML)
time 12.5 Test execution time in seconds (from JUnit XML)
notSuccessTestInfo JSON string JSON details of failed, errored, and skipped tests (from JUnit XML)

📚 Usage Examples

Basic Usage

Standard PR Comment
- name: Run tests
  run: |
    pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=src tests/ | tee pytest-coverage.txt

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

Coverage from XML

Using coverage.xml instead of text output
- name: Generate XML coverage
  run: |
    pytest --cov-report=xml:coverage.xml --cov=src tests/

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-xml-coverage-path: ./coverage.xml
    junitxml-path: ./pytest.xml

Monorepo Support

Multiple coverage reports in a single comment
- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    multiple-files: |
      Backend API, ./backend/pytest-coverage.txt, ./backend/pytest.xml
      Frontend SDK, ./frontend/pytest-coverage.txt, ./frontend/pytest.xml
      Data Pipeline, ./pipeline/pytest-coverage.txt, ./pipeline/pytest.xml

This creates a consolidated table showing all coverage reports:

Title Coverage Tests Time
Backend API 85% 156 23.4s
Frontend SDK 92% 89 12.1s
Data Pipeline 78% 234 45.6s

Output: Combined table showing coverage and test results for all packages.

Multiple Files Mode Example

Docker Workflows

Running tests inside Docker containers
- name: Run tests in Docker
  run: |
    docker run -v /tmp:/tmp $IMAGE_TAG \
      python -m pytest \
        --cov-report=term-missing:skip-covered \
        --junitxml=/tmp/pytest.xml \
        --cov=src tests/ | tee /tmp/pytest-coverage.txt

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: /tmp/pytest-coverage.txt
    junitxml-path: /tmp/pytest.xml

Matrix Builds

Separate comments for each matrix combination
strategy:
  matrix:
    python-version: ['3.9', '3.10', '3.11']
    os: [ubuntu-latest, windows-latest]

steps:
  - name: Coverage comment
    uses: MishaKav/pytest-coverage-comment@main
    with:
      pytest-coverage-path: ./pytest-coverage.txt
      junitxml-path: ./pytest.xml
      unique-id-for-comment: ${{ matrix.python-version }}-${{ matrix.os }}
      title: Coverage for Python ${{ matrix.python-version }} on ${{ matrix.os }}

Auto-update README Badge

Keep coverage badge in README always up-to-date

First, add placeholders to your README.md:

<!-- Pytest Coverage Comment:Begin -->
<!-- Pytest Coverage Comment:End -->

Then use this workflow:

name: Update Coverage Badge
on:
  push:
    branches: [main]

permissions:
  contents: write

jobs:
  update-badge:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false
          fetch-depth: 0

      - name: Run tests
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing --cov=src tests/ | tee pytest-coverage.txt

      - name: Coverage comment
        id: coverage
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml
          hide-comment: true

      - name: Update README
        run: |
          sed -i '/<!-- Pytest Coverage Comment:Begin -->/,/<!-- Pytest Coverage Comment:End -->/c\<!-- Pytest Coverage Comment:Begin -->\n${{ steps.coverage.outputs.coverageHtml }}\n<!-- Pytest Coverage Comment:End -->' ./README.md

      - name: Commit changes
        uses: stefanzweifel/git-auto-commit-action@v5
        with:
          commit_message: 'docs: update coverage badge'
          file_pattern: README.md

📋 Output Example

Here's what the generated coverage comment looks like:

Coverage

Coverage Report
FileStmtsMissCoverMissing
functions/example_completed
   example_completed.py641970%33, 39–45, 48–51, 55–58, 65–70, 91–92
functions/example_manager
   example_manager.py441175%31–33, 49–55, 67–69
   example_static.py40295%60–61
functions/my_exampels
   example.py20200%1–31
functions/resources
   resources.py26260%1–37
TOTAL105573930% 

Tests Skipped Failures Errors Time
109 2 💤 1 ❌ 0 🔥 0.583s ⏱️

🔬 Advanced Features

📊 Using Output Variables
- name: Coverage comment
  id: coverage
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

- name: Dynamic Badges
  uses: schneegans/dynamic-badges-action@v1.7.0
  with:
    auth: ${{ secrets.GIST_SECRET }}
    gistID: your-gist-id
    filename: coverage.json
    label: Coverage
    message: ${{ steps.coverage.outputs.coverage }}
    color: ${{ steps.coverage.outputs.color }}

- name: Fail if coverage too low
  if: ${{ steps.coverage.outputs.coverage < 80 }}
  run: |
    echo "Coverage is below 80%!"
    exit 1
🎯 Show Only Changed Files
- name: Coverage comment (changed files only)
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
    report-only-changed-files: true

This is particularly useful for large codebases where you want to focus on coverage for files modified in the PR.

🔀 Workflow Dispatch Support
name: Manual Coverage Report
on:
  workflow_dispatch:
    inputs:
      pr_number:
        description: 'Pull Request number'
        required: true

jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - name: Coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml
          issue-number: ${{ github.event.inputs.pr_number }}
⚡ Performance Optimization

For large coverage reports that might exceed GitHub's comment size limits:

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
    hide-report: true # Show only summary and badge
    xml-skip-covered: true # Skip files with 100% coverage
    report-only-changed-files: true # Only show changed files
    remove-links-to-files: true # Remove clickable file links
    remove-links-to-lines: true # Remove clickable line number links

Link Removal Options:

  • remove-links-to-files: true - Removes clickable links to files. Instead of [example.py](link), shows plain example.py
  • remove-links-to-lines: true - Removes clickable links to line numbers. Instead of [14-18](link), shows plain 14-18

These options significantly reduce comment size while preserving all coverage information.

🎨 Badge Colors

Coverage badges automatically change color based on the percentage:

Coverage Badge Color
0-40% Coverage 0-40 Red
40-60% Coverage 40-60 Orange
60-80% Coverage 60-80 Yellow
80-90% Coverage 80-90 Green
90-100% Coverage 90-100 Bright Green

🔄 Auto-Updating Badge on README

If you want auto-update the coverage badge on your README, you can see the workflow example above.

Auto Updating Badge

Auto-updating badge example

📸 Result Examples

View example outputs

Standard Comment (Collapsed)

Collapsed Comment

Expanded Coverage Report

Expanded Report

Multiple Files (Monorepo)

Multiple Files

🔧 Troubleshooting

Common Issues and Solutions

Comment Not Appearing

Issue: The action runs successfully but no comment appears on the PR.

Solutions:

  • Ensure proper permissions are set:
    permissions:
      contents: write
      pull-requests: write
  • For workflow_dispatch, provide the issue-number input
  • Check if hide-comment is set to false

Coverage Report Too Large

Issue: "Comment is too long (maximum is 65536 characters)"

Solutions:

  • Use xml-skip-covered: true to hide fully covered files
  • Enable report-only-changed-files: true
  • Set hide-report: true to show only summary
  • Use remove-links-to-files: true to remove clickable file links
  • Use remove-links-to-lines: true to remove clickable line number links
  • Use --cov-report=term-missing:skip-covered in pytest

GitHub Step Summary Too Large

Issue: "GitHub Action Summary too big" (exceeds 1MB limit)

Solution: As of v1.1.55, the action automatically truncates summaries that exceed GitHub's 1MB limit.

Files Not Found

Issue: "No such file or directory" errors

Solutions:

  • Use absolute paths or paths relative to $GITHUB_WORKSPACE
  • For Docker workflows, ensure volumes are mounted correctly
  • Check that coverage files are generated before the action runs

Wrong File Links

Issue: Links in the coverage report point to wrong files or 404

Solutions:

  • Set default-branch to your repository's main branch
  • Use coverage-path-prefix if your test paths differ from repository structure
  • Ensure the action runs on the correct commit SHA

🤝 Contributing

PRs Welcome

We welcome all contributions! Please feel free to submit pull requests or open issues for bugs, feature requests, or improvements.

Development Setup

# Clone the repository
git clone https://github.com/MishaKav/pytest-coverage-comment.git
cd pytest-coverage-comment

# Install dependencies
npm install

# Run tests (if available)
npm test

# Build the action
npm run build

👥 Contributors

Contributors

📄 License

MIT © Misha Kav


🔗 Similar Actions

For JavaScript/TypeScript projects using Jest: Check out jest-coverage-comment - a similar action with even more features for Jest test coverage.


If you find this action helpful, please consider giving it a ⭐ on GitHub!

Pytest Coverage Comment is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.

About

Comments a pull request with the pytest code coverage badge and full report
v1.1.56
Latest

Pytest Coverage Comment is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.