Skip to content

c8 Code Coverage Displaying Code Misses on Multi-line Comments #515

@eliatcodecov

Description

@eliatcodecov

(Note that the reported issue below was encountered in CI on github action's ubuntu-latest (the ci.yaml used). I'm not directly sure what version of Node is packaged with github action's ubuntu-latest, but I'll try to find out.)

  • Version:
  • Platform: ubuntu-latest image on GitHub Actions.

Greetings from Codecov! Recently we've encountered some strange reporting using the v8 (i.e., c8) reporter in vitest that I wanted to surface as an Issue for your consideration.

It seems like multiline comments are are showing up as coverage misses. Additionally, it seems like -- in some cases -- the first line of these multi-line comments is counted as a conditional false (an uncovered branch in other words). You can see precisely what this looks like on Codecov for this commit.

Note the addition of the license comment in the diff and how Codecov is reporting as fully uncovered.

We can see v8 as the coverage reporter in the vitest config of the repo here

I dug into the raw uploaded reports a bit (the can be downloaded from the Uploads sidebar here). And this is clearly an issue with the coverage output itself and not Codecov directly. Here's a snippet of the output for the tracker.js file at SHA 250b1b2:

 <package name="counterscale.public">
      <metrics statements="165" coveredstatements="0" conditionals="1" coveredconditionals="0" methods="1" coveredmethods="0"/>
      <file name="tracker.js" path="/home/runner/work/counterscale/counterscale/public/tracker.js">
        <metrics statements="165" coveredstatements="0" conditionals="1" coveredconditionals="0" methods="1" coveredmethods="0"/>
        <line num="1" count="0" type="cond" truecount="0" falsecount="1"/>
        <line num="2" count="0" type="stmt"/>
        <line num="3" count="0" type="stmt"/>
        <line num="4" count="0" type="stmt"/>
        <line num="5" count="0" type="stmt"/>
        <line num="6" count="0" type="stmt"/>
        <line num="7" count="0" type="stmt"/>
        <line num="8" count="0" type="stmt"/>
        <line num="9" count="0" type="stmt"/>
        <line num="10" count="0" type="stmt"/>
        //...truncated
        <line num="32" count="0" type="stmt"/>
        <line num="33" count="0" type="stmt"/>
        <line num="34" count="0" type="stmt"/>
        <line num="35" count="0" type="stmt"/>
        //...

In the above xml output you can clearly see the conditional false reporting on line 1 as well as the multiple coverage misses for the multiline comment itself. I'm not sure if the issue is with the clover based XML output, or something more fundamental, but thought I'd surface it here since the example that led to the misreported coverage is so straightforward. If there's anything else you need, please let me know. Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions