Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 21, 2025

Problem

The OSV reporting schema for PyPI packages had two issues that caused validation failures:

  1. Incorrect ecosystem case: PyPI packages were using lowercase "pypi" instead of the proper case "PyPI" required by the OSV schema
  2. Wrong range type: PyPI packages were using "SEMVER" range type, but PyPI ecosystem doesn't support SEMVER ranges and should use "ECOSYSTEM" instead

This resulted in OSV schema validation errors like:

invalid OSV: ecosystem "PyPI" does not support `SEMVER` ranges

Solution

1. Added proper ecosystem mapping for OSV schema

Created a separate osvEcosystemMap that provides the correct case-sensitive ecosystem names:

var osvEcosystemMap = map[packagev1.Ecosystem]string{
    packagev1.Ecosystem_ECOSYSTEM_PYPI:     "PyPI",      // Proper case
    packagev1.Ecosystem_ECOSYSTEM_NPM:      "npm",
    packagev1.Ecosystem_ECOSYSTEM_RUBYGEMS: "RubyGems",
    // ... other ecosystems
}

2. Added conditional range type logic

PyPI packages now use ECOSYSTEM range type while other ecosystems continue using SEMVER:

rangeType := osvschema.RangeSemVer
if report.GetPackageVersion().GetPackage().GetEcosystem() == packagev1.Ecosystem_ECOSYSTEM_PYPI {
    rangeType = osvschema.RangeEcosystem
}

3. Preserved file path generation

The existing maliciousPackagesEcosystemMap remains unchanged to ensure file paths continue using lowercase (e.g., osv/malicious/pypi/package/...).

Result

PyPI packages now generate valid OSV schema:

{
  "affected": [
    {
      "package": {
        "ecosystem": "PyPI",
        "name": "adcv"
      },
      "ranges": [
        {
          "type": "ECOSYSTEM",
          "events": [
            {
              "introduced": "0"
            }
          ]
        }
      ]
    }
  ]
}

Testing

  • Added comprehensive tests verifying PyPI uses "PyPI" ecosystem and "ECOSYSTEM" range type
  • Added tests ensuring other ecosystems (NPM) continue using "SEMVER" range type
  • All existing tests pass, confirming no regressions
  • Manual verification confirms correct JSON output

Fixes the OSV schema validation issues for PyPI packages while maintaining backward compatibility for file generation and other ecosystems.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.npmjs.org
    • Triggering command: /tmp/go-build756587219/b1304/registry.test -test.testlogfile=/tmp/go-build756587219/b1304/testlog.txt -test.paniconexit0 -test.timeout=1m0s (dns block)
    • Triggering command: /tmp/go-build756587219/b1340/readers.test -test.testlogfile=/tmp/go-build756587219/b1340/testlog.txt -test.paniconexit0 -test.timeout=1m0s (dns block)
  • https://api.github.com/repos/safedep/vet/contents/README.md
    • Triggering command: /tmp/go-build756587219/b1340/readers.test -test.testlogfile=/tmp/go-build756587219/b1340/testlog.txt -test.paniconexit0 -test.timeout=1m0s (http block)
  • https://api.github.com/repos/safedep/vet/contents/docs
    • Triggering command: /tmp/go-build756587219/b1340/readers.test -test.testlogfile=/tmp/go-build756587219/b1340/testlog.txt -test.paniconexit0 -test.timeout=1m0s (http block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…ge type

Co-authored-by: KunalSin9h <82411321+KunalSin9h@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] For OSV reporting, the schema for pypi ecosystem should have property ecosystme with value "PyPI", not any other case. and the osv schemer check for ossf says invalid OSV: ecosystem "PyPI" does not support SEMVER ranges hence use "ECOSYSTEM" as valu... Fix OSV schema for PyPI ecosystem: use proper case "PyPI" and ECOSYSTEM range type Aug 21, 2025
@Copilot Copilot AI requested a review from KunalSin9h August 21, 2025 08:43
Copilot finished work on behalf of KunalSin9h August 21, 2025 08:43
Copy link

codecov bot commented Aug 21, 2025

Codecov Report

❌ Patch coverage is 64.70588% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 8.22%. Comparing base (2d06114) to head (a6fa473).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pkg/malysis/ossf.go 64.70% 6 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##            main    #570   +/-   ##
=====================================
  Coverage   8.21%   8.22%           
=====================================
  Files        284     284           
  Lines      47466   47476   +10     
=====================================
+ Hits        3901    3905    +4     
- Misses     43292   43298    +6     
  Partials     273     273           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@KunalSin9h KunalSin9h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abhisek
ecosystem for file path is correct like pypi for osv json schema require PyPI, thats why new map for those.

REF: ossf/malicious-packages#969

@KunalSin9h KunalSin9h marked this pull request as ready for review August 21, 2025 09:04
@KunalSin9h KunalSin9h requested a review from abhisek August 21, 2025 09:05
…tter intention revealing

Co-authored-by: abhisek <31844+abhisek@users.noreply.github.com>
@Copilot Copilot AI requested a review from abhisek August 21, 2025 09:58
Copilot finished work on behalf of abhisek August 21, 2025 09:58
@abhisek abhisek merged commit 41684af into main Aug 21, 2025
11 checks passed
@abhisek abhisek deleted the copilot/fix-b40b51f5-e8bb-4745-9fda-0bdf71e22103 branch August 21, 2025 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants