This PR makes the following changes:

• detects when license values are really full text, attempts to identify the license ID, and migrate the contents to the pkg.License.Content field and fills in the ID to pkg.License.Value
• adds a new license.content configuration, deprecating the existing license.include-unknown-license-content configuration. The configuration allows for the following values:
  • unknown (default): only include contents for licenses that cannot be identified
  • none: never include license contents
  • all: include license contents wherever possible
• replaces the license.license-coverage with license.coverage

# .syft.yaml configuration summary
license:
  # include the content of licenses in the SBOM for a given syft scan; valid values are: [all unknown none] (env: SYFT_LICENSE_CONTENT)
  content: 'none'

  # deprecated: please use 'license-content' instead (env: SYFT_LICENSE_INCLUDE_UNKNOWN_LICENSE_CONTENT)
  include-unknown-license-content:

  # adjust the percent as a fraction of the total text, in normalized words, that
  # matches any valid license for the given inputs, expressed as a percentage across all of the licenses matched. (env: SYFT_LICENSE_COVERAGE)
  coverage: 75

  # deprecated: please use 'coverage' instead (env: SYFT_LICENSE_LICENSE_COVERAGE)
  license-coverage:

Development details

The current license constructors have been deprecated. The new constructors are copies of the old ones but WithContext and accept context.Context as their new initial argument.

By refactoring these constructors we can now access the License Scanner during license construction. This allows all catalogers the have a file.ReadCloser to query the license scanner during license construction.

This enables #3088 to be solved.

The new builder (under the ctx license constructors) has better logic surrounding how to detect if a metadata value is a license ID, some custom license title string, or actually the full contents itself. By detecting this at construction and running the scanner against metadata values that are likely license contents we can prevent license.Value from being populated with the full contents of the license text.

All catalogers now return the full content of the licenses discovered. These contents are dropped from the licenses in a post processing step.

The user is given the option to enable if they want the contents of licenses to appear in their SBOM.

None
Unknown
All

SPDX OtherLicenses

Given some of the changes made regarding how the values and IDs are set during license construction in this PR, the SPDX OtherLicenses format code has been moved to be done with SPDX package creation in syft's format modules.

When SPDX packages are having their concluded and declared fields decorated, the spdx format code is now using the available license information to maintain a set of spdx.OtherLicense that will be returned with the packages.

This is different from our previous approach where spdx.OtherLicense would be recomputed after packages had already been assembled. This new approach is easier to test given that the licenses under packages are compared with spdx.OtherLicense when external validator tools are run against spdx formatted SBOM.

• Fixes Detect whether full license text or a license name has been provided #3088

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Documentation (updates the documentation)

Checklist

• I have added unit tests that cover changed behavior
• I have tested my code in common scenarios and confirmed there are no regressions
• I have added comments to my code, particularly in hard-to-understand sections