Skip to content

artifactType in index manifest and in its "manifests" descriptors #1670

@apparentlymart

Description

@apparentlymart

What is the version of your ORAS CLI

1.3.0-beta.2

What would you like to be added?

I've been experimenting with using the new oras manifest index create command to construct multi-platform index manifests for the new (currently in development) OCI Registry-based provider package distribution mechanism in OpenTofu.

The index-building functionality broadly worked, but in order to give better feedback to users when they provide us the wrong kind of artifact we currently require the artifactType property to be populated in two locations:

  1. In the index manifest's top-level artifactType property, we use a type that represents a provider version as a whole, agnostic of platform.

  2. In each of the OpenTofu-relevant descriptors in the manifests property, we expect to find a copy of the artifactType property from the manifest itself, so that we can efficiently filter any manifests that are not relevant.

    (This particular situation is primarily aimed at forward compatibility with future changes to the manifest layout that we cannot anticipate today, in case a future requirement causes us to mix two different kinds of "image" artifact into the same index, with older versions of OpenTofu ignoring the newer ones they don't understand.)

For my problem 1, I'd propose adding an --artifact-type option to oras manifest index create which has similar meaning to the option of the same name on oras push, causing the given type to be written into the top-level artifactType property.

For my problem 2, I wonder if the code that builds the index manifest could copy the artifactType property from the target manifest into the corresponding descriptor, in a similar way to how it currently populates the platform property based on information from the manifest's config (or, if #1538 is implemented, from some ORAS-specific annotations in the manifest.)

Overall then, I'd like to be able to:

  • Construct one or more image manifests that each have our artifactType and the appropriate platform by running oras push --artifact-type=application/vnd.opentofu.provider-target --artifact-platform $os/$arch ...
  • Construct the image manifest by running oras manifest index create --artifact-type=application/vnd.opentofu.provider ...
  • Have the final index manifest include the relevant artifactType properties, so we can quickly reject/ignore irrelevant or incompatible artifacts based only on the index manifest. without fetching any image manifests:
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "artifactType": "application/vnd.opentofu.provider",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:f22b608064cba5d2bcea544ea636acca254ff3dcfcf7e39f6218b302c07c0a80",
      "size": 616,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:9d4fa52874a3beb018d3c22683b3c49c2e2821a29c5479454958a59b5d8fee9b",
      "size": 616,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:7e60f35171ad8a53cfe1b5e0bb771e464b748e9468e62151712947c16e5c9618",
      "size": 618,
      "platform": {
        "architecture": "amd64",
        "os": "windows"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:e90dba706bdf891c1c9b5a83758061d64c9a4e7f7bea6b7895b6259fb27c6bd1",
      "size": 618,
      "platform": {
        "architecture": "arm64",
        "os": "windows"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:d8635054545b76d40048e81d46a0620e49ff1c94831d864a37ab1ac3d4cefdec",
      "size": 617,
      "platform": {
        "architecture": "amd64",
        "os": "darwin"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "artifactType": "application/vnd.opentofu.provider-target",
      "digest": "sha256:7f591880d0cc947eb723dd5091ae360e7a1ba5d49e9d8478c6f6df59021c4b5c",
      "size": 617,
      "platform": {
        "architecture": "arm64",
        "os": "darwin"
      }
    }
  ]
}

Why is this needed for ORAS?

OpenTofu's provider installer would like to be able to:

  • Immediately reject any index manifest that doesn't have the artifact type application/vnd.opentofu.provider, to give users good feedback that they've selected the wrong artifact.
  • Silently ignore any manifests in the index that don't have the artifact type application/vnd.opentofu.provider-target, so that a future version of OpenTofu could potentially introduce a new artifact type to meet an as-yet-unrealized need while allowing the publication of index manifests that are backward-compatible with older versions that only understand application/vnd.opentofu.provider-target.

We could potentially meet these requirements by downloading each index manifest that has the appropriate platform and checking its artifactType, but we'd prefer to be able to deal with this using only information in the index manifest.

Are you willing to submit PRs to contribute to this feature?

  • Yes, I am willing to implement it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions