Building towards OCI v1.1 support in cosign

The Open Container Initiative (OCI) is inching closer to a v1.1 release which provides official guidance on how to connect things in a registry. In our blog post from last year Intro to OCI Reference Types, we walkthrough the current state of OCI and why these changes are becoming increasingly necessary in the realm of supply chain security.

‍

Latest developments in OCI land

The OCI Reference Types Working Group has completed its mission in proposing how to describe and query relationships between objects stored in an OCI registry. The group is no longer meeting, and the project has been archived. Proposal E (“cherry pick”) was introduced, which combined various ideas from other proposals. The changes defined in Proposal E were then organized and submitted as pull requests which were subsequently approved by OCI spec maintainers:

‍

• opencontainers/distribution-spec#335

• opencontainers/image-spec#934

All that is left to do is to cut v1.1 releases for both distribution-spec and image-spec for these new features to be considered OCI-supported.

‍

Since the PRs above have been merged, there have been small updates made to the specs to solidify some of the proposed behavior. There has also been some ongoing debate regarding some of the more controversial changes. Despite the delay, members of the OCI community appear determined to resolve these issues and release v1.1 specs as soon as possible.

‍

What OCI v1.1 means for cosign

As we described in Intro to OCI Reference Types, cosign has already “solved” the problem of relationships between objects in registries. It does so using a method that the OCI community has come to refer to as “polluting the tag space”.

‍

As a recap, cosign takes the SHA256 checksum of an image, and pushes a tag to the registry in the following format:

sha256-<sha256_checksum_of_image>.<sig/sbom/att>

This approach works just fine, but presents a couple downsides:

‍

• The registry ends up with tons of non-descriptive tags (vs. “latest”, “1.2.3”, etc.)

• The manifests for these attachments are not easily distinguishable from that of an ordinary OCI image

• A large number of HTTP requests are made to the registry

• This type of behavior is not defined by any OCI specification, and therefore isn’t considered interoperable across the ecosystem

‍

In OCI v1.1, a variation of this approach has been introduced called the “Referrers Tag Schema”. Instead of publishing a new tag for each attachment, the following tag is updated by the client each time a new reference is added:

sha256-<sha256_checksum_of_image>

The manifest published to this tag should be a valid OCI Image Index. In the manifests array should be a list of all references to this image. Here is an example of what this might look like:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 5825,
      "digest": "sha256:478cf2f3284811b500cc3aee71b7d8c3a744917e863a78922674211c75b1e36c",
      "artifactType": "application/vnd.dev.cosign.artifact.sig.v1+json"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 539,
      "digest": "sha256:e952330096c5349c253bb11c09a25c546ca35d4b2a1bfaac805fee90cbe8fde1",
      "artifactType": "application/vnd.dev.cosign.artifact.sbom.v1+json"
    }
  ]
}

Notice in the JSON above the new artifactType field. This is how arbitrary types in the registry can be distinguished instead of appending “.sig” or “.sbom” to the tag. The artifactType is derived from the config.mediaType field on the manifest of the attachment. For example, here is what the beginning of an SBOM manifest might look like:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.dev.cosign.artifact.sbom.v1+json",
. . .

Keep in mind that this is the same method that Helm, OPA, and other tools have already started storing arbitrary content. If you’ve ever come across the opaque term “OCI Artifacts”, this is what we are talking about. Even though this method and terminology have been around for a long time now, OCI never officially codified this into a specification.

‍

In OCI v1.1, the config.mediaType field is considered important, as it maps to the artifactType field when we are dealing with references. The underlying behavior of “OCI Artifacts” will finally be considered a supported mechanism for storing arbitrary content in a registry! You can learn more about the term "OCI Artifacts" on Chainguard Academy.

‍

So I still need to use tags?

The Referrers Tag Schema described above is considered in OCI v1.1 to be simply a fallback mechanism. This means that this behavior should work in existing v1.0 registries without any changes needed.

‍

One of the biggest additions in OCI v1.1 is the new “Referrers API”. This defines a brand new endpoint on the registry which allows fetching a dynamic list of references:

GET /v2/<name>/referrers/<digest>

The response of this call should be identical to OCI Image Index described above using the Referrers Tag Schema. This means clients can first check this endpoint for references, and if a 404 is returned, then the client can try to fetch the fallback tag. This response from either request should be able to be processed in the exact same way.

‍

The Referrers API has additional features such as pagination and the ability to filter by artifactType (e.g., “give me only SBOMs”).

‍

In addition, if a registry supports the Referrers API, no additional tags are needed! This is enabled by another feature coming in OCI v1.1 - a new field on the OCI Image Manifest called subject.

‍

When publishing an “OCI Artifact”, you can include an optional subject section which points to an image which this artifact references:

. . .

 "subject": {
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "size": 491,
    "digest": "sha256:97d731da806109c4f61ac32e8830e13952ca2c65bc7fbfe32babd7d267e633bf"
  }
}

When uploading a manifest to the registry containing a subject section, a registry supporting the Referrers API is required to respond with the following HTTP header to indicating that the subject was processed:

OCI-Subject: <subject digest>

If this header is detected, then the client knows that it does not need to upload a fallback tag using the Referrers Tag Schema.

‍

Putting the pieces together

Even though OCI v1.1 has not yet been officially released, we decided to put together a prototype of cosign using some of these new features. You can find a working version of this prototype on cosign main.

This prototype is powered by an unreleased version of google/go-containerregistry (“GGCR”), which is the library that cosign leverages under the hood to work with OCI registries.

‍

Here we will walk through the new behavior found in the prototype.

‍

Note: if you are trying these steps against a different registry, it must accept the new subject field. Most OCI v1.0 registries should allow this. If they do not, they are technically out of compliance.

‍

First clone the repo and build cosign:

git clone https://github.com/sigstore/cosign.git
cd cosign
make cosign

Next, set the COSIGN_EXPERIMENTAL environment variable to enable the new behavior:

export COSIGN_EXPERIMENTAL=1

We will need a registry to work with. First, let's try using an OCI v1.0 registry (one that does not support the new Referrers API). We can run a local instance of CNCF Distribution on port 5001 in the background:

docker run -d --name oci-1-0-reg -p 5001:5000 registry:2.8.1

Next, let’s copy an existing image into the registry using crane:

crane cp cgr.dev/chainguard/wolfi-base localhost:5001/wolfi-base

Create a dummy SBOM:

echo '{"hello": "world"}' > sbom.json

Now, using our locally-built version of cosign, attach this SBOM to the image:

./cosign attach sbom --sbom sbom.json --type spdx --registry-referrers-mode oci-1-1 localhost:5001/wolfi-base

Note the new --registry-referrers-mode flag which must be set to “oci-1-1”. Using crane, check the list of tags on the repository:

crane ls localhost:5001/wolfi-base

You should notice 2 tags: a “latest” tag, as well as a tag using the Referrers Tag Schema (actual digest will vary depending on wolfi-base image):

latest
sha256-66bc3f5a4eb524faa2b1128f36cf77cbde57078e710f5e2730d375b93c9ee3d7

Notice the digest tag does not have a “.sbom” suffix. Using crane and jq, take a look at the manifest for this tag:

TAG="$(crane ls localhost:5001/wolfi-base | grep sha256-)"
crane manifest localhost:5001/wolfi-base:${TAG} | jq

This should look something like the following:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "size": 538,
      "digest": "sha256:b0f73f6a06cdd374a61e59e566f62c3d91e3af0814d6685e0cbc7cd17b4e5d9c",
      "artifactType": "application/vnd.dev.cosign.artifact.sbom.v1+json"
    }
  ]
}

We can also inspect the manifest of the SBOM itself:

SBOM_DIGEST=$(crane manifest localhost:5001/wolfi-base:${TAG} | jq -r .manifests[0].digest)
crane manifest  localhost:5001/wolfi-base@${SBOM_DIGEST} | jq

This should look something like the following:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.dev.cosign.artifact.sbom.v1+json",
    "size": 233,
    "digest": "sha256:a41602efc76dd664b9003584aade9a2d73fb57064e1ef7930a3cf328f82de67f"
  },
  "layers": [
    {
      "mediaType": "text/spdx+json",
      "size": 9,
      "digest": "sha256:437db28f0dee16cf955bb3cb4cefee59fde5d4b360c6acfaef21b5a1f83e8d37"
    }
  ],
  "subject": {
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "size": 491,
    "digest": "sha256:66bc3f5a4eb524faa2b1128f36cf77cbde57078e710f5e2730d375b93c9ee3d7"
  }
}

Notice the config.mediaType on this manifest indicating the artifactType, as well as the subject section pointing to the digest of the image.

So that’s neat and all, but what would happen if we were using an OCI v.1.1 registry?

‍

We can use zot, an open-source registry which has been closely tracking the progress of OCI v1.1 and has support for the Referrers API.

‍

Let’s start a local zot registry in the background on another port (5002):

docker run -d --name oci-1-1-reg -p 5002:5000 ghcr.io/project-zot/zot-minimal-linux-arm64:v2.0.0-rc2

Once again, copy an image and attach an SBOM:

crane cp cgr.dev/chainguard/wolfi-base localhost:5002/wolfi-base
./cosign attach sbom --sbom sbom.json --type spdx --registry-referrers-mode oci-1-1 localhost:5002/wolfi-base

Now list the tags:

crane ls localhost:5002/wolfi-base

This time we should only see just one tag: “latest”:

latest

So where is the SBOM?

Since zot supports the Referrers API, cosign does not have to create the digest tag, but should still be able to retrieve SBOM since it was uploaded with a subject section.

To prove it, we can try downloading the SBOM:

./cosign download sbom localhost:5002/wolfi-base

That should return us the contents of the dummy SBOM:

{"hello": "world"}

🤯

‍

If you rerun the previous command with the --verbose flag and inspect the logs, you can see all of the HTTP requests made to the registry. Here you will find evidence of cosign making calls to the Referrers API (ctrl+f “referrers”):

‍

--> GET http://localhost:5002/v2/wolfi-base/referrers/sha256:66bc3f5a4eb524faa2b1128f36cf77cbde57078e710f5e2730d375b93c9ee3d7

‍

When you’re done playing around, make sure to shutdown your local registry instances:

docker rm -f oci-1-0-reg oci-1-1-reg

What’s next?

We will continue to work with the OCI community in making the v1.1 release a reality. Once this is finalized, you can expect to see some form of stable support land in an upcoming cosign release. Learn more about the OCI project on Chainguard Academy.

‍

Stay tuned!