Intro to OCI Reference Types

What is OCI?

OCI stands for Open Container Initiative. This is a group which oversees a collection of open specifications relating to containers. If you have ever run an application on Kubernetes, then you have leveraged OCI.

There are 3 primary OCI specifications:

(1) the runtime spec, which defines how to properly run a container,

(2) the image spec, which defines how to package and store a container image

(3) the distribution spec, which defines how to transfer a container image.

Registries: not just for your container images

It’s worth noting that over the past several years, there has been a trend in using OCI registries to store non-container things (notably, Helm charts).

People have been doing this for a long time. Under the hood, the API defined in the distribution spec is extremely generic, and can apply to just about any artifact type. Besides, every major cloud vendor provides a registry, so consolidation means less infrastructure to manage and secure.

Various efforts, such as OCI Artifacts, attempt to provide guidelines for how to properly achieve this, but not everyone is in full agreement on this topic.

What’s important is that there is increasing appetite and acceptance for putting all types of things inside the registry, for example SBOMs and signatures.

Cosign and OCI: a clever hack

The Cosign project (part of Sigstore) has been doing something very interesting with OCI, which might even be considered to be a hack of sorts.

Without any changes to the OCI Specs listed above, Cosign provides a way to link signatures to a given container image. But how?

Well, it's pretty neat, actually. It takes the SHA256 checksum of the image, and pushes a signature object with a tag in the following format:

sha256-<sha256_checksum_of_image>.sig

When Cosign attempts to verify an image, it looks for a tag in this format to find the signature, fetches it from the registry, and compares it to the user-provided public key. Simple and brilliant.

This method works, but it is clearly a workaround for the current limitations of OCI. Tags in this format are not defined by any OCI specification. This results in a polluted list of tags which you might see in your registry’s UI:

The Harbor project, in an upcoming release (v2.5.0), will address this issue by assuming these tags to be Cosign-related, and marking them as signature icons in the UI. Time will tell if other registries will follow suit.

Beside this, an excessive number of HTTP requests to the registry is required just to determine if an image contains an associated signature.

Enter the OCI Reference Types Working Group

The growth of Cosign and Sigstore gave rise to concerns about this usage pattern. Longer term, what is the best way to link container images to signatures and other types of artifacts?

There were a number of separate proposals made across OCI designed to tackle this problem. Unfortunately, getting changes ratified in OCI is no small feat. The group is notably conservative about making any changes whatsoever, since any such modification could have unforeseen consequences on the infrastructure underpinning the entire Cloud Native ecosystem (yikes and wow).

It started to become clear that if OCI were to make any progress in this area, it would require general consensus across the various stakeholders and vendors who would be most affected by changes of this scale.

All of this resulted in the formation of the OCI Reference Types Working Group, which has been operating now for a few months.

The working group (“WG”) has declared the following mission statement:

Propose how to describe and query relationships between objects stored in an OCI registry

In other words, the WG is coming up with a plan for how the existing OCI specifications might be modified to provide a more official solution to do the types of things Cosign does today.

A Peek at the Proposals

At the time of writing, there are a total of 4 proposals contributed by members of the WG to solve the issue of reference types.

It’s worth mentioning that the WG will likely combine various elements from each of these proposals to form the final solution. This is to say that these proposals should not be viewed as competing with one another, but rather providing a variety of options for the group to work with.

The descriptions below are merely summaries, and you are encouraged to read the proposals in their original form.

Proposal A: Defines a new artifact manifest and corresponding referrers extension API

Link

This proposal declares a new manifest type called an “artifact manifest” similar in form to an OCI image manifest. There are various additional fields, such as subject which declares a reference to another manifest, and artifactType, which declares the type of artifact (signature, SBOM, etc.), and blobs, which is a non-ordinal replacement for the layers field.

Additionally, a new GET endpoint is proposed to the registry HTTP API called /referrers, which returns a list of artifacts which reference a provided image/artifact. This endpoint also supports advanced filtering, sorting, and pagination using query parameters.

Proposal B: Add "reference" field to existing schemas

Link

This proposal suggests the addition of a single new field on existing manifest types, reference, which allows the creation of a relationship between an image and an artifact (such as a signature).

Additionally, a new GET endpoint is proposed to the registry HTTP API called /references, which returns a list of artifacts which reference a given image. The format of this response matches the schema of an OCI image index.

Proposal C: Create Node manifest

Link

This proposal declares a new manifest type called a “node manifest”. This new manifest type declares a new objects field, which allows a list of artifacts which comprise the node, and a new reference field, which specifies the parent reference.

Additionally, 4 new GET endpoints are proposed to the registry HTTP API: the (1) /referrers endpoint returns a list of descriptors referencing a tag or digest, (2) the /reference endpoint which returns the parent reference, (3) the /objects endpoint, which returns a list of objects for a given parent reference, and (4) the /description endpoint, which returns a simple text description of the reference.

Proposal D: No Changes

Link

This proposal is unique in that it does not actually suggest any changes. Instead, it aims to provide official guidance on how the existing OCI schemas and APIs can be used today to declare relationships between objects.

Through the use of 2 reserved annotations, org.opencontainers.reference and org.opencontainers.reference.type, existing manifest types can reference another digest in the registry and declare the reference type (e.g. “sig”).

In addition, the tagging format <repo>:<alg>-<ref>.<hash>.<type> is defined explicitly for how to determine if a reference exists for a given digest. This is strikingly similar to the way that Cosign already functions today.

What’s Next?

With these proposals on the table, the WG will continue to meet to discuss the best path forward.

One of the primary challenges will be how to handle backwards compatibility, since there will undoubtedly be a gap between the time that changes appear in OCI specs and the time that registries decide to adopt them. If history is any lesson, this could mean years. In fact, some registries may never decide to update.

Another challenge will be getting the changes suggested by the WG actually merged into the OCI specs. There is unfortunately no guarantee that the maintainers on each of the affected specs (image-spec, distribution-spec) will accept the final output of the WG. Luckily there is some maintainer overlap here.

As far as Cosign is concerned, there is already an open issue to track any changes that might be recommended by the WG in order to stay ahead of the curve.

Finally, if you have any opinions on this topic or want to help out, you are highly encouraged to get involved with the WG! For more information on meeting times, etc, please see the project README.