A more secure (and smaller) Big Bang

Over three years ago, when the container universe was young(er), Platform One created Big Bang, an open source tool to streamline the building and maintenance of secure, on-premise software factories. Big Bang’s goal was to empower those managing software factories with an easy “software factory in a box” experience. Since its inception, it has seen over 6,500 commits, 130 releases, and contributions from over 100 contributors. Moreover, Big Bang is still the baseline for a secure software factory for many organizations, including Platform One’s own Party Bus.

‍

Platform One is the United States Air Force’s accredited software factory used for building, deploying, and maintaining software factories in support of the greater Department of Defense community. It encompasses several products, including Iron Bank and Big Bang. Iron Bank is "Platform One's hardened container image repository," used exclusively by Big Bang as the source of all images.

‍

Today, we’re excited to announce that Chainguard, in cooperation with Iron Bank, has begun mirroring to Iron Bank the Chainguard Images versions of the core set of Big Bang images. These images will be freely available as a drop in replacement to the current Big Bang images.

‍

Until now, the only images available as the foundation for Big Bang were the upstream alternatives redistributed through Iron Bank. With Chainguard Images, there is now a complete alternative for Big Bang Core’s baseline. Users of Big Bang can now enjoy a substantial, as in 100%, decrease in CVEs, software components, and bits. This work involves over 30 images representing all eight of the core “components” being mirrored to Iron Bank’s registry. We’re grateful to the Iron Bank team for their help in mirroring these images and look forward to working with the team on future initiatives.

‍

Getting started with Big Bang (Chainguard’s version)

For those familiar with deploying Big Bang, you can get started using Chainguard Images as the base using the following values override:

helm install bigbang chart/ -f https://gist.githubusercontent.com/joshrwolf/27b0469f0f02ecfb28efb2b31f7110f1/raw/f47a668ef559f020878d8c55aa45ac313c403703/chainguard-values.yaml

The above method assumes you already have set up the Big Bang prerequisites and will point to Chainguard’s registry until the mirroring has been complete. These images represent their cgr.dev equivalents. When the remaining images get merged into Iron Bank, this gist will be replaced with their full registry1.dso.mil equivalents, and an MR will be opened against Big Bang with those values.

‍

For the BLUF, here is the end result one can expect in numbers:

‍

In short, the Chainguard Images version of Big Bang core, in comparison to the current Iron Bank status quo, has:

• 100% fewer CVEs

• 40% fewer components

• 72% fewer bytes pulled (diode users rejoice)

‍

Images are, by design, snapshots in time. These numbers can and will change with time. As such, the numbers above and for the rest of this post are accurate as of the time of publishing. Additionally, the analysis uses the linux/amd64 architecture because Iron Bank does not fully support aarch64.

‍

A hard look at the Iron Bank status quo

Iron Bank has over 1,200 images available to pull. These range from free and open source software images maintained by the Iron Bank team to commercial offerings or closed source licensed software maintained by external vendors in collaboration with the Iron Bank team. Most notably, Iron Bank leverages its Certificate to Field (CTF) to produce a body of evidence (ABC and ORA) used to streamline the approval process for organizations with auditors accepting the reciprocity. Unfortunately, for the remainder of folks, Iron Bank falls short of the promise of hardened images.

‍

To demonstrate, let's go through a simple yet very common example. Underpinning all of Iron Bank is the hosted Harbor registry, the open source image registry at registry1.dso.mil. If we sort by the highest pulled image on Harbor, we find istio/proxyv2. So let's start there and put our auditor hat on, the first thing we do is see what the CVEs are:

grype registry1.dso.mil/ironbank/opensource/istio/proxyv2:1.21.0
 ✔ Vulnerability DB                [no update available]
 ✔ Loaded image                                                                                                              registry1.dso.mil/ironbank/opensource/istio/proxyv2:1.21.0
 ✔ Parsed image                                                                                                 sha256:89642eb0677923b6c462cb9f7246dcefaa76934e9970a582abbfd4df403da044
 ✔ Cataloged contents                                                                                                  caaaedd6c28b9e78c98541171be817730e36b1f0a6606211e1949c0449609536
   ├── ✔ Packages                        [123 packages]
   ├── ✔ File digests                    [1,228 files]
   ├── ✔ File metadata                   [1,228 locations]
   └── ✔ Executables                     [426 executables]
 ✔ Scanned for vulnerabilities     [17 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 6 medium, 0 low, 9 negligible (2 unknown)
   └── by status:   3 fixed, 14 not-fixed, 0 ignored

Let’s dig into these 17findings. In addition to a registry, Iron Bank also maintains a CVE “advisories feed” at vat.dso.mil, which aggregates justifications for each of the identified CVEs. Unfortunately, this feed isn’t recognized by vulnerability scanners, which leaves it up to us to manually review the feed. Within the VAT we can view justifications for all the known CVEs. For our image, we discover that 5/6 of the medium CVEs are marked as True Positive, and are waiting for the upstream istioproject to pull in the patches upstream. Hold on, aren’t these hardened images?

‍

Admittedly this is an oversimplified “audit,” but it is just realistic enough to illustrate a fundamental problem with the modern day Iron Bank: images are not built from source. Let's go back to istio/proxyv2to illustrate. The source of all Images live at repo1.dso.mil, where we can find our istio/proxyv2Dockerfile. From here, it’s clear this image is little more than a repackage of upstream, with manual COPYdirectives used to copy the pre-built upstream binaries. Not only does this confuse scanners, it is error prone, not reproducible, and makes Iron Bank entirely dependent on upstream for little to no additional value.

‍

We acknowledge that so far we’ve only examined istio. We leave it as an exercise for the reader to inspect additional images they may care about and validate the claims. Here are a few more Big Bang Core images to get the reader started: kyverno, prometheus, grafana, loki.

‍

Leveraging Wolfi, our linux undistro, Chainguard Images are rebuilt daily from source, stripped down to contain only the components needed to run the application, and rapidly patched against new vulnerabilities. The result is what we see here: drastically reduced CVE counts that eliminate the mundane, expensive toil of vulnerability management.

‍

If you’re a user of Big Bang and this approach resonates with you, we encourage you to try these images out. Currently, you can pull them directly from Chainguard’s registry at cgr.dev. And, soon, they will be available as mirrors through Iron Bank. Additionally, you can pull them through Docker Hub.

‍

We aim to collaborate with the United States Air Force and the Platform One team to enhance Iron Bank's security and realize its potential as the leading container registry, delivering the world's most secure and hardened container images.

‍

If you are interested in learning more about Chainguard and its work to support public sector organizations, join us on Monday, April 29 at 4pm ET at Carahsoft in Reston, VA for a discussion and happy hour with Dan Lorenc, CEO and Co-founder of Chainguard. Register here.