Hardened Container Images: Images for a Secure Supply Chain

Observers of software supply chain security products often wonder what market category Chainguard Images fits in. For instance, some analysts locate Chainguard Images within a “container security” category, which is, at first glance, reasonable. Chainguard Images do, in fact, improve the security of containers. Relatedly, at a recent KubeCon conference, attendees who stopped by the Chainguard booth often wondered what kind of “container scanner” Chainguard is. Given that many container security companies do offer container scanners, this question is understandable.

‍

But we think it’s time for “container security” to be split in the same way that a crowd at a house party needs to spill into the back yard. The container scanning companies have been successful, which is why the party got crowded. But Chainguard is headed to the backyard for some breathing space.

‍

This is why Chainguard is announcing its new inaugural State of Hardened Container Images report. It’s the report for companies using containers that want to start left, not just shift left. The report analyzes the security of a wide variety of “hardened” container images including, among others, those from Red Hat, the U.S. Air Force’s Iron Bank (more details about this one here), and Chainguard.

‍

Read on to understand why Chainguard believes this container security party needs a closer look, and what our analysis says about the state of hardened container images.

‍

The Need for a Hardened Container Image Category

Container vulnerability scanners, which inspect containers for known common vulnerabilities and exposures (or CVEs), have become wildly popular. Whether it’s open source scanners like Grype, or proprietary ones from companies like Snyk or Wiz, there are more scanners today than one can shake a stick at. Consequently, container vulnerability scanners have largely become a commodity.

‍

As the scanner rave dragged on, there is now a CVE hangover. All those companies deploying containers, a newfangled way to bundle custom code and its dependencies and ship it to the cloud DevOps-style, have discovered that container scanners often find tens, or even hundreds, of CVEs per container. While scanner companies hoped offering CVE remediation advice would fix the problems their scanners identified, it’s, as the research later will suggest, not that simple.

‍

The high CVE numbers can not only be alarming, but also costly for a variety of reasons. While many of the CVEs are false positives, a vulnerability that is flagged by a scanner despite not actually being present (you can learn more here), there are plenty of real ones.

‍

This inundation of CVEs in containers motivated Chainguard to build a product solely focused on redesigning container images from the ground up with security and compliance in mind, hence “hardened” container images.

‍

This is Chainguard Images, our suite of minimal container images with low-to-no CVEs, and the option of a FIPS-validated version. The minimalism, which means fewer packages and smaller size, reduces the attack surface and lowers the CVE count. But when CVEs do inevitably pop up in Chainguard Images, we remediate them within a defined service-level agreement.

‍

Introducing the Hardened Container Image Report

To help container users appreciate the landscape of hardened containers, Chainguard has released its inaugural “Hardened Container Image” report. The report surveys a range of container images, some more hardened than others, and a couple of container hardening techniques.

‍

The analysis compares each set of container images with an equivalent or similar set of Chainguard Images. The report uses the open source vulnerability scanner Grype to measure the number of CVEs reported in each image or set of images.

‍

A selection of key findings include:

• Popular Debian-based, community-supported images that have a Chainguard Images equivalent have, on average, nearly 300 CVEs. This number of CVEs is at least in part due to these container images including, on average, nearly 300 components, or open source packages.

• Simply updating the packages in a sub-sample of popular Debian-based images to the latest package version available provides only a modest five percent reduction in the total count of CVEs.

• Detailed analysis of one container “debloating” technology, which finds a CVE reduction rate of approximately 65 percent, suggests that this technique is only moderately effective.

• An analysis of Red Hat-provided container images suggests that these images contain, on average, nearly 200 CVEs. This count excludes the hundreds of CVEs that Red Hat’s security team has labeled “will not fix.”

• The 50 most downloaded images in Iron Bank, a U.S. Air Force repository of hardened container images, have, on average, 110 CVEs.

• Canonical’s Chiselled images have few or no CVEs and are minimal, but the collection of available images is currently small, and adoption requires power-users.

‍

To put it in stark terms, this analysis suggests that the container — the crucial building block of the cloud and the unit upon which many other elements of digital infrastructure are built — is a security dumpster fire.

‍

The results eventually lead us back to Chainguard Images, an ever-growing collection of minimal, hardened, low-to-no CVE container images. The report also addresses likely objections, including the question of whether a focus on CVEs is short-sighted. Essentially, CVEs are only one measure of security, but ignoring CVEs is like ignoring an open wound on the grounds that blood is only one measure of health.

‍

Hardened container images: Come join the new party

This whole container security shindig has gotten popular, perhaps too popular. “Container security” has become a catch-all market category too wide for its own good.

‍

Container security, as it currently stands, is like a market category that includes both toast (browned bread) and a toaster (a device that makes toast). Container security currently includes tools for scanning containers (often, but not exclusively, for CVEs), and hardened container images. Just as toast and a toaster aren’t the same thing — container scanners and hardened container images aren’t the same thing either.

‍

That’s why it’s time for a hardened container images market category. It’s a chance for cloud-native software teams to build left, not shift left. By using minimal, low-to-no-CVE containers built for compliance requirements, software teams can avoid the security risks and regulatory headaches of the current container status quo.

‍

It’s a chance for software companies to stop wasting staff time on tedious CVE management and to get out of the business of building and maintaining their own fleet of trusted or “golden” container images, and get back into innovating their own business.

‍

And for companies with a massive compliance nightmare (for instance, companies maintaining or seeking FedRAMP compliance), it’s a chance to unlock new markets faster and at less cost with less risk of falling out of compliance.

‍

In short, it feels good out here. Come join the new hardened container images party with Chainguard.

‍

Download the report and learn how the current landscape of hardened container images shapes up. If you want to learn more about Chainguard’s hardened container image solutions, get in touch with our team today.