Making vulnerability data better for machines (and humans!) with OpenVEX: How Isovalent and Chainguard use OpenVEX

If every spreadsheet that is routinely emailed with the instructions “edit this and email it back to me” is an invitation to start a new software business, then the current state of software vulnerability management is a daily deluge of invitations to software professionals around the world to start a new company.

‍

Software vulnerability management is, to put it lightly, terrible. It’s mostly incomprehensible spreadsheets with little context sent from one person who poorly understands the information to another person who understands the information even more poorly. One of us has observed vulnerability data stored in spreadsheets, Notion pages, Jira tickets and a number of other idiosyncratic approaches. This leads to fragmentation, duplication of work, and no machine-readable way to ingest and analyze this data. Consequently, these vulnerabilities represent not just a security risk but also a giant drain on the time and attention of skilled software professionals.

‍

Enter VEX, the Vulnerability Exploitability eXchange, a security advisory that indicates the status of a software product or component with respect to a vulnerability. It’s a machine-readable document that often indicates whether a piece of software is or is not affected by vulnerability. And OpenVEX is an implementation of VEX, offering a specification, software library, and set of tools to help humans turn vulnerability data from a mess of spreadsheets into helpful information.

‍

Both Isovalent and Chainguard have adopted OpenVEX. This post explains the uses and benefits of OpenVEX at both organizations. If you or your organization is interested in adopting OpenVEX, you can learn more about OpenVEX, learn about the specification and tools, or join the bi-weekly OpenVEX meeting.

‍

How Isovalent Uses OpenVEX

At Isovalent, we aggressively minimize the number of third-party vulnerabilities in our packages identified by container vulnerability scanners. For virtually all of the code that we ship, we use automatic dependency updates via Renovate to ensure that we’re on the latest version of a particular dependency. Even though most vulnerabilities identified by container scanners are not practically exploitable, we find that not having to triage vulnerabilities individually saves a significant amount of time.

‍

This approach only gets us so far, however. There are numerous situations where supply chain vulnerability reports cannot be resolved via a trivial upgrade. For example:

• Base image providers may decide that a particular issue is a ‘wontfix’ because it’s not exploitable.

• The intricacies of our dependencies sometimes makes us reluctant to upgrade a package if the vulnerability is not “reachable.

• ”Third-party developers (justifiably) might not have the time or the interest to cut new releases, in particular when container scanners are reporting false positives.

‍

To deal with these and related problems, Isovalent has started using OpenVEX to simplify our security release processes. Our continuous integration (CI) system builds and scans our container images every day using multiple scanners, and we have an internal threshold at which the scan will fail. If a scan fails, we then examine the vulnerabilities that caused the failure. If it’s possible to bump the vulnerable dependencies, we do so. If it’s not, we triage the report for applicability. For applicable issues, we devise mitigations and implement them via our standard vulnerability management processes. For inapplicable issues, however, we use OpenVEX.

‍

We maintain golden OpenVEX files in our repositories, and we use them to collect exploitability information for the container images that we ship. After an inapplicable vulnerability has been identified, we open a pull request to update the vulnerability, adding the relevant data, for example:

{
     "vulnerability": "CVE-2023-29405",
     "products": [
       ...
     ],
     "status": "not_affected",
	 ...
   },

Once the file has been updated, it automatically feeds into the next CI run and the container scanners results are filtered using vexctl and our golden OpenVEX document, ideally allowing the check to pass again. The advantages are several:

• Vulnerability applicability information can be discussed using a traditional code review workflow, and when changes are made the git log can explain why and by whom.

• Multiple systems from different vendors can take advantage of the open format to process the OpenVEX data, removing the need for manual document wrangling.

• Collecting the data in a computer-friendly format allows it to be processed in a variety of ways, enabling CI pipelines and automatic metric generation.

‍This is just the first step in our journey with VEX. As adoption grows, we are hopeful that our customers will start accepting VEX documents instead of spreadsheets when triaging vulnerability information. Bundling OpenVEX documents alongside the SBOMs that we already ship will allow our customers to get to production much faster.‍

How Chainguard Contributes to OpenVEX

Over the past year, Chainguard has been busy at work to make VEX a reality. Chainguard’s support for VEX stems from our belief that vulnerability management is too painful, in part because our customers are drowning in false positives from container vulnerability scanners. Additionally, we saw that there was a need for a lightweight VEX format, one that is independent of SBOM formats. This is why we started working with industry partners and the VEX Working Group, facilitated by CISA, on a simple format: OpenVEX. OpenVEX was designed from the ground up to be a working VEX interchange format, not to be an extension of other standards. ‍

To date, Chainguard has helped sponsor the research and development behind the OpenVEX project (now hosted in the Vulnerabilities Disclosure working group of the OpenSSF) including the specification and the related tooling. The OpenVEX specification reflects community-defined requirements and best practices. And although the specification is young, it has already had its first major revision. Additionally, there are libraries that perform common tasks required for adoption. Scanners and other VEX processors can (or will soon) rely on these libraries to not only read and write documents, but to match vulnerabilities and software identifiers, find relevant VEX documents, establish a chain of trust and more.‍

To help end users, the project actively maintains vexctl, which allows for easy document creation, manipulation and consumption. vexctl also enables signing of documents and will eventually handle the verification of the trust model.‍

Chainguard is also working with strategic partners to speed up adoption. Recent examples of collaboration include changes to support OpenVEX in Aqua Security’s Trivy scanner and cooperation with Anchore’s Grype team to integrate OpenVEX.‍

So if you’ve ever wondered if there is a better way to manage vulnerabilities than “edit this and email it back to me,” welcome to OpenVEX. Humans that want better vulnerability data should start here.‍

If you or your organization is interested in adopting OpenVEX, you can learn more about OpenVEX, learn about the specification and tools, or join the bi-weekly OpenVEX meeting.‍

Container users who want to avoid drowning in scanner findings, including false positives, can check out the zero-CVE SLA container images offered by Chainguard.‍‍

Users interested in a hardened eBPF-powered solution for networking, observability, and security can check out Isovalent Cilium for Enterprise.