How can one measure and mitigate the security risks in open source software? This is a crucial topic that is now becoming a major issue in the software development community. In this chapter, we will dive into the work we have done at Google, in collaboration with Open Source Security Foundation (OSSF) to make it easy for both Open Source developers and consumers alike. Make no mistake — attacks on Open Source Software (OSS) are on a constant rise. Open source supply chain attacks grew 650 percent last year – a staggering number.
During 2021 and into 2022, every month there has been a significant attack which impacted more than 1000 organizations. The most prominent recent OSS attacks happened as a result of the Log4Shell vulnerability disclosed in December 2021. Attacks on the OSS supply chain and on OSS impacted every single software organization on the planet. Everybody is scrambling to continuously patch systems as new vulnerabilities continue pouring in.
To properly navigate this environment and keep our systems safe, we as a community need to understand the security risks of OSS and be better prepared to handle them, collectively and individually. This chapter discusses how to mitigate risks and, more importantly, how to think about OSS.
Open Source Risks
There are a number of risks that make it more dangerous to consume and trust OSS without taking proper security measures.
OSS Is Free. OSS Security Is Expensive.
As a starting point, we must recognize that open source software may be free to download and require no payment, but it comes with a massive security cost.
A significant portion of the open source software is developed by volunteers in their free time. Most of the prominent open source volunteers actually have full-time jobs, and the code they contribute to projects like the Log4J logging libraries they write on nights and weekends, on their own time and not on the company dime. Not surprisingly, security often comes last in their priority list. They don’t have the resources to invest in a secure development lifecycle or have the time and bandwidth to maintain their code. This may mean that security vulnerabilities remain unaddressed for long periods of time, sometimes indefinitely.
All Bugs Are Shallow. But There Aren’t Enough Eyeballs.
In software development, Linus’s law states “…given enough eyeballs, all bugs are shallow”. This law is only true if those eyes are looking in the right place. In the case of open source, all code is open but we don’t have enough eyes paying attention to it to spot all the bugs. In addition, organizations don’t realize the security consequences of their decision before consuming a particular piece of open source software and adding it as a dependency. Dependency trees become more and more complex, and harder and harder to monitor and secure, as you keep adding more third party components to your core codebase. Each new dependency might bring additional nested dependencies — perhaps hundreds or thousands, depending on the size of the library added or the role it plays in the application architecture. For example, the Kubernetes Project — one of the most successful and widely adopted OSS projects — has several thousand dependencies, many of which have additional secondary and tertiary downstream dependencies.
So Much Code. So Many Choices. Not All Are Good.
The beauty of the Open Source ecosystem is that there is a solution for everything. The problem is, you don’t have one solution, but several dozen of them. For example, if you need a JSON parser for your application, there are several of them to choose from. If your application is running a critical service or infrastructure, then picking the right JSON parser — one that is secure and comes from a responsible project or maintainer — becomes very important. A JSON parser will serve as a critical component for your application, so it will impact the security of your entire system. If you chose the wrong JSON parser, you might pay a big price down the road in security risk and subsequent breaches.
Proposal: An Entry Gate Security Risk Assessment
One potential solution to enhance OSS code security is the creation and implementation of a security risk assessment gate as part of your OSS ingestion pipeline. Putting this process in place will help you understand whether consuming a certain piece of software is high risk and avoid software that does not meet your risk criteria.
Here is an example of how this might work. Consider five different JSON parser libraries in GitHub, all of which have a good number of stars and appear to be highly rated. How can we assess the risk of these libraries and eliminate ones that are too risky quickly and easily? One of these JSON libraries is hosted under a user account. Most of the commits are done by a single developer without reviews. This alone indicates that using this library in a critical system presents high risk. This lone developer could commit malicious code without review. The maintainer’s token access could be leaked, allowing attackers to insert malicious code. The developer might also turn over committer control of their library to a malicious actor without notifying users (who might not pay attention to the notification anyway). Identifying this risk required no more than a few minutes of review of the this repository and cost little in time or effort.
A Broader View of Security Posture for OSS
More broadly, we need to understand the full security posture of an organization or developer and their open source code prior to adding it to critical systems. Because OSS is a complex ecosystem and many pieces of OSS are complex, the risks they generate are similarly complex. Our traditional, myopic view of these risks is that we just need to find all the security vulnerabilities in our codebase and fix them. This does not resolve all risks and may lull us into a false sense of security.
We need to understand the full comprehensive security posture of an open source project. That involves understanding a broad range of risks beyond code vulnerabilities. Here are some of the key risks to consider beyond ensuring that your codebase is well secured and patched.
Supply Chain Attacks on the Build Pipeline
Attacks like SolarWinds and CodeCove have taught us that supply chain risk is a very big deal and that our tools are as much of a risk as our code. Attackers have found it much easier and much more reliable to attack our build pipelines, rather than finding vulnerabilities in our code.
Security Risks Revealed by Continuous Security Testing
Continuous security testing is critical to maintaining the security of your code base. You might not have any code vulnerabilities today, but as developers continuously keep modifying your code and iterate on applications, vulnerabilities may be introduced. This can mean testing code as part of the CI/CD process with every new code push. For example, we have found numerous critical bugs in the OpenSSL libraries. All were caught on a master or main branch, and never made it to a production release.
Project Maintenance and Capabilities
One of the really big risks with open source is the quality and frequency of project maintenance. Issues raised may not be addressed in a timely fashion. Maintainers of a project may have a long backlog of issues, indicating that they are not actively maintaining a project. It is not uncommon for critical open source projects that are used in millions of applications and comprise a significant dependency to the global software supply chain have not been updated in a year or more. In addition, maintainers of OSS projects may not know how to respond to security incidents and fix security vulnerabilities. This is particularly true if the maintainers do not work at a large organization, do not have cybersecurity or secure development experience, or have never suffered through the toil and trauma of addressing a major vulnerability in their code on short notice.
Tools for Measuring and Addressing Supply Chain Risk
A number of efforts are underway to create free tools to help developers and application security teams better measure and address supply chain risk programmatically and analytically.
SLSA — Supply Chain Level for Software Artifacts.
As a tool to help organizations better address software supply chain risks we at Google created a novel framework that we call Supply Chain Level for Software Artifacts. The project has been underway at Google for more than a decade and has been open-sourced as part of the OSSF. Contributors include VMware, Citibank and DataDog, among others. We hope to standardize SLSA and make it an industry benchmark for evaluating supply chain risk. Right now SLSA is focused primarily on build process risks but we hope to expand it in the near future to other risks such as vulnerability and maintainer risks.
OSS Security Scorecards Project
In order to scale the process of assessing, identifying and mitigating risks in OSS, we need robust automation. As part of the OSSF we released in 2020 a new project called Open Source Security Scorecard. The project aims to automate the analysis and trust aspect when evaluating the security posture of open source code. Further, the Scorecards can provide specific security recommendations. The Scorecard process can incorporate many other automated tools, including checking against vulnerabilities using the Open Source Vulnerability (OSV) database and running dependency checks using open source dependency trackers like Dependabot. The scorecards project also makes it easier to report new vulnerabilities and get those to the right people in a project.
Conclusion: What Is Not Measured Is Not Mitigated
No one denies that OSS has become crucial to the vast majority of applications running the world’s technology fabric. The span of OSS continues to grow, as more and more languages and frameworks emerge, and specialization increases. No one wants to go back to the past of reinventing the wheel and rewriting fresh code to perform the same tasks with every new application. The model of reuse, dependency, and distributed software development works and speeds innovation. That said, the Open Source Way and the supply used to build OSS needs a more rigorous approach to reducing risk. Given the scope of the problem, automation and standardization are the only pat’s that can scale towards the stated goal. We are in early stages of creating the tools and infrastructure to automate the risk identification, measurement, assessment and mitigation processes. Security frameworks like SLSA and new processes like entry gating new OSS code with Security Scorecards, will head us down that path and move us towards a more secure open source future.
