Context for this meeting: White House national security adviser asks software companies to discuss cybersecurity.
The content below was developed on the security-discuss@community.apache.org mailing list, and perusing the archives there from late December 2021 to early January 2022 may provide important context. Everyone is invited to participate there, but be aware that posts to that mailing list are publicly archived.The best place to get started to know what the ASF is is here: https://apache.org/. That page is full of statistics, videos, and links.
The following links in particular are important background to this discussion:
- https://www.apache.org/foundation/how-it-works.html
- https://www.apache.org/dev/
- https://www.apache.org/dev/pmc.html
- https://apache.org/security/
- https://apache.org/theapacheway/index.html
- https://www.apache.org/foundation/getinvolved.html
Key bullet points
- The entire "supply chain" needs to be considered:
Software developed at the ASF is made available at no cost and without warranty
Commercial products may include this software without entering into any form of contract with the ASF, or even notifying us
- End users may purchase these products but have little interest or ability to apply fixes
- This will require collective action:
- There are things we can do, both individually and together, to reduce the number of vulnerabilities.
- There are things, such as SBOMs, that can help identify what is affected once a vulnerability is found.
- Much of this is moot if patches are never applied.
- On the topic of volunteers/community/participation:
- Our contributors tend to be seasoned software professionals whose employers include ASF releases in their commercial products.
- Our communities are healthy, open, and transparent.
- As a result of vulnerability reports, more attention is focused on the specific product, at least for a period of time. Open source enables a more diverse group of people to participate in these efforts.
- Companies and government agencies that want to help don't need money or formal contracts to do so. Join our mailing lists, review our code, contribute fixes.
Background reading:
- EO - Mark Cox - https://s.apache.org/3nctr
- SBOM - David Nalley - https://s.apache.org/hccur
- Applying updates - Mark Thomas - https://s.apache.org/5jqab
- Collective action - Phil Steitz - https://s.apache.org/ljzn0
- Volunteers - Sam Ruby - https://s.apache.org/3vkpr
- Contributors/maintenance - Dominik Psenner - https://s.apache.org/3lrk1
- CISA - David Nalley - https://s.apache.org/1gr1c
- White box testing - John Kinsella - https://s.apache.org/hn2ba
- 2016 EU Audit - Dirk-Willem van Gulik - https://s.apache.org/fhoji
There are really three main supply-chain related ideas that people need to get:
- While we may not all agree at this point that it works, our biggest defense against vulnerabilities getting introduced is eyeballs, which our open development process supports. Any interested human (or bot) can be a QC inspector in our direct contribution part of the supply chain. We have always welcomed this, especially when reports come with patches.
- There is a natural nesting that happens as software dependencies propagate through applications. Addressing vulnerabilities in base level components (e.g. log4j) has a cascading impact. Unless and until all downstream systems have effectively automated build, test and deployment systems, this creates systemic risk which has nothing to do with OSS per se.
- End of life is an industry standard concept that we apply at the ASF. End of life means no more patches. End of life software running in critical systems is another systemic risk.
Annotated notes and links
- Foundation organization and governance structure (https://www.apache.org/foundation/governance/)
The ASF is an operating US 501(c)3 organization that supports several hundred open source projects. The operating part of the ASF is managed by the President, supported by the Treasurer, Vice Presidents, committees and third parties. The ASF board provides oversight for operating functions as well as ASF project communities. The Legal Affairs, Security, and Treasurer departments report directly to the board, which means, for example that the Security team acts with the authority of the board.
- How oversight works (https://www.apache.org/foundation/how-it-works.html)
...
...
Invitation
On December 23rd the "Chairman, ASF" received an invitation from the White House Deputy National Security Advisor for Cyber & Emerging Technology for a one-day discussion, on January 13, 2022, to join a White House initiative to improve open source software security. More meeting details: White House national security adviser asks software companies to discuss cybersecurity.
Pre-Call
Mark Cox, David Nalley, Sam Ruby (and optionally Roy Fielding, Mark Thomas) are having a pre-call with NSC during the week starting 3rd January. Read-ahead material will be supplied based on this page: Read Ahead Material, pre-call, Apache Software Foundation (ASF):
Jan 13 Meeting
Meeting is in-person, Washington USA, and will be attended by David Nalley, Sam Ruby
- How development works (https://www.apache.org/dev/pmc.html)
ASF projects don’t all work the same and the ASF does not prescribe any kind of SDLC for our projects. What we do require is that everything happens in the open:
- All bug reports are visible to all interested parties (with exception of reported security vulnerabilities, which are kept private until patched)
- All commits to project source code are tracked and visible to all interested parties
- All project-related discussion that can be public is public
ASF projects differ in how they do things, but most have documented process for working bug reports, introducing new features and planning and executing releases. As new features are developed and bugs are addressed, code changes are made that will go into releases. Typically, proposed changes are submitted as “pull requests” (PRs) that explicitly describe how the source will change if they are applied. The original source, the PR and any changes that result from applying it are all publicly visible. Any interested party can comment on proposed changes and it frequently happens that performance, security or other issues are discovered in the open development process that creates and modifies ASF software.
- How releases are versioned, validated and distributed (https://www.apache.org/dev/#releases)
ASF software is distributed and meant to be consumed in the form of versioned releases. ASF PMCs have various ways of packaging software for distribution, but the core asset being released is always source code. Releases must be voted on by PMCs, who are responsible for validating release candidates (RCs). Most PMCs also make RCs available to the public prior to or during release votes. Library or infrastructure projects sometimes make backward-incompatible changes as they develop new versions of their software. Sometimes, a backward-compatible version of the software is added and maintained in parallel, but it often happens that the actively supported version is not backward compatible with an older version. When library or infrastructure components make backward incompatible changes, downstream users may have to make code changes to upgrade their applications.
- How ASF code is incorporated in other software
ASF software is used in running systems in basically 2 ways:
- A distributed system includes a running ASF product (e.g. Apache web server)
- ASF software is integrated with other software to build applications. When applications created in 2. are themselves used to create other applications, nesting happens, causing the impact of a vulnerability or incompatible change at the lowest level to have a cascading impact. When a new release is made available by an ASF project, downstream users need to minimally make configuration changes and test their systems. In case 2. above they typically have to rebuild and redeploy their applications. In nested situations, this often has to happen sequentially.
- How security reports are handled (https://www.apache.org/security/)
The ASF encourages responsible disclosure of security vulnerabilities discovered in software managed by ASF projects. The ASF security team sets a common policy, maintains security contacts for PMCs, and provides support for projects responding to security issues. Reporters are encouraged to use the designated security contacts to report vulnerabilities privately. PMCs are required to respond to security reports promptly, working with reporters to investigate and if necessary develop patches. The ASF Security team assist the PMC in publishing a CVE once the vulnerability has been patched and a release containing the patch has been made available. The security team oversee all reported issues across all ASF projects.
- How end of support works
...
.
...