...
You don't fix supply chain issues by focusing exclusively on the means of production.
If we could somehow make perfect releases of all all ASF projects tomorrow, it would literally take years before we would see the effects. And plenty of cases where the update would never be applied.
Community is defined by those who show up.
We have plenty of components which are widely used, yet many of those who use them either don't review the code or don't chose not to contribute back their fixes.
Vulnerabilities aren't always isolated to a single component.
In the case of the recent log4j vulnerability, JNDI (a part of the Java runtime library) is a critical part of the exploit. We tend to treat concepts of "JNDI injection" as if it were a known problem, but just like "SQL injection" and other injection problems, this is demonstrably not the case. If it were possible to improve those common components, that could be a very big win.
...
- ASF projects are run by Project Management Committees (PMCs). PMC members nominate and vote on new PMC members and collectively manage community, security, trademarks, or other problems.
- Projects to have to adhere to various foundation-wide policies including trademarks, release policy, security vulnerability handling policy. ASF policies require PMC reviews with minimum levels of voting required for new releases.
- All changes to project source code are tracked and visible to all interested parties.
- The ASF has a security team and a VP of security and who acts with board authority. The ASF security team sets a common policy, maintains security contacts for PMCs, and provides support for projects responding to security issues.
- When faced with a critical vulnerability we aim to get it mitigated fast. Once an issue is public and has increased attention, it sometimes leads to additional issues being detected or workarounds not being complete and we aim to be fast to release and fast to fix.
- We publish metadata about all vulnerabilities across all our projects in a consistent and machine-readable way.
...
Businesses using open source need to react faster. Log4J and HeartBleed are being used as examples of open source vulnerability risks but it must be remembered that once these issues were reported to their respective projects they were dealt with quickly and efficiently. An update correcting HeartBleed was released within a week of notification of the issue to OpenSSL and prior to any exploitation. An update mitigating the Log4J vulnerability was released two weeks after notification to the Apache Software Foundation (with the release escalated and published a day after it was found to be being exploited in the wild). Both of these vulnerabilities were also examples which required skilled human researchers to discover and were not detected by automated tools.
What caused these, and other vulnerabilities, such as the Apache Struts issue in 2017, to be widely exploited was a failure of businesses mitigate in a timely manner: either by updating to a new release or applying mitigations.
While part of the solution may be to ensure companies know what they’ve included in their supply chain, they will also need to have processes for rapidly handling and disclosing vulnerabilities in their dependencies.
Users of open source software also need to keep track of lifecycles and ensure the projects they are using are still getting security updates.
- One of the most valuable things businesses that use open source can do is contribute back. Help fix bugs. Conduct security audits and feed back the results. Cash, while welcome and useful, isn't sufficient. We eagerly welcome audits and fixes from any source. We have a process defined for doing so: https://www.apache.org/security/. Our one caution is that we have prior experiences with audits and it is not productive when those audits produce a number of false positives. See Feedback from The Apache Software Foundation on the Free and Open Source Security Audit (FOSSA). It is worth noting that many automated means of performing an audit are more suited to detecting bugs inside a single component rather than identifying a vulnerability that is composed of combination of intentional features such as the one we saw with log4j. It is better when these audits are performed by skilled resources who are able to accurately identify actual issues, produce fixes, and therefore reduce the burden on each project.
Open Source projects should prioritise publishing security metadata quickly after a critical security issue is public. Many end users and tool vendors look to information published in the MITRE CVE database and in the downstream NIST NVD database. During 2021 MITRE have given CVE naming authorities (such as the ASF) the ability to publish information rapidly. Future changes are being made to that format and process in 2022 which evolve and enhance that capability.
For example for Log4J, the CVE was live on cve.org with full machine-readable metadata including affected versions in under 30 minutes from release.
For projects outside of the ASF, initiatives from platforms such as GitHub give projects the ability to obtain CVE names and publish vulnerability metadata. Other Open source projects which are deemed critical should work with the CVE project to become CVE naming authorities, allowing them to provide rapid publication and consumable updates in machine readable formats.
- We support the notion of identifying critical components. As our software is without cost and can be downloaded by anybody, we have no way of knowing how widely our projects are used. This means that by necessity determining if a component is critical can only either be determined by automated means which is always going to be incomplete and imperfect, or more manually such as by identifying critical products first, and then getting an inventory of what components those products embed. This problem is not unique to the ASF, so it makes sense for there to be industry standards and best practices. We are prepared to participate in these activities.
- As noted above, vulnerabilities can be a result of a how a number of components are combined.
The owner of a system is best placed to determine the criticality of that system and the degree to which individual components contribute to that criticality.
Criticality depends on usage. We need to avoid solutions that create burdens for all system owners using a software component when the component is only critical for some systems.
...