To understand Apache Metron, we have to first start with the origins of the project which emerged from the Cisco Project called OpenSoc. The below diagram highlights some of the key events in the history of Apache Metron starting with Cisco OpenSoc.
- 2005 to 2008 The Problem - Cyber crime spiked significantly and a severe shortage of security talent arose. The first set of companies alerted to this issue are high profile banks and large organizations with interesting proprietary information to state sponsored agents. All of the best investigators and analysts were gobbled up by multinational banking and financial services firms, large hospitals, telcos, and defense contractors. The Rise of a New Industry, the Managed SOC - Those who could not acquire security talent were still in need of a team. Cisco was sitting on a gold mine of security talent that they had accumulated over the years. Utilizing this talent, they produced a managed service offering around managed security operations centers.
- Post 2008 The Age of Big Data Changed Everything - The Age of Big Data arrived, bringing more streaming data, virtualized infrastructure, data centers emitting machine exhaust from VMs, and Bring Your Own Device programs. The amount of data exploded and so did the cost of the required tools like traditional SIEMs. These tools became cost prohibitive as they changed to data driven licensing structures. Cisco’s ability to operate the managed SOC with these tools was in jeopardy and security appliance vendors took control of the market.
- 2013 OpenSOC is Born and Hadoop Matures - Cisco decided to build a toolset of their own. They didn’t just want to replace these tools but they wanted to improve and modernize them, taking advantage of open source. Cisco released its managed SOC service to the community as Hadoop matured and Storm became available. It was a perfect combination of a use case need and technology. OpenSOC was the first project to take advantage of Storm, Hadoop, and Kafka, as well as migrate the legacy ways into a forward thinking future type paradigm.
- September 2013 thru April 2015The Origins of Apache Metron - For about 24 months, a Cisco team, led by their chief data scientist James Sirota, with the help of a Hortonworks team, led by platform architect Sheetal Dolas, worked to create a next generation managed SOC service built on top of open source big data technologies. The Cisco OpenSOC managed SOC offering went into production for a number of customers in April of 2015. A short time after, Cisco made a couple of acquisitions that brought in third party technologies transforming OpenSOC into a closed source, hardware based version.
- December 2015 Metron Accepted into Apache Incubation - Metron (renamed from OpenSOC) is submitted and accepted as an Apache incubator project. The Metron team builds an extensible, open architecture to account for the variety of tools used in customer environments (thousands of firewalls, thousands of domains and a multitude of Intrusion Detection Systems). Metron’s open approach makes it much easier to tailor to the community’s use cases.
- April 2016 First official Release of Apache Metron 0.1 - After 4 months of hard work and rapid innovation by the Metron community, Apache Metron’s first release Metron 0.1 is cut.