...
- HBase - HBase is the Hadoop database. Its an open-source, distributed, column-oriented store modeled after the Google paper, Bigtable: A Distributed Storage System for Structured Data by Chang et al.
We use ZooKeeper for master election, server lease management, bootstrapping, and coordination between servers.
- Hedwig Hedwig is a publish-subscribe system designed to carry large amounts of data across the internet in a guaranteed-delivery fashion from those who produce it (publishers) to those who are interested in it (subscribers).
- Kafka Kafka is a distributed publish/subscribe messaging system
- Norbert Partitioned routing and cluster management.
- Mesos Mesos is a cluster management platform that provides resource sharing and isolation across distributed applications, or frameworks. For example, Hadoop MapReduce, MPI, Hypertable, and Spark
- Neo4j - Neo4j is a Graph Database. It's a disk based, ACID compliant transactional storage engine for big graphs and fast graph traversals, using external indicies like Lucene/Solr for global searches.
We use ZooKeeper in the Neo4j High Availability components for write-master election, read slave coordination and other cool stuff. ZooKeeper is a great and focused project - we like!
- Rackspace - The Email & Apps team uses ZooKeeper to coordinate sharding and responsibility changes in a distributed e-mail client that pulls and indexes data for search. ZooKeeper also provides distributed locking for connections to prevent a cluster from overwhelming servers.
- S4 S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.
- Yahoo! - ZooKeeper is used for a myriad of services inside Yahoo! for doing leader election, configuration management, sharding, locking, group membership etc.
...