...
- The filesystem looks like a "native" filesystem, and is accessed as a local FS, perhaps with some filesystem-specific means of telling the MapReduce layer which TaskTracker is closest to the data.
- The filesystem provides an implementation of the
org.apache.hadoop.fs.FileSystem
class (and in Hadoop v2, in implementation of theFileContext
class}
Implementing the FileSystem
class ensures that there is an API for applications such as MapReduce, Apache HBase, Apache Giraph and others can use -including third-party applications as well as code running in a MapReduce job that wishes to read or write data.
...
Other known members of the HCFS community:
- /* Alphabetical order, no endorsements, please */. */
- Windows Azure Blob Storage
- CassandraFS
- CephFS
- CleverSafe Object Store
- Google Cloud Storage Connector
- Lustre
- MapR FileSystem
- Quantcast File System
- Symtantec Veritas Cluster File System
- Apache Ozone
Even if the filesystem is supported by a library for tight integration with Apache Hadoop, it may behave differently from what Hadoop and applications expect: this is something to explore with the supplier of the filesystem.
...