Initial Analysis of Hadoop FileSystem 1.0 Tests by the HFCS working group.
The term "Impl Test" used below is intended to assert that the test validates a single particular Hadoop FileSystem Implementation and should not belong with a Hadoop FS Test Library that focuses on any FileSystem Plugins ability to implement the Hadoop FileSystem contract.
|
Test |
What it does |
Runtime Test (main) |
no FS |
TestDU.java |
Validates DU.java works properly |
|
no FS |
TestGlobPattern.java |
Validates Glob Pattern Matching Support |
|
|
DFSCIOTest.java |
Why do we have this AND TestDFSIO ? |
x |
no, but references org.apache.hadoop.hdfs.MiniDFSCluster |
TestDisableCache.java |
Validates the ability to turn FS caching on and off |
|
no, but references org.apache.hadoop.hdfs.MiniDFSCluster |
TestHarFileSystem.java |
Validates Hadoop Archives (HAR) Support |
|
|
DistributedFSCheck.java |
Validates FS Consistency |
x |
|
FSMainOperationsBaseTest.java |
Validates FS Operations. Needs to be disambiguated from FileSystemContractBaseTest |
|
no, but references org.apache.hadoop.hdfs.MiniDFSCluster |
TestFileSystem.java |
Validates Reads, Seeks and Writes via a M/R Job. Needs to be disambiguated from DFSCIOTest and TestDFSIO |
x |
|
FileSystemContractBaseTest.java |
Validates FS Impl adherence to FS Contract |
|
|
FileSystemTestHelper.java |
Helper/Utility Class for FS Testing |
|
no, but references and configured with RawLocalFileSystem. |
TestFsShellReturnCode.java |
Validates that Hadoop FS chmod, chown, chgrp commands work correctly |
|
no FS |
TestPath.java |
I think this is a Helper class for testing Paths. Seems to be a mix of local and (perhaps) FS Tests |
|
|
TestGetFileBlockLocations.java |
Validates the ability to determine Block/File Locations |
|
|
TestTrash.java |
Validates FS support of Trash Operations |
|
no, but references org.apache.hadoop.hdfs.MiniDFSCluster |
TestCopyFiles.java |
Test Copy Operations but needs to be disambiguated from FileSystemContractBaseTest and other FS Test Classes |
|
|
TestTruncatedInputBug.java |
Test for HADOOP-1489. Needs to be changed from Explicit Local to FS |
|
|
TestDFSIO.java |
Validate Read and Write Throughput |
x |
no FS |
TestGlobPaths.java |
Tests Glob Paths. Needs to be disambiguated from other Glob Tests |
|
no but depends on org.apache.hadoop.hdfs.MiniDFSCluster |
TestUrlStreamHandler.java |
High level validation of FS Input Streams |
|
x |
AccumulatingReducer.java |
Reducer that accumulates values based on their type |
|
x |
TestLocalDirAllocator.java |
Does not use FS. This is a LocalFileSystem test. |
|
x |
TestFileSystemCaching.java |
Does not use FS. This is a LocalFileSystem test. |
|
x |
TestLocalFileSystem.java |
Does not use FS. This is a LocalFileSystem test. |
|
x |
TestFileUtil.java |
Does not use FS. This is a LocalFileSystem test |
|
x |
TestLocalFileSystemPermission.java |
Does not use FS. This is a LocalFileSystem test |
|
x |
IOMapperBase.java |
Not sure where this is used. Seems redundant |
|
x |
TestChecksumFileSystem.java |
Does not use FS. This is a ChecksumFileSystem InMemoryFileSystem test |
|
x |
TestGlobExpander.java |
Helper class for Glob Operations. Not clear who uses it |
|
x |
TestHardLink.java |
Does not use FS. We should remove. Validates Single and Multi File Hard linking |
|
x |
TestFSMainOperationsLocalFileSystem.java |
Does Nothing. Recommend that we remove this out of the FS Library |
|
Hadoop FS Test sub-packages
|
Test |
What it does |
|
permission |
Validates the Hadoop FS Plugin adherence to Permissions |
|
slive |
|
|
loadGenerator |
A load testing tool for testing Hadoop FS behavior under different mixes of read, write and list requests |
x |
ftp |
Validates the Hadoop FS Plugin for an FTP Server |
x |
kfs |
Validates the Hadoop FS Plugin for the Kosmos FS. Now defunct due to the Quantcast FS Plugin |
x |
s3 |
Validates the Hadoop FS Plugin for the Amazon S3 Object Store |
x |
s3native |
Pseudo S3 Implementation to allow testing of the Hadoop FS Plugin for S3 without connecting to Amazon S3 |