HDDS-1469 introduced a new way to use the classic hadoop configuration. Instead of create java constans and xml fragments which are aligned, it's enough to create a well-annotated Java class.
How to use it?
(1) Create a configuration pojo
First of all, you need a class which can hold the configuration values.
- Annotate the class with @ConfigGroup
- Annotate the setters with @Configuration
Note: the final confiuration key will be the concatenation of @ConfigGroup.prefix and the @Config.key strings.
(2) Use the configuration class instead of OzoneConfiguration
You can retrieve your configuration class which is filled with the real configuration values (or defaults) with the help of good old OzoneConfiguration.
Note: It's a good idea to use the more type-safe, meaningful configuration instances instead of the OzoneConfiguration. It helps to understand what variables are used and you don't need to create a real OzoneConfiguration for a real unit test.
How does it work?
The implementation based on two parts:
- An annotation processor generates the ozone-default.xml fragments which was created manually earlier (see hadoop-ozone/configuration project, especially the ConfigFileGenerator.java)
- OzoneConfiguration is extended with a method which can create the object and set the values based on the current configuration values.
From these points the second one is more obvious (see the source OzoneConfiguraiton.getObject) in can be done with some lightweight reflection magic.
The first part is harder. As we may have annotated classes in all of the projects we have no one place to create the ozone-default.xml from all the fragments. Instead of this we generate ozone-default-generated.xml files from the annotation processor (ConfigFileGenerator class).
We need the merge the ozone-default-generated.xml to to the existing ozone-site.xml to get the results what we have. Unfortunately it's not an easy task. The fragments files are generated in all the projects, so we can combine them only at the end of the build process (maybe in the dist project), but we also need the final ozone-default.xml for all the unit tests in the small projects where the combined files are not yet available.
As creating one huge file is hard we started to use an other approach: We improved the OzoneConfiguration to read all the ozone-default-generated.xml files (see the constructor) which are available on the classpath. With this approach we can't have one generated file. But we can generate one combined descriptor/documentation for the docs (where we have no circular dependency as the generated docs/descriptor is not used for unit tests).