This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

You can retrieve your configuration class which is filled with the real configuration values (or defaults) with the help of good old OzoneConfiguration.

Code Block
languagejava
confReplicationManagerConfiguration replicationManagerConf = ozoneConfiguration.getObject(ReplicationManagerConfiguration.class)

Note: It's a good idea to use the more type-safe, meaningful configuration instances instead of the OzoneConfiguration. It helps to understand what variables are used and you don't need to create a real OzoneConfiguration for a real unit test.

Code Block
languagejava
replicationManager = new ReplicationManager(
conf.getObject(ReplicationManagerConfiguration.class),
containerManager,
containerPlacementPolicy,
eventQueue,
new LockManager<>(conf));

...

  • An annotation processor generates the ozone-default.xml fragments which was created manually earlier (see hadoop-ozone/configuration project, especially the ConfigFileGenerator.java)
  • OzoneConfiguration is extended with a method which can create the object and set the values based on the current configuration values.

From these points the second one is more obvious (see the source OzoneConfiguraiton.getObject) in can be done with some lightweight reflection magic.

The first part is harder. As we may have annotated classes in all of the projects we have no one place to create the ozone-default.xml from all the fragments. Instead of this we generate ozone-default-generated.xml files from the annotation processor (ConfigFileGenerator class).

We need the merge the ozone-default-generated.xml to to the existing ozone-site.xml to get the results what we have. Unfortunately it's not an easy task. The fragments files are generated in all the projects, so we can combine them only at the end of the build process (maybe in the dist project), but we also need the final ozone-default.xml for all the unit tests in the small projects where the combined files are not yet available.

As creating one huge file is hard we started to use an other approach: We improved the OzoneConfiguration to read all the ozone-default-generated.xml files (see the constructor) which are available on the classpath. With this approach we can't have one generated file. But we can generate one combined descriptor/documentation for the docs (where we have no circular dependency as the generated docs/descriptor is not used for unit tests).