This page adds to the over all guide How-To Define Stacks and Services and focuses on various aspects of adding a config property.
A config property is generally added to a Service and is associated with a config type.
For example,
| <property><name>zookeeper.znode.parent</name><value>/hbase-unsecure</value></property> |
zookeeper.znode.parent is a config property associated with hbase-site in HBase.
While adding config properties there are a few scenarios to keep in mind. Go through them to see what is applicable for the specific set of config changes you are doing.
Where should the property be added?
Identify the service version that the property is valid for. Ensure that the version is already defined in common-services. If not, then add a new version to the common-service. Do not add config properties directly in the stack versions unless the config property is only valid for a specific stack (e.g. HDP-2.4)
E.g https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml#L27-L32 added to KAFKA/0.9.0 and not KAFKA/0.8.1. KAFKA in HDP-2.3 derives from KAFKA/0.9.0 https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/metainfo.xml#L24
If the property is no longer supported starting a Service version then ensure that the property is deleted starting that specific Service version. E.g. https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml#L226-L230
Note: It is not standard to delete config properties from a released stack definition. If you absolutely must then ensure that there is corresponding Ambari upgrade code to remove the config property as well.
What should the property value be
Within the stack definition only a default value can be provided. There are several choices:
- The default value may be fine (very typical)
- The default value may be modified for specific deployments using stack advisor (e.g. default is not optimal) - E.g. https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py#L80 oozie_admin_users should include the falcon service user if falcon exists.
- The default value must be modified (typically properties with localhost or host list) - E.g. https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py#L496 ATLAS rest address, atlas.rest.address, must be constructed based on where the ATLAS server is installed.
What should happen on Ambari upgrade
The typical behavior is that Ambari upgrade must not modify stack configurations. It is critical for deployed stack to not take an outage (e.g. restart services) after performing an Ambari upgrade. It is ensured by specifying <on-ambari-upgrade add="false"/> to the config property. Due to historical reasons this is not the default behavior. Earlier versions of Ambari added any missing properties on upgrade. So several existing python scripts assume the existence of the properties.
So unless absolutely required, do not specify a config property to be added by default during Ambari upgrade.
If its needed that a property must be added/updated during Ambari upgrade, there are two choices:
- Specify <on-ambari-upgrade add="true"/> and if the property does not exist in the configurations, it will be added with the default value specified
- Explicitly add config property through upgrade catalog. This is needed if appropriate value must be calculated based on the cluster details. E.g. https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java#L2928 set HBase, hbase.coprocessor.master.classes, to appropriate values based on if Kerberos/Ranger is enabled.
Note: As Ambari upgrade does not add new config properties by default ensure that you have appropriate defaults while reading the configs. E.g. https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py#L81
What should happen on Stack upgrade
In contrast, Stack upgrade is a good time to update configs as needed. During stack upgrade all services go through restarts so new config values can easily take effect. By default, stack upgrade will add all new config properties that are missing in the stored configs. In addition, conditional update or deletion of config properties can be achieved through custom declarartive upgrade packs. E.g. https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml#L97 deletes a property, https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml#L120 modifies the value.
Display characteristics of the config property
See - Enhanced Configs
Misc best-practices
- Prefer setting config property values using stack-advisor (its common across all deployment mechanism, UI, blueprint)
- Avoid inclusion of new config properties or modification of config properties in the configure() implantation of the component. This is not visible to the user and thus results in configs on disk not being the same as the config accessible via API/UI
- Do add description for all config properties
- Add dependencies to other config properties as needed. Dependencies are required to ensure that stack advisor is called when dependencies change in value or are deleted (e.g. service delete)