Configuration update involves the following steps:

  • Identify the config type to update and note the latest version applied. When a config type is updated the whole property set needs to be updated. So copying the values from the latest and updating specific values or adding/removing properties as needed is the easiest option.
  • Read the cluster resource and note the version of the type you want to update
  • Read the config type with the tag and note the properties
  • Edit the properties as needed and then update the config type
    • Config update requires creation of a new version (typically current timestamp is a good choice)
    • The new version of the config type must be added and applied to the cluster
  • Restart affected services/components to have the config take effect

You can use APIs or a wrapper script (/var/lib/ambari-server/resources/scripts/configs.py or /var/lib/ambari-server/resources/scripts/configs.sh) to edit configurations.


Edit configuration using APIs (2.4.0+)

Verified against releases 2.4.x (should work with 2.2.x)

1. Find the latest version of the config type that you need to update.

curl -u admin:admin -H "X-Requested-By: ambari" -X GET  http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs",
  "Clusters" : {
    "cluster_name" : "CLUSTER_NAME",
    "version" : "HDP-2.0.6",
    "desired_configs" : {
	  ...
      "mapred-site" : {
        "user" : "admin",
        "tag" : "version1384716039631"
      }
      ...
    }
  }
}

 

2. Read the config type with correct tag

 

curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631"

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
  "items" : [
    {
      "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
      "tag" : "version1384716039631",
      "type" : "mapred-site",
      "Config" : {
        "cluster_name" : "CLUSTER_NAME"
      },
      "properties" : {
      ... THESE ARE THE PROPERTY KEY-VALUE PAIRS ...
      }
    }]
}

 

3 Save a new version of the config and apply it as desired config

 Ensure that a unique tag is provided - best option is to use current timestamp

 

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '[{"Clusters":{
  "desired_config":[{
    "type":"zoo.cfg",
    "tag":"version1480557385509",
    "properties":{
      "autopurge.purgeInterval":"24",
      "autopurge.snapRetainCount":"30",
      "dataDir":"/hadoop/zookeeper",
      "tickTime":"2000",
      "initLimit":"11",
      "syncLimit":"5",
      "clientPort":"2181"},
    "service_config_version_note":"New config version"}]}}]' 
"http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME"

 

4. Restart all components or services to have the config change take effect

E.g. Stop and Start a service

 

curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Stop HDFS"}, "ServiceInfo": {"state": "INSTALLED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS
curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Start HDFS"}, "ServiceInfo": {"state": "STARTED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS


Edit configuration using configs.py

configs.py is a helper script available with ambari-server installation that makes it easy to manipulate configuration. When using the script it will edit the configuration and apply it to cluster. You still need to restart services/components for the config to take effect.

configs.sh usage 

/var/lib/ambari-server/resources/scripts/configs.py --help
Usage: configs.py [options]
Options:
  -h, --help            show this help message and exit
  -t PORT, --port=PORT  Optional port number for Ambari server. Default is
                        '8080'. Provide empty string to not use port.
  -s PROTOCOL, --protocol=PROTOCOL
                        Optional support of SSL. Default protocol is 'http'
  -a ACTION, --action=ACTION
                        Script action: <get>, <set>, <delete>
  -l HOST, --host=HOST  Server external host name
  -n CLUSTER, --cluster=CLUSTER
                        Name given to cluster. Ex: 'c1'
  -c CONFIG_TYPE, --config-type=CONFIG_TYPE
                        One of the various configuration types in Ambari. Ex:
                        core-site, hdfs-site, mapred-queue-acls, etc.
  To specify credentials please use "-e" OR "-u" and "-p'":
    -u USER, --user=USER
                        Optional user ID to use for authentication. Default is
                        'admin'
    -p PASSWORD, --password=PASSWORD
                        Optional password to use for authentication. Default
                        is 'admin'
    -e CREDENTIALS_FILE, --credentials-file=CREDENTIALS_FILE
                        Optional file with user credentials separated by new
                        line.
  To specify property(s) please use "-f" OR "-k" and "-v'":
    -f FILE, --file=FILE
                        File where entire configurations are saved to, or read
                        from. Supported extensions (.xml, .json>)
    -k KEY, --key=KEY   Key that has to be set or deleted. Not necessary for
                        'get' action.
    -v VALUE, --value=VALUE
                        Optional value to be set. Not necessary for 'get' or
                        'delete' actions.

 

Edit configuration using APIs (1.4.1/1.2.5)

Verified against releases 1.4.1/1.2.5

Starting 1.4.2/1.4.3 you will have to add -H option to the curl calls. E.g. -H "X-Requested-By: ambari"

1. Find the latest version of the config type that you need to update.

curl -u admin:admin -X GET  http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs",
  "Clusters" : {
    "cluster_name" : "CLUSTER_NAME",
    "version" : "HDP-2.0.6",
    "desired_configs" : {
	  ...
      "mapred-site" : {
        "user" : "admin",
        "tag" : "version1384716039631"
      }
      ...
    }
  }
}

2. Read the config type with correct tag

curl -u admin:admin "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631"

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
  "items" : [
    {
      "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
      "tag" : "version1384716039631",
      "type" : "mapred-site",
      "Config" : {
        "cluster_name" : "CLUSTER_NAME"
      },
      "properties" : {
      ... THESE ARE THE PROPERTY KEY-VALUE PAIRS ...
      }
    }]
}

3a. Save a new version of the config and apply it (see 3b for doing it using one call)

curl --user admin:admin -i -X POST -d '{"type": "mapred-site", "tag": "version1384716041120", "properties" : {"mapreduce.admin.map.child.java.opts" : "-Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN",...}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations
curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120"}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

3b. Save a new version of the config and apply it using one call

curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120", "properties" : {...}}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

4. Restart all components or services to have the config change take effect
E.g. Stop and Start a service

curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Stop HDFS"}, "ServiceInfo": {"state": "INSTALLED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS
curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Start HDFS"}, "ServiceInfo": {"state": "STARTED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS

Edit configuration using configs.sh

configs.sh is a helper script available with ambari-server installation that makes it easy to manipulate configuration. When using the script it will edit the configuration and apply it to cluster. You still need to restart services/components for the config to take effect.

1. Update a parameter

/var/lib/ambari-server/resources/scripts/configs.sh set localhost c1 mapred-site "mapreduce.map.memory.mb" "512"

2. Read updated config

/var/lib/ambari-server/resources/scripts/configs.sh get localhost c1 mapred-site

Hint: Use export to file (option CONFIG_KEY) and import from file to edit large number of properties.
3. configs.sh usage
Note: If you have an older version then it may not have a way to provide custom userId and password. Edit the script to use the correct password if default admin:admin is not applicable.

/var/lib/ambari-server/resources/scripts/configs.sh
Usage: configs.sh [-u userId] [-p password] [-port port] <ACTION> <AMBARI_HOST> <CLUSTER_NAME> <CONFIG_TYPE> [CONFIG_FILENAME | CONFIG_KEY [CONFIG_VALUE]]

       [-u userId]: Optional user ID to use for authentication. Default is 'admin'.
       [-p password]: Optional password to use for authentication. Default is 'admin'.
       [-port port]: Optional port number for Ambari server. Default is '8080'. Provide empty string to not use port.
       <ACTION>: One of 'get', 'set', 'delete'. 'Set' adds/updates as necessary.
       <AMBARI_HOST>: Server external host name
       <CLUSTER_NAME>: Name given to cluster. Ex: 'c1'
       <CONFIG_TYPE>: One of the various configuration types in Ambari. Ex:global, core-site, hdfs-site, mapred-queue-acls, etc.
       [CONFIG_FILENAME]: File where entire configurations are saved to, or read from. Only applicable to 'get' and 'set' actions
       [CONFIG_KEY]: Key that has to be set or deleted. Not necessary for 'get' action.
       [CONFIG_VALUE]: Optional value to be set. Not necessary for 'get' or 'delete' actions.
  • No labels