This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Skip to end of metadata
Go to start of metadata

Configuration update involves the following steps:

  • Identify the config type to update and note the latest version applied. When a config type is updated the whole property set needs to be updated. So copying the values from the latest and updating specific values or adding/removing properties as needed is the easiest option.
  • Read the cluster resource and note the version of the type you want to update
  • Read the config type with the tag and note the properties
  • Edit the properties as needed and then update the config type
    • Config update requires creation of a new version (typically current timestamp is a good choice)
    • The new version of the config type must be added and applied to the cluster
  • Restart affected services/components to have the config take effect

You can use APIs or a wrapper script (/var/lib/ambari-server/resources/scripts/configs.py or /var/lib/ambari-server/resources/scripts/configs.sh) to edit configurations.


Edit configuration using APIs (2.4.0+)

Verified against releases 2.4.x (should work with 2.2.x)

1. Find the latest version of the config type that you need to update.

curl -u admin:admin -H "X-Requested-By: ambari" -X GET  http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs",
  "Clusters" : {
    "cluster_name" : "CLUSTER_NAME",
    "version" : "HDP-2.0.6",
    "desired_configs" : {
	  ...
      "mapred-site" : {
        "user" : "admin",
        "tag" : "version1384716039631"
      }
      ...
    }
  }
}

 

2. Read the config type with correct tag

 

curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631"

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
  "items" : [
    {
      "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
      "tag" : "version1384716039631",
      "type" : "mapred-site",
      "Config" : {
        "cluster_name" : "CLUSTER_NAME"
      },
      "properties" : {
      ... THESE ARE THE PROPERTY KEY-VALUE PAIRS ...
      }
    }]
}

 

3 Save a new version of the config and apply it as desired config

 Ensure that a unique tag is provided - best option is to use current timestamp

 

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '[{"Clusters":{
  "desired_config":[{
    "type":"zoo.cfg",
    "tag":"version1480557385509",
    "properties":{
      "autopurge.purgeInterval":"24",
      "autopurge.snapRetainCount":"30",
      "dataDir":"/hadoop/zookeeper",
      "tickTime":"2000",
      "initLimit":"11",
      "syncLimit":"5",
      "clientPort":"2181"},
    "service_config_version_note":"New config version"}]}}]' 
"http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME"

 

4. Restart all components or services to have the config change take effect

E.g. Stop and Start a service

 

curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Stop HDFS"}, "ServiceInfo": {"state": "INSTALLED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS
curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Start HDFS"}, "ServiceInfo": {"state": "STARTED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS


Edit configuration using configs.py

configs.py is a helper script available with ambari-server installation that makes it easy to manipulate configuration. When using the script it will edit the configuration and apply it to cluster. You still need to restart services/components for the config to take effect.

configs.sh usage 

/var/lib/ambari-server/resources/scripts/configs.py --help
Usage: configs.py [options]
Options:
  -h, --help            show this help message and exit
  -t PORT, --port=PORT  Optional port number for Ambari server. Default is
                        '8080'. Provide empty string to not use port.
  -s PROTOCOL, --protocol=PROTOCOL
                        Optional support of SSL. Default protocol is 'http'
  -a ACTION, --action=ACTION
                        Script action: <get>, <set>, <delete>
  -l HOST, --host=HOST  Server external host name
  -n CLUSTER, --cluster=CLUSTER
                        Name given to cluster. Ex: 'c1'
  -c CONFIG_TYPE, --config-type=CONFIG_TYPE
                        One of the various configuration types in Ambari. Ex:
                        core-site, hdfs-site, mapred-queue-acls, etc.
  To specify credentials please use "-e" OR "-u" and "-p'":
    -u USER, --user=USER
                        Optional user ID to use for authentication. Default is
                        'admin'
    -p PASSWORD, --password=PASSWORD
                        Optional password to use for authentication. Default
                        is 'admin'
    -e CREDENTIALS_FILE, --credentials-file=CREDENTIALS_FILE
                        Optional file with user credentials separated by new
                        line.
  To specify property(s) please use "-f" OR "-k" and "-v'":
    -f FILE, --file=FILE
                        File where entire configurations are saved to, or read
                        from. Supported extensions (.xml, .json>)
    -k KEY, --key=KEY   Key that has to be set or deleted. Not necessary for
                        'get' action.
    -v VALUE, --value=VALUE
                        Optional value to be set. Not necessary for 'get' or
                        'delete' actions.

 

Edit configuration using APIs (1.4.1/1.2.5)

Verified against releases 1.4.1/1.2.5

Starting 1.4.2/1.4.3 you will have to add -H option to the curl calls. E.g. -H "X-Requested-By: ambari"

1. Find the latest version of the config type that you need to update.

curl -u admin:admin -X GET  http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME?fields=Clusters/desired_configs",
  "Clusters" : {
    "cluster_name" : "CLUSTER_NAME",
    "version" : "HDP-2.0.6",
    "desired_configs" : {
	  ...
      "mapred-site" : {
        "user" : "admin",
        "tag" : "version1384716039631"
      }
      ...
    }
  }
}

2. Read the config type with correct tag

curl -u admin:admin "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631"

Sample OUTPUT
{
  "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
  "items" : [
    {
      "href" : "http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=mapred-site&tag=version1384716039631",
      "tag" : "version1384716039631",
      "type" : "mapred-site",
      "Config" : {
        "cluster_name" : "CLUSTER_NAME"
      },
      "properties" : {
      ... THESE ARE THE PROPERTY KEY-VALUE PAIRS ...
      }
    }]
}

3a. Save a new version of the config and apply it (see 3b for doing it using one call)

curl --user admin:admin -i -X POST -d '{"type": "mapred-site", "tag": "version1384716041120", "properties" : {"mapreduce.admin.map.child.java.opts" : "-Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN",...}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/configurations
curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120"}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

3b. Save a new version of the config and apply it using one call

curl --user admin:admin -i -X PUT -d '{"Clusters":{"desired_config" : {"type": "mapred-site", "tag": "version1384716041120", "properties" : {...}}}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME

4. Restart all components or services to have the config change take effect
E.g. Stop and Start a service

curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Stop HDFS"}, "ServiceInfo": {"state": "INSTALLED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS
curl --user admin:admin -i -X PUT -d '{"RequestInfo": {"context": "Start HDFS"}, "ServiceInfo": {"state": "STARTED"}}' http://AMBARI_SERVER_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/HDFS

Edit configuration using configs.sh

configs.sh is a helper script available with ambari-server installation that makes it easy to manipulate configuration. When using the script it will edit the configuration and apply it to cluster. You still need to restart services/components for the config to take effect.

1. Update a parameter

/var/lib/ambari-server/resources/scripts/configs.sh set localhost c1 mapred-site "mapreduce.map.memory.mb" "512"

2. Read updated config

/var/lib/ambari-server/resources/scripts/configs.sh get localhost c1 mapred-site

Hint: Use export to file (option CONFIG_KEY) and import from file to edit large number of properties.
3. configs.sh usage
Note: If you have an older version then it may not have a way to provide custom userId and password. Edit the script to use the correct password if default admin:admin is not applicable.

/var/lib/ambari-server/resources/scripts/configs.sh
Usage: configs.sh [-u userId] [-p password] [-port port] <ACTION> <AMBARI_HOST> <CLUSTER_NAME> <CONFIG_TYPE> [CONFIG_FILENAME | CONFIG_KEY [CONFIG_VALUE]]

       [-u userId]: Optional user ID to use for authentication. Default is 'admin'.
       [-p password]: Optional password to use for authentication. Default is 'admin'.
       [-port port]: Optional port number for Ambari server. Default is '8080'. Provide empty string to not use port.
       <ACTION>: One of 'get', 'set', 'delete'. 'Set' adds/updates as necessary.
       <AMBARI_HOST>: Server external host name
       <CLUSTER_NAME>: Name given to cluster. Ex: 'c1'
       <CONFIG_TYPE>: One of the various configuration types in Ambari. Ex:global, core-site, hdfs-site, mapred-queue-acls, etc.
       [CONFIG_FILENAME]: File where entire configurations are saved to, or read from. Only applicable to 'get' and 'set' actions
       [CONFIG_KEY]: Key that has to be set or deleted. Not necessary for 'get' action.
       [CONFIG_VALUE]: Optional value to be set. Not necessary for 'get' or 'delete' actions.
  • No labels