...

• Move cluster-specific service HA configuration from the HaProvider configuration to the service declaration.
  
  • This just makes more logical sense
  • Moving cluster-specific details out of the provider configuration will make shared provider configurations applicable to more topologies (i.e., more reusable), across clusters.
  • The cluster-specific configuration could be discovered along with the service URL details, rather than having to be hard-coded in a provider configuration.
  • This will be treated as override configuration, so that the existing approach (i.e., complete configuration as HaProvider param values) will continue to work, to satisfy backward-compatibility requirements.
    
    

• Support new enabled value "auto" to allow service-specific configuration to determine whether HA treatment is enabled for that service.

  Code Block
  language xml title Example HaProvider Configuration with enabled=auto
  <provider> <role>ha</role> <name>HaProvider</name> <enabled>true</enabled> <param> <name>WEBHDFS</name> <value>maxFailoverAttempts=3;failoverSleep=1000;maxRetryAttempts=300;retrySleep=1000;enabled=auto</value> </param> </provider>

  In this case, the topology generation will reference the dfs.ha.automatic-failover.enabled property in the HDFS configuration to determine whether the HaProvider should be enabled for WEBHDFS.
  
  

Open Items

• Name the service-level override param for the service-specific HaProvider param component named 'enabled'
  • 'enabled'
  • 'haEnabled'
  • 'ha.enabled'
    
    
• Identify the nature of all the supported services
  • Which services are currently purely topology-based (DefaultURLManager)?
    
    • WEBHCAT
    • OOZIE
    • WEBHDFS
    • ?
      
      
  • Which services are currently ZooKeeper-based (BaseZooKeeperURLManager)?
    • HIVE (HS2ZooKeeperURLManager)
    • HBASE (HBaseZooKeeperURLManager)
    • Kafka (KafkaZooKeeperURLManager)
    • SOLR (SOLRZooKeeperURLManager)
    • ATLAS (AtlasZookeeperURLManager)
      
      
  • Could ZooKeeper support be added for any services which do no currently support it?
    • These have zookeeper-related configuration (see table):
      • WEBHDFS
      • OOZIE
      • YARN
      • WEBHCAT
      • RESOURCEMANAGER
        
        
        
• For the ZooKeeper-based-HA services, determine if the ZooKeeper details are available from the service's configuration via Ambari.
  
  
• Can "HA mode" be determined for every service type from the cluster configuration details? Can Knox dynamically identify HA-configured services, and generate the topology accordingly?
  
  
• Determine how to leverage the cluster discovery data to generate the ZooKeeper HA configuration for the relevant declared topology services.

...