1. Setup a Kubernetes cluster, please refer Kubernetes Getting Started Guide for more information.
    In this example we will be using a sample Vagrant script to setup a Kubernetes cluster with Kubernetes 0.5.2:

    1. Download and Install VirtualBox 4.3.20.

    2. Download and install Vagrant 1.6.5.

    3. Clone the following Vagrant Git repository. This folder will be referred as <VAGRANT-KUBERNETES-SETUP> for later reference.
       

      git clone https://github.com/nirmal070125/vagrant-kubernetes-setup.git

    4. Start the Kubernetes cluster by navigating to Vagrant setup folder and executing the following Vagrant command:
       

      cd <VAGRANT-KUBERNETES-SETUP>/ 
vagrant up

    5. Following are the IP Addresses of virtual machine instances of this Kubernetes cluster:

      Virtual Machine InstanceIP Address
      Kubernetes Discovery172.17.8.10
      Kubernetes Master172.17.8.100
      Kubernetes Minion 1172.17.8.101
      Kubernetes Minion 2172.17.8.102

    6. Ensure that all the following prerequisites have been met based on your environment:

      Software ProductVersion
      Java Development Kit (JDK)1.7.0_XX
      Apache Maven3.1.1 or later
      Git Client2.2.1 or later
      Apache ActiveMQ5.10

  2. SSH into the Kubernetes master:
    If you are using Vagrant you can SSH into the Kubernetes master as follows:

    vagrant ssh master


  3. Restart Kubernetes controller manager from the master node:

    This step only needs to be executed if you are using Kubernetes 0.5.2 or above Vagrant setup. This is a workaround for a known issue in Kubernetes 0.5.2.

    sudo systemctl restart controller-manager

  4. Verify Kubernetes cluster status. 
    This needs to be done approximately around 30 seconds after creating the cluster. 

    kubecfg list minions

    Once the following command is run all the minions in the Kubernetes cluster should be listed, otherwise there could a problem in the Kubernetes cluster. If so please refer the Kubernetes Getting Started Guide and try to solve the problem.

     

  5. Pull the test Docker image gurpartap/redis from the Docker Hub.

    docker pull gurpartap/redis

  6. Navigate to the Kubernetes client component.

    cd <STRATOS-SOURCE-HOME>/components/org.apache.stratos.kubernetes.client/

  7.  

    Run the Kubernetes client live tests:
    The above tests will make sure that the Kubernetes cluster is working as expected and it is compatible with the Stratos version you are using. If the tests fail, either there could be issues in the Kubernetes cluster or it may not be compatible with the Stratos version you are using.

     

    • If you wish to use the default configurations, execute the following command: 

      mvn test -o -Plive

    • If you wish to use customized configurations, execute the following command:

      mvn test -Plive -Dkubernetes.api.endpoint=http://<KUBERNETES-MASTER-IP-ADDRESS>:8080/api/v1beta1/

  8. Pull the required Docker images into each of the Kubernetes hosts (i.e., master, minion-1 and minion-2) from the Docker Hub. To do this SSH into each Kubernetes host and execute the docker pull command. 

    It is not mandatory to push Docker images to the Docker Hub and pull them back, they could be directly imported into the Kubernetes hosts. This is a workaround for a known issue in Kubernetes 0.5.2.

    A sample Docker image (php:4.1.0-alpha) has been created and pushed to Docker Hub for testing purposes. However, in a production scenario you will need to create the required Docker images and push them to Docker Hub before executing this step. In this example we will pull Stratos PHP Docker image as follows:  

    • If you are using Vagrant, you will have to use the following code format:

      vagrant ssh <KUBERNETES-HOST> 
docker pull <REPOSITORY>/<IMAGE-NAME>:<VERSION>

      For example:

      vagrant ssh master
docker pull stratos/php:4.1.0-alpha
 
vagrant ssh minion-1
docker pull stratos/php:4.1.0-alpha
 
vagrant ssh minion-2
docker pull stratos/php:4.1.0-alpha

    • If you are trying to access the Kubernetes host without using Vagrant, you will have to use the following command format:

      ssh <USERNAME>@<KUBERNETES-HOST-IP>
docker pull <REPOSITORY>/<IMAGE-NAME>:<VERSION>

  9. Clone Stratos remote Git repository:

    git clone https://git-wip-us.apache.org/repos/asf/stratos.git

    Please switch to the relevant branch or tag according to the Stratos version you are using. This folder will be referred as <STRATOS-SOURCE-HOME> for later reference.

     

  10. Download Stratos binary distribution (apache-stratos-version.zip) and unzip it. This folder will be referred as <STRATOS-HOME> for later reference.

    This can be done using any of the following methods:

    • Method 1 - Download the Stratos binary distribution from Apache Download Mirrors and unzip it.

    • Methods 2 - Build the Stratos source to obtain the binary distribution and unzip it.

      1. Build Stratos using Maven.

        1. Navigate to the <STRATOS-SOURCE-HOME>/ directory, which is the directory that you checked out the source: 
          cd <STRATOS-SOURCE-HOME>/ 
           

        2. Use Maven to build the source:
          mvn clean install
           

      2. Obtain the Stratos binary distribution apache-stratos-version.zip from the <STRATOS-SOURCE-HOME>/products/stratos/modules/distribution/target/ directory and unzip it.

  11. Start ActiveMQ:

    1. Download and unzip ActiveMQ.

    2. Navigate to the <ACTIVEMQ-HOME>/bin/ directory, which is in the unzipped ActiveMQ distribution.

    3. Run the following command to start ActiveMQ.

      ./activemq start

  12. Start Stratos server: 

    bash <STRATOS-HOME>/bin/stratos.sh start

    If you wish you can tail the log and verify Stratos server is starting without any issues:
    tail -f repository/logs/wso2carbon.log  

  13. Create a Kubernetes cluster definition:

    Kubernetes cluster definition will provider information required for Stratos to communicate with Kubernetes. Please update the IP addresses of the Kubernetes cluster, list of minions, Message Broker IP address, Message Broker port, CEP IP address and CEP port in the following example definition:

    kubernetes-cluster.json
    {
      "clusterId": "kubernetes-cluster-1",
      "description": "Kubernetes CoreOS cluster",
      "kubernetesMaster": {
                  "hostId" : "KubHostMaster1",
                  "hostname" : "master.dev.kubernetes.example.org",
                  "hostIpAddress" : "172.17.8.100",
                  "property" : [
                  ]
        },

        "portRange" : {
           "upper": "5000",
           "lower": "4500"
        },

        "kubernetesHosts": [
              {
                     "hostId" : "KubHostSlave1",
                     "hostname" : "slave1.dev.kubernetes.example.org",
                     "hostIpAddress" : "172.17.8.101",
                     "property" : [
                     ]
                },
                {
                     "hostId" : "KubHostSlave2",
                     "hostname" : "slave2.dev.kubernetes.example.org",
                     "hostIpAddress" : "172.17.8.102",
                     "property" : [
                     ]
                }
    ],
    "property":[
       {
          "name":"payload_parameter.MB_IP",
          "value":"172.17.8.1"
       },
       {
          "name":"payload_parameter.MB_PORT",
          "value":"1883"
       },
       {
          "name":"payload_parameter.CEP_IP",
          "value":"172.17.8.1"
       },
       {
          "name":"payload_parameter.CEP_PORT",
          "value":"7711"
       },
       {
          "name":"payload_parameter.LOG_LEVEL",
          "value":"DEBUG"
       }
    ]
}

  14. Add Kubernetes cluster to Stratos:

    Execute the following curl command to add Kubernetes cluster to Stratos:

    curl -X POST -H "Content-Type: application/json" -d "@kubernetes-cluster.json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/kubernetesClusters
    STRATOS-HOST-IP: The IP address of the host running Stratos server
    STRATOS-HTTPS-PORT: Default is 9443, if you have defined an offset in the carbon.xml file, it would be 9443 + offset

  15. Create cartridge definitions for each Docker image:

    In this example we will use the following cartridge definition to add the Stratos PHP Docker image to Stratos:

    php.json
    {
    "type": "php",
    "provider": "apache",
    "category": "data",
    "host": "stratos.org",
    "displayName": "php",
    "description": "php Cartridge",
    "version": "7",
    "multiTenant": "false",
    "portMapping": [
        {
            "protocol": "http",
            "port": "22",
            "proxyPort": "8280"
        }
    ],
    "deployment":{
        "baseDir":"/var/www"
    },
    "iaasProvider":[
       {
          "type":"kubernetes",
          "imageId":"stratos/php:4.1.0-alpha",
          "networkInterfaces":[
          ],
          "property":[
          ]
       }
    ]
}

  16. Add cartridges to Stratos:

    In this example we will add the Stratos PHP cartridge created above by executing the following cURL command:

    curl -X POST -H "Content-Type: application/json" -d "@php.json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/cartridges

    STRATOS-HOST-IP : The ip address of the host running Stratos server 
    STRATOS-HTTPS-PORT : Default is 9443, if you have defined an offset in carbon.xml, it would be 9443 + offset

  17. Create autoscaling policies for each cartridge:

    In each auto-scaling policy define threshold values according to the requirements of the cartridge. In this example we will use the following auto-scaling policy for Stratos PHP cartridge:

    autoscaling-policy.json
    {
  "id": "autoscaling-policy-1",
  "loadThresholds": {
    "requestsInFlight": {
      "threshold": 50
    },
    "memoryConsumption": {
      "threshold": 40
    },
    "loadAverage": {
      "threshold": 40
    }
  }
}

  18. Add auto-scaling policies to Stratos:

    In this example we will add the auto-scaling policy created above by executing the following cURL command:

    curl -X POST -H "Content-Type: application/json" -d "@autoscaling-policy.json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/autoscalingPolicies

  19. Create an application:

    An application needs to be created by aggregating required cartridges. In this example we will use the following sample application definition that contains the Stratos PHP cartridge:

    application.json
    {  
   "applicationId":"single-cartridge-app",
   "alias":"single-cartridge-app",
   "multiTenant":true,
   "components":{  
      "cartridges":[  
         {  
            "type":"php",
                "cartridgeMin": 1,
                "cartridgeMax": 10,    
            "subscribableInfo":{  
                   "alias":"php1",
                   "autoscalingPolicy":"autoscale-policy-1",
                "artifactRepository":{
                   "privateRepo":false,
                   "repoUrl":"https://github.com/imesh/stratos-php-applications.git",
                   "repoUsername":"",
                   "repoPassword":""
                }
            }
         }
      ]
   }
}

  20. Add application to Stratos:

    Execute the following cURL command to add application to Stratos:

    curl -X POST -H "Content-Type: application/json" -d "@application.json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/applications

  21. Create a deployment policy:

    The deployment policy defines the cloud partitions that the application will use. In this example we will use the following deployment policy:

    deployment-policy.json
    {
    "applicationPolicy": {
        "networkPartition": [
            {
                "id": "network-partition-1",
                "activeByDefault": "true",
                "kubernetesClusterId":"kubernetes-cluster-1",
                "partitions": [
                    {
                        "id": "partition-1",
                        "provider": "kubernetes",
                        "property": [
                            {
                                "name": "region",
                                "value": "default"
                            }
                        ]
                    }
                ]
            }
        ]
    },
    "childPolicies": [
        {
            "alias": "php1",
            "networkPartition": [
                {
                    "id": "network-partition-1",
                    "partitionAlgo": "one-after-another",
                    "partitions": [
                        {
                            "id": "partition-1",
                            "max": 5
                        }
                    ]
                }
            ]
        }
    ]
}

  22. Deploy the application:

    Execute the following cURL command to deploy the above application together with the created deployment policy:

    curl -X POST -H "Content-Type: application/json" -d "@deployment-policy.json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/applications/<APPLICATION-ID>/deploy

    APPLICATION-ID : Application ID defined in the application definition. In this example it would be single-cartridge-app

  23. Wait until the application becomes active:

    Tail Stratos server log and wait until the application becomes active. Once the application is active, the following log message will appear:

    tail -f <STRATOS-HOME>/repository/logs/wso2carbon.log
 
# INFO {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} -  Publishing Application Active event for [application]: single-cartridge-app [instance]: single-cartridge-app-1

  24. Find the default public IP address of a member:

    Execute the following cURL command to query the application runtime and find the default public IP address of a member:

    curl -X GET -H "Content-Type: application/json" -k -v -u admin:admin https://<STRATOS-HOST-IP>:<STRATOS-HTTPS-PORT>/api/applications/<APPLICATION-ID>/runtime
    {
   "applicationInstances":[
      {
         "applicationId":"single-cartridge-app",
         "clusterInstances":[
            {
               "alias":"php1",
               "clusterId":"php1.php.domain",
               "hostNames":[
                  "php1.php.stratos.org"
               ],
               "instanceId":"single-cartridge-app-1",
               "member":[
                  {
                     "clusterId":"php1.php.domain",
                     "defaultPrivateIP":"10.10.10.1",
                     "defaultPublicIP":"20.20.20.1",
                     "memberId":"php1.php.domain7b4328f5-b135-4a91-ae6f-f4839b53fee4",
                     "networkPartitionId":"network-partition-1",
                     "partitionId":"partition-1",
                     "property":[
                        {
                           "name":"PRIMARY",
                           "value":false
                        },
                        {
                           "name":"MIN_COUNT",
                           "value":1
                        }
                     ],
                     "serviceName":"php",
                     "status":"Active"
                  }
               ],
               "parentInstanceId":"single-cartridge-app-1",
               "serviceName":"php",
               "status":"Active",
               "tenantRange":"*"
            }
         ],
         "instanceId":"single-cartridge-app-1",
         "status":"Active"
      }
   ],
   "id":"single-cartridge-app",
   "status":"Active",
   "tenantAdminUsername":"admin",
   "tenantDomain":"carbon.super"
}

    In this example we can use default public IP address: 20.20.20.1

     

  25. Access the service via Kubernetes proxy service:

    Stratos will create a proxy service for each service cluster. In this example we have defined Kubernetes proxy service port range as 4500 - 5000 in the Kubernetes cluster definition. Therefore, the first proxy service that will be created will get port 4500. Use the URL below to access the service:

    http://<MEMBER-PUBLIC-IP-ADDRESS>:<PROXY-SERVICE-PORT>/<CONTEXT-PATH>

    Currently, it is not possible to query auto generated Kubernetes proxy service ports via the Stratos API. However, they can be found on the Stratos server log. Once, an API method is introduced documentation will be updated.

    In this example use the URL below to access the service:

    http://20.20.20.1:4500/index.php

For troubleshooting see, Troubleshooting Docker Support Related Issues.

  • No labels