Running Stratos inside docker

Introduction

There is currently a show stopper bug with the Stratos docker images. See STRATOS-776 for more info

To make it easier for administrators to deploy a basic Stratos environment, docker images have been created for:

Bind9
PuppetMaster
Puppet test node
ActiveMQ
MySQL
Stratos

Advanced Users:

Stratos can be started with default, CC, AS, or SM profiles. For more information on Stratos profiles see: TODO

Installing docker

Install docker as described on the Docker Website

Downloading the Docker Images

You will need to pull in the docker images which are a total of about 1.6 GB in size:

Size (MB)	Image
225	Ubuntu 14.04 Base Image (used by all other images)
479	Stratos
233	ActiveMQ
395	Puppet Master
61	Puppet Test Node
58	Bind
150	MySQL
1604	TOTAL

To download the images run the following commands:

docker pull apachestratos/stratos:4.1.0-SNAPSHOT
docker pull apachestratos/activemq:4.1.0-SNAPSHOT
docker pull apachestratos/mysql:4.1.0-SNAPSHOT
docker pull apachestratos/puppettestnode:4.1.0-SNAPSHOT
docker pull apachestratos/puppetmaster:4.1.0-SNAPSHOT
docker pull apachestratos/bind:4.1.0-SNAPSHOT

Quickstart Script

An example run script is provided for you to quick get started.

The script is available in $STRATOS_SOURCE/tools/stratos-docker-images/run-example.sh.

At a minimum, set these environment variables to reflect your environment:

DOMAIN
IP_ADDR
EC2_x | OPENSTACK_x | VCLOUD_x
STRATOS_VERSION

After running the script, you should see the Stratos console log.

Press CTRL-C to exit the log, then enter the command docker ps to list your containers. You should see output similar to this:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b4bef4c36b28 apachestratos/stratos:latest /usr/local/bin/run 8 minutes ago Up 8 minutes 0.0.0.0:9443->9443/tcp goofy_ritchie
89205be82619 apachestratos/mysql:latest /usr/local/bin/run 9 minutes ago Up 9 minutes 0.0.0.0:49174->3306/tcp distracted_kirch
9b8254f6c82d apachestratos/puppetmaster:latest /usr/local/bin/run 9 minutes ago Up 9 minutes 0.0.0.0:49173->8140/tcp pensive_sammet
7c21012782ac apachestratos/activemq:latest /bin/sh -c 'java -Xm 9 minutes ago Up 9 minutes 8161/tcp, 0.0.0.0:49172->61616/tcp distracted_fermi
0f13e535f6bc apachestratos/bind:latest /usr/local/bin/run 9 minutes ago Up 9 minutes 0.0.0.0:53->53/udp sleepy_goldstine

Description of the Quickstart Script

The Quickstart script is divided into the following sections: Bind9 (DNS), ActiveMQ, MySQL and Stratos containers.

Bind9 (DNS)

The commands below start an Bind docker container.

# Set the DOMAIN to the dns domain name you want to use for your Stratos environment
# The DNS domain name is only used internally by the stratos docker images and should not be exposed publically
export DOMAIN=example.com 
 
# Set the IP_ADDR to the IP address you use to reach the docker host
export IP_ADDR=192.168.56.5
 
# We need to grant access to hosts or networks to allow DNS records to be added from those hosts
# See http://www.zytrax.com/books/dns/ch7/address_match_list.html for more info
export UPDATE_ADDR_LIST=any 
 
# Set the version of Stratos docker images
export STRATOS_VERSION=4.1.0-SNAPSHOT
 
export BIND_ID=$(docker run -d -p 53:53/udp -e "DOMAIN=$DOMAIN" -e "IP_ADDR=$IP_ADDR" -e "UPDATE_ADDR_LIST=$UPDATE_ADDR_LIST" apachestratos/bind:$STRATOS_VERSION); sleep 2s;
export BIND_IP_ADDR=$(docker inspect --format '{{ .NetworkSettings.Gateway }}' $BIND_ID)

ActiveMQ

The commands below start an ActiveMQ docker container.

export MB_ID=$(docker run -p 61616 -d apachestratos/activemq:$STRATOS_VERSION); sleep 2s;
export MB_IP_ADDR=$(docker inspect --format '{{ .NetworkSettings.Gateway }}' $MB_ID)
export MB_PORT=$(docker port $MB_ID 61616 | awk -F':' '{ print $2 }')

PuppetMaster

The commands below start a PuppetMaster docker container.

First, we need to create a host record in the DNS server:

# Create a file containing instructions for the nsupdate tool
cat > addpuppetdomain.txt <<EOF
server 127.0.0.1
zone $DOMAIN
prereq nxdomain puppet.$DOMAIN.
update add puppet.$DOMAIN. 10  A $IP_ADDR
send
EOF
 
# Run the nsupdate tool to add puppetmaster to the domain
nsupdate addpuppetdomain.txt
 
# Check the update was successful
dig @$IP_ADDR ANY puppet.$DOMAIN.
 
# You should see:
# ...
# ;; ANSWER SECTION:
# puppetmaster.<<your domain>>. 10    IN      A       <<your ip address>>
# ...

Now we need to start the PuppetMaster docker image:

export MASTERHOSTNAME=puppet.$DOMAIN
export TRUSTSTORE_PASSWORD=wso2carbon


export PUPPET_ID=$(docker run -d -h ${MASTERHOSTNAME} --dns=${BIND_IP_ADDR} -e "DOMAIN=${DOMAIN}" -e "MASTERHOSTNAME=${MASTERHOSTNAME}" -e "MB_HOSTNAME=${MB_IP_ADDR}" -e "MB_PORT=${MB_PORT}" -e "TRUSTSTORE_PASSWORD=${TRUSTSTORE_PASSWORD}" -p 8140 apachestratos/puppetmaster:$STRATOS_VERSION); sleep 2s;
export PUPPET_IP_ADDR=$(docker inspect --format '{{ .NetworkSettings.Gateway }}' $PUPPET_ID)
export PUPPET_PORT=$(docker port $PUPPET_ID 8140 | awk -F':' '{ print $2 }')

Puppet Test

The commands below start a Puppet test docker container to test that the puppetmaster has been setup correctly.

First, we need to create a host record in the DNS server:

cat > addpuppettestnode.txt <<EOF
server 127.0.0.1
zone $DOMAIN
prereq nxdomain testnode.$DOMAIN.
update add testnodde.$DOMAIN. 10  A $IP_ADDR
send
EOF

nsupdate addpuppettestnode.txt

Now run the image:

docker run -d=false -i -t --dns=$BIND_IP_ADDR -h testnode.$DOMAIN -e "PUPPET_HOST=puppet.$DOMAIN" -e "PUPPET_PORT=$PUPPET_PORT" apachestratos/puppettestnode:$STRATOS_VERSION

You should see puppet setting up the node:

=============================
Environment Variable Settings
=============================
Setting PUPPET_HOST=puppet.example.com
Setting PUPPET_PORT=49164
=============================
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1095:in `block in issue_deprecations')
Info: Creating a new SSL key for testnode.example.com
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for testnode.example.com
Info: Certificate Request fingerprint (SHA256): 38:BC:A3:EE:36:69:C0:B3:D0:28:E6:9F:45:88:49:41:B5:F1:C9:B6:CB:44:DE:F9:73:E7:CD:92:A6:A9:7A:01
Info: Caching certificate for testnode.example.com
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for ca
Info: Retrieving pluginfacts
Info: Retrieving plugin
Notice: /File[/var/lib/puppet/lib/facter]/ensure: created
...

MySQL

The commands below start a MySQL container that has the Stratos database schema installed.

NOTE:

The docker image does not persist the mysql data - all data is lost when the docker image stops.

To persist the mysql data, or use your existing MySQL server, see the Mysql heading in the Advanced Configuration section.

export USERSTORE_ID=$(docker run -d -p 3306 -e MYSQL_ROOT_PASSWORD=password apachestratos/mysql:$STRATOS_VERSION); sleep 2s;
export USERSTORE_IP_ADDR=$(docker inspect --format '{{ .NetworkSettings.Gateway }}' $USERSTORE_ID)
export USERSTORE_PORT=$(docker port $USERSTORE_ID 3306 | awk -F':' '{ print $2 }')

Stratos

Next we create an environment variable docker_env that is passed to the Stratos docker image to configure Stratos when the container starts.

If you have used the ActiveMQ and MySQL Container, you may leave the settings as they are below.

You will need to provide settings pointing to your puppet environment.

unset docker_env # Ensure docker environment variable is clean to start with

# Database Settings
docker_env+=(-e "USERSTORE_DB_HOSTNAME=${USERSTORE_IP_ADDR}")
docker_env+=(-e "USERSTORE_DB_PORT=${USERSTORE_PORT}")
docker_env+=(-e "USERSTORE_DB_SCHEMA=USERSTORE_DB_SCHEMA")
docker_env+=(-e "USERSTORE_DB_USER=root")
docker_env+=(-e "USERSTORE_DB_PASS=password")

# Puppet Setings
docker_env+=(-e "PUPPET_IP=${IP_ADDR}")
docker_env+=(-e "PUPPET_HOSTNAME=${DOMAIN}")
docker_env+=(-e "PUPPET_ENVIRONMENT=none")

# MB Settings
docker_env+=(-e "MB_HOSTNAME=${MB_IP_ADDR}")
docker_env+=(-e "MB_PORT=${MB_PORT}")

Change the IaaS settings to reflect your environment.

docker_env+=(-e "EC2_ENABLED=true")
docker_env+=(-e "EC2_IDENTITY=none")
docker_env+=(-e "EC2_CREDENTIAL=none")
docker_env+=(-e "EC2_CREDENTIAL=none")
docker_env+=(-e "EC2_OWNER_ID=none")
docker_env+=(-e "EC2_AVAILABILITY_ZONE=none")
docker_env+=(-e "EC2_SECURITY_GROUPS=none")
docker_env+=(-e "EC2_KEYPAIR=none")

docker_env+=(-e "OPENSTACK_ENABLED=false")
docker_env+=(-e "OPENSTACK_IDENTITY=none")
docker_env+=(-e "OPENSTACK_CREDENTIAL=none")
docker_env+=(-e "OPENSTACK_ENDPOINT=none")

docker_env+=(-e "VCLOUD_ENABLED=false")
docker_env+=(-e "VCLOUD_IDENTITY=none")
docker_env+=(-e "VCLOUD_CREDENTIAL=none")
docker_env+=(-e "VCLOUD_ENDPOINT=none")

Set the Stratos profile to 'default'. See the section Stratos in the Advanced Configuration section for configuring other profiles.

# Stratos Settings [STRATOS_PROFILE=default|cc|as|sm]
docker_env+=(-e "STRATOS_PROFILE=default")

Finally, start Stratos and tail the logs.

# Start Stratos container as daemon
STRATOS_ID=$(docker run -d "${docker_env[@]}" --dns=$BIND_IP_ADDR -p 9443:9443 apachestratos/stratos:$STRATOS_VERSION)
sudo docker logs -f $STRATOS_ID

Building the docker images

To build the docker images from source:

Ensure you can run docker without requiring sudo (Ubuntu Example)
Build Stratos as described in Building from Source.
Change to the $STRATOS_SOURCE/tools/stratos-docker-images folder
Run the command: mvn install -Pdocker-build

Volunteers needed! Please see STRATOS-764 for improvements to the docker build process.

Publishing the docker images

To push the docker images to the public docker registry:

Ensure you can run docker without requiring sudo (Ubuntu Example)
Ensure you have an account on Docker.com and that account is linked to the 'apachestratos' organisation. If you need your account linked, please send an email to dev@stratos.apache.org
Change to the $STRATOS_SOURCE/tools/stratos-docker-images folder
Login to docker: sudo docker login
Run the command mvn install -Pdocker-push

The upload will take a long time as it pushes approx 1.4GB of data.

Advanced Configuration

MySQL

Persisting MySQL docker image data

If you would the MySQL docker image data to persist so that it does not disappear when the container stops you can mount a folder from the docker host to the docker container using the -v option:

-v /hostfolder:/var/lib/mysql

Full example:

sudo docker run -d -p 3306 -v /home/me/mysql-data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=password apachestratos/mysql

Using your existing MySQL server

If you would like to use your existing MySQL server instead of the Apache Stratos MySQL docker image, you will need to run the following script to create the Stratos schema:

$STRATOS_SOURCE/tools/stratos-installer/resources/mysql.sql

Stratos

This section describes advanced configuration for the Stratos Image.

X_HOSTNAME variables

The hostname variables can be configured with a hostname or an IP address. If using a hostname the value must be resolvable to the docker host.

Cloud Controller

To run a standalone Cloud Controller, the following docker_env environment variables are required:

docker_env+=(-e "STRATOS_PROFILE=cc")

docker_env+=(-e "MB_HOSTNAME=none")
docker_env+=(-e "MB_PORT=none")

docker_env+=(-e "EC2_ENABLED=true")
docker_env+=(-e "EC2_IDENTITY=none")
docker_env+=(-e "EC2_CREDENTIAL=none")
docker_env+=(-e "EC2_CREDENTIAL=none")
docker_env+=(-e "EC2_OWNER_ID=none")
docker_env+=(-e "EC2_AVAILABILITY_ZONE=none")
docker_env+=(-e "EC2_SECURITY_GROUPS=none")
docker_env+=(-e "EC2_KEYPAIR=none")

docker_env+=(-e "OPENSTACK_ENABLED=false")
docker_env+=(-e "OPENSTACK_IDENTITY=none")
docker_env+=(-e "OPENSTACK_CREDENTIAL=none")
docker_env+=(-e "OPENSTACK_ENDPOINT=none")

docker_env+=(-e "VCLOUD_ENABLED=false")
docker_env+=(-e "VCLOUD_IDENTITY=none")
docker_env+=(-e "VCLOUD_CREDENTIAL=none")
docker_env+=(-e "VCLOUD_ENDPOINT=none")

Autoscaler

To run a standalone Autoscaler, the following docker_env environment variables are required:

docker_env+=(-e "STRATOS_PROFILE=as")

docker_env+=(-e "MB_HOSTNAME=none")
docker_env+=(-e "MB_PORT=none")
 
docker_env+=(-e "CC_HOSTNAME=none")
docker_env+=(-e "CC_LISTEN_PORT=none")
 
docker_env+=(-e "SM_HOSTNAME=none")
docker_env+=(-e "SM_LISTEN_PORT=none")

Stratos Manager

To run a standalone Stratos Manager, the following docker_env environment variables are required:

docker_env+=(-e "STRATOS_PROFILE=SM")

docker_env+=(-e "MB_HOSTNAME=none")
docker_env+=(-e "MB_PORT=none")
 
docker_env+=(-e "USERSTORE_DB_HOSTNAME=none")
docker_env+=(-e "USERSTORE_DB_PORT=none")
docker_env+=(-e "USERSTORE_DB_SCHEMA=none")
docker_env+=(-e "USERSTORE_DB_USER=none")
docker_env+=(-e "USERSTORE_DB_PASS=none")
 
docker_env+=(-e "PUPPET_IP=none")
docker_env+=(-e "PUPPET_HOSTNAME=none")
docker_env+=(-e "PUPPET_ENVIRONMENT=none")

docker_env+=(-e "CC_HOSTNAME=none")
docker_env+=(-e "CC_LISTEN_PORT=none")

docker_env+=(-e "AS_HOSTNAME=none")
docker_env+=(-e "AS_LISTEN_PORT=none")

Debugging a container

SSHD is not installed on any of the containers, so if you need to debug a container, use the run-nsenter.sh script in the $STRATOS_SOURCE/tools/stratos-docker-images folder. For example:

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a64b770fb52a apachestratos/stratos:4.1.0-SNAPSHOT "/usr/local/bin/run" 32 minutes ago Up 32 minutes 0.0.0.0:9443->9443/3306/tcp kickass_franklin


sudo -i . $STRATOS_SOURCE/tools/stratos-docker-images/run-nsenter.sh a64b770fb52a
root@a64b770fb52a:/# <- you are now in a bash shell inside the docker image

More information on nsenter here.

See here for the rationale why SSHD has not been installed.

Docker Tips

Definitive guide to docker - see The Docker Book

Bugs and outstanding Items

The JIRA ticket STRATOS-705 has been created to track tasks related to running Stratos in Docker.

If you find a bug, please mention it on the dev mailing list, or add a sub-ticket to STRATOS-705.

Child pages

Running Stratos inside docker

Introduction

Installing docker

Downloading the Docker Images

Quickstart Script

Description of the Quickstart Script

Bind9 (DNS)

ActiveMQ

PuppetMaster

Puppet Test

MySQL

Stratos

Building the docker images

Publishing the docker images

Advanced Configuration

MySQL

Persisting MySQL docker image data

Using your existing MySQL server

Stratos

X_HOSTNAME variables

Cloud Controller

Autoscaler

Stratos Manager

Debugging a container

Docker Tips

Bugs and outstanding Items