Apache Myriad can be deployed as a Docker container either standalone or via Marathon
Step-by-Step: Myriad on Docker via Standalone Deployment (command sequence below)
- The Docker image is created by executing the gradlew buildRMDocker task and pushing the resulting image to a local or remote Docker registry.
- Access the machine where the Hadoop distribution is located and cd to the $HADOOP_HOME directory
- If you have not already done so, create the $HADOOP_HOME/dist directory that contains the config.tgz and binary.tgz files as detailed in the installation guide.
- Execute the docker run command as detailed below
- Check Apache Myriad UI and Hadoop Application Master UI to confirm the NodeManagers are up
Building and Deploy Apache Myriad Docker Image
#Build Docker image with desired username
./gradlew -P dockerTag=<username>/myriad buildRMDocker
#Push the Docker image to Docker hub
docker push <username>/myriad
#Create dist, dist/config directories and copy required files
cd $HADOOP_HOME
mkdir $HADOOP_HOME/dist
mkdir $HADOOP_HOME/dist/config
cp $HADOOP_HOME/etc/hadoop/* $HADOOP_HOME/dist/config
#Build config.tgz file where filename matches servedConfigPath variable in myriad-config-default.yml
tar czf dist/<filename>.tgz dist/config
#Build binary.tgz file where filename matches servedBinaryPath variable in myriad-config-default.yml
tar czf <filename>.tgz $HADOOP_HOME/ --exclude=HADOOP_HOME/dist
#Start Myriad Docker container ENSURE YOU ARE IN $HADOOP_HOME DIRECTORY
docker run -d --name myriad-rm --net=host -v $PWD/dist:/dist -v $PWD/dist/config/:/usr/local/hadoop/etc/hadoop <username from step 1 above>/myriad
#Confirm that Myriad Docker container is up
docker ps | grep myriad-rm
#Confirm Apache Myriad UI is up
http://<hadoop master ip address/dns>:8192
#Confirm Hadoop Application Master is up
http://<hadoop master ip address/dns>:8188
#Run example Map/Reduce job such as teragen to confirm all installation details are correct
$HADOOP_HOME/bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar pi 100 10000
Step-by-Step: Myriad on Docker via Marathon
TODO