Contributed by Umesh Kaushik <umesh.kaushik@bhujang.net>
Introduction
These instructions are for an Ubuntu host, with occasional comments about how to do similar tasks in CentOS. It uses Metron's Vagrant support to launch a single-node VM installation. The Vagrant image actually uses Debian instead of Ubuntu, because that was readily available, and they are very similar. Resource requirements: These steps were tested on a Vagrant VM assigned 20 GB RAM, 100GB disk, and 2 CPUs. Installation may work with lesser resources, but insufficient RAM will require turning off some processes.
Note
As usual, remember to re-type commands, or at least be careful with cut-and-paste, as the cwiki formatter may have changed hyphens, quotes, etc., into similar-looking non-ascii characters.
Step-by-step guide
- Install pip:
- sudo apt-get install python-pip python-dev build-essential
- sudo pip install --upgrade pip
- Install virtual box ( ubuntu, centos )
- Edit sources.list
- sudo vim /etc/apt/sources.list
- Add following two lines at the end of the file (or any place you like, I prefer end):
- Save the file and exit
- Setup Oracle public key
- Install Oracle Virtualbox
- sudo apt-get update (must do)
- sudo apt-get install virtualbox-5.0
- Install Java
- (For centos, see here: centos)
- sudo add-apt-repository ppa:webupd8team/java
- sudo apt update
- sudo apt install oracle-java8-installer
- Install Vagrant:
- Version issue:
- 1.8.1 -> Plugin issue, can not download the pulgin
- 1.8.5 -> Authentication issue while connecting to vmbox
- Default apt-get install vagrant will give v 1.8.1
- Download latest version from here for debian and then dpkg the package OR
- (not tested) download any version other than 1.8.1 and 1.8.2 with:
- Install Maven
- Install ansible
- Only version 2.0.0.2 is supported by metron (as per the documents available online)
- Installation instructions here
- While installing ansible one may face errors regarding cryptography package following command should resolve that.
- sudo apt-get install python-pip python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev libjpeg8-dev zlib1g-dev
- For Centos this should probably solve it
- Try following first:
- sudo pip install -U setuptools
- sudo pip install distribute
- Try running the anisble installation command
- If this doesn’t work:
- sudo yum install gcc libffi-devel python-devel openssl-devel
- Else use:
- yum search python | grep -i devel
- And see what all packages can resolve the issue (hit and trial)
- Set Java and Maven Paths
- mvn -v in terminal should show the maven path
- Edit /etc/environments
- Add: JAVA_HOME="java path"
- Add: MAVEN_HOME="maven path"
- Clone project from Apache Metron (incubating) repo
- Now build the packages and launch the Vagrant VM:
- cd incubator-metron
- mvn clean package -DskipTests
- vagrant plugin install vagrant-hostmanager
- cd metron-deployment/vagrant/(select the module you want to run)
- vagrant up
- If things fail, then run: vagrant provision (multiple times)
- Once the launch is successful:
- The Vagrant VM will be running as "node1"
- Ambari should be available in your browser at http://node1:8080/
- Kibana should be available in your browser at http://node1:5000/
- Monit should be available in your browser at http://node1:2812/
- If Monit fails to work, or does not provide the expected data, then do the following:
- vagrant ssh
- sudo vi (or nano or vim) /etc/hosts
- There should be a line near the top of the file that reads:
127.0.0.1 localhost
(this is good)
- However, sometimes Vagrant hostmanager adds a line that says:
- If both lines exist, remove the bad line. If the good line does not exist, replace the bad line with the good line.
- Save and exit the editor
- The rest should work fine
Related articles
The above steps are based on:
Apache Metron TP 1 Install Instructions- Single Node Vagrant Deployment
which is also related to Metron cwiki page:
Metron 0.3.1 with HDP 2.5.0, Ambari 2.4.1, Centos 6, bare-metal install
Other useful links: