You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Contributed by Umesh Kaushik <umesh.kaushik@bhujang.net>

Introduction

These instructions are for an Ubuntu host, with occasional comments about how to do similar tasks in CentOS.  It uses Metron's Vagrant support to launch a single-node VM installation.  The Vagrant image actually uses Debian instead of Ubuntu, because that was readily available, and they are very similar.  Resource requirements: These steps were tested on a Vagrant VM assigned 20 GB RAM, 100GB disk, and 2 CPUs.  Installation may work with lesser resources, but insufficient RAM will require turning off some processes.

Note

As usual, remember to re-type commands, or at least be careful with cut-and-paste, as the cwiki formatter may have changed hyphens, quotes, etc., into similar-looking non-ascii characters.

Step-by-step guide

  • Install pip:
  • sudo apt-get install python-pip python-dev build-essential
  • sudo pip install --upgrade pip
  • Install virtual box ( ubuntu, centos )
  • Install Java
    • (For centos, see here: centos)
    • sudo add-apt-repository ppa:webupd8team/java
    • sudo apt update
    • sudo apt install oracle-java8-installer
  • Install Vagrant:
    • Version issue:
      • 1.8.1 -> Plugin issue, can not download the pulgin
      • 1.8.5 -> Authentication issue while connecting to vmbox
      • Default apt-get install vagrant will give v 1.8.1
    • Download latest version from here for debian and then dpkg the package OR 
  • Install Maven
  • Install ansible
    • Only version 2.0.0.2 is supported by metron (as per the documents available online)
    • Installation instructions here
    • While installing ansible one may face errors regarding cryptography package following command should resolve that.
      • sudo apt-get install python-pip python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev libjpeg8-dev zlib1g-dev
      • For Centos this should probably solve it
        • Try following first:
          • sudo pip install -U setuptools
          • sudo pip install distribute
          • Try running the anisble installation command
        • If this doesn’t work:
          • sudo yum install gcc libffi-devel python-devel openssl-devel
          • Else use:
            • yum search python | grep -i devel
            • And see what all packages can resolve the issue (hit and trial)
  • Set Java and Maven Paths
    • mvn -v in terminal should show the maven path
    • Edit /etc/environments
    • Add: JAVA_HOME="java path"
    • Add: MAVEN_HOME="maven path"
  • Clone project from Apache Metron (incubating) repo
  • Now build the packages and launch the Vagrant VM:
    • cd incubator-metron
    • mvn clean package -DskipTests
    • vagrant plugin install vagrant-hostmanager
    • cd metron-deployment/vagrant/(select the module you want to run)
    • vagrant up
    • If things fail, then run: vagrant provision (multiple times)
  • Once the launch is successful:
    • The Vagrant VM will be running as "node1"
    • Ambari should be available in your browser at http://node1:8080/
    • Kibana should be available in your browser at http://node1:5000/
    • Monit should be available in your browser at http://node1:2812/ 
      • If Monit fails to work, or does not provide the expected data, then do the following:
      • vagrant ssh
      • sudo vi (or nano or vim) /etc/hosts
      • There should be a line near the top of the file that reads:
        • 127.0.0.1  localhost

        • (this is good)

      • However, sometimes Vagrant hostmanager adds a line that says:
        • 127.0.0.1  node1  node1

        • (this is bad)

      • If both lines exist, remove the bad line.  If the good line does not exist, replace the bad line with the good line.
      • Save and exit the editor
      • The rest should work fine

The above steps are based on:

Apache Metron TP 1 Install Instructions- Single Node Vagrant Deployment

which is also related to Metron cwiki page:

Metron 0.3.1 with HDP 2.5.0, Ambari 2.4.1, Centos 6, bare-metal install

Other useful links:

 

  • No labels