Describes the steps required to build and run Apache Trafodion.
Supported Platforms
Red Hat or Centos 6.x (6.4 or later) versions are supported as development and production platforms.
Prerequisites
You need to install the following packages before you can install Apache Trafodion.
sudo yum install epel-release sudo yum install alsa-lib-devel ant ant-nodeps boost-devel cmake \ device-mapper-multipath dhcp flex gcc-c++ gd git glibc-devel \ glibc-devel.i686 graphviz-perl gzip java-1.7.0-openjdk-devel \ libX11-devel libXau-devel libaio-devel \ libcurl-devel libibcm.i686 libibumad-devel libibumad-devel.i686 \ libiodbc libiodbc-devel librdmacm-devel librdmacm-devel.i686 \ libxml2-devel log4cxx log4cxx-devel lua-devel lzo-minilzo \ net-snmp-devel net-snmp-perl openldap-clients openldap-devel \ openldap-devel.i686 openmotif openssl-devel openssl-devel.i686 \ openssl-static perl-Config-IniFiles perl-Config-Tiny \ perl-DBD-SQLite perl-Expect perl-IO-Tty perl-Math-Calc-Units \ perl-Params-Validate perl-Parse-RecDescent perl-TermReadKey \ perl-Time-HiRes protobuf-compiler protobuf-devel python-qpid \ python-qpid-qmf qpid-cpp-client \ qpid-cpp-client-ssl qpid-cpp-server qpid-cpp-server-ssl \ qpid-qmf qpid-tools readline-devel saslwrapper sqlite-devel \ unixODBC unixODBC-devel uuid-perl wget xerces-c-devel xinetd
Once installed, check the following.
Java Version
The Java version must be 1.7.x. Check as following:
$ java -version java version "1.7.0_85" OpenJDK Runtime Environment (rhel-2.6.1.3.el6_6-x86_64 u85-b01) OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)
Ensure that the Java environment exists and points to your JDK installation.
$ echo $JAVA_HOME $ which java /usr/bin/java $ export JAVA_HOME=/usr/bin $
You should export JAVA_HOME in your .bashrc or .profile file.
Verify Trafodion Download
Verify that the Trafodion source has been either:
- Downloaded and unpackaged.
- Cloned from github.
If not, please do so now. Refer to Contributor Workflow - Code/Docs.
Install Required Build Tools
Refer to Required Build Tools for instructions.
Verify System Limits
Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.
ulimit –a core file size (blocks, -c) 1000000 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 515196 max locked memory (kbytes, -l) 49595556 max memory size (kbytes, -m) unlimited open files (-n) 32000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 267263 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
Please refer to this article for information on how you change system limits.
Build Trafodion
Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.
cd <download directory>/incubator-trafodion export TOOLSDIR=<tools installation directory> source ./env.sh
If you do not set up TOOLSDIR before sourcing in env.sh, then tools location defaults to /opt/home/tools. You may want to edit your .bashrc or .profile file to always export TOOLSDIR.
Build using one of the following options:
Command | What It Builds |
---|---|
make all | Trafodion, DCS, and REST. |
make package | Trafodion, DCS, REST, and Client Drivers. |
make package-all | Trafodion, DCS, REST, Client Drivers, and tests for all components. |
Verify the build:
$ sqvers -u
The output should show that seven jar files exist.
Setup Test Environment
You should test your installation using:
- A local Hadoop environment
- Trafodion installation
You're installation approach depends on whether you already have installed Hadoop.
If Hadoop is Already Installed
Build binary tar files and then install Trafodion following instructions described in Installation.
cd <download directory>/incubator-trafodion make package
The binary tar files will be created in <download directory>/incubator-trafodion/distribution directory.
Install a Local Hadoop Environment
The install_local_hadoop script downloads compatible versions of Hadoop, HBase, Hive, and MySQL. Then, it starts Trafodion.
Time Saver
install_local_hadoop downloads Hadoop, HBase, Hive, and MySql jar files from the internet. To avoid this overhead, you can download the required files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to this directory.
Prerequisite
You need to ensure that you've set up passwordless authentication.
ssh localhost The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is 88:f7:90:fe:35:76:10:92:cf:6d:c4:60:ca:b9:43:9f. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (RSA) to the list of known hosts. gtapper@localhost's password: Last login: Fri Nov 6 22:44:00 2015 from 192.168.1.9
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
Usage
Command | What It Does |
---|---|
install_local_hadoop | Uses default ports for all services. |
install_local_hadoop -p fromDisplay | Start Hadoop with a port number range determined from the DISPLAY environment variable. |
install_local_hadoop -p rand | Start with any random port number range between 9000 and 49000. |
install_local_hadoop -p <port > | Start with the specified port number. |
Default ports.
Port Variable | Default Value |
---|---|
MY_DCS_MASTER_PORT | 23400 |
MY_DCS_MASTER_INFO_PORT | 24400 |
MY_REST_SERVER_PORT | 4200 |
MY_REST_SERVER_SECURE_PORT | 4201 |
Sample Procedure
# Ensure that the Trafodion environmental variables have been loaded. cd <download directory>/incubator-trafodion source ./env.sh
Install the Hadoop software.
cd $MY_SQROOT/sql/scripts install_local_hadoop ./install_traf_components
Verify installation.
swstatus
Six java servers and two mysqld processes should be running.
Manage Hadoop Environment
Use the following commands to manage the Hadoop environment.
Command | Usage |
---|---|
swstartall | Start the complete Hadoop environment. |
swstopall | Stops the complete Hadoop environment. |
swstatus | Checks the status of the Hadoop environment. |
swuninstall_local_hadoop | Removes the Hadoop installation. |
Run Trafodion
This section describes how to start Trafodion and run operations.
Each Time New Source is Downloaded
You need to do the following each time you download new source code.
cd <download directory>/apache-trafodion-1.3.0-incubating source ./env.sh cd $MY_SQROOT/etc # delete ms.env, if it exists rm ms.env cd $MY_SQROOT/sql/scripts sqgen
Start Trafodion
Do the following to start the Trafodion environment.
cd $MY_SQROOT/sql/scripts sqstart sqcheck
Management Scripts
Component | Start | Stop | Status |
---|---|---|---|
All of Trafodion | sqstart | sqstop | sqcheck |
DCS (Database Connectivity Services) | dcstart | dcsstop | dcscheck |
REST Server | reststart | reststop | |
LOB Server | lobstart | lobstop | |
RMS Server | rmsstart | rmsstop | rmscheck |
Create Trafodion Metadata
# Ensure that the Trafodion environmental variables have been loaded. cd <download directory>/incubator-trafodion source ./env.sh
Assumption: Trafodion is up and running.
Use sqlci to create the Trafodion metadata.
$ sqlci >> initialize trafodion; . . . >> exit; $
Validate Your Installation
You can use sqlci or trafci (connects via DCS) to validate your installation.
get schemas; create table table1 (a int); invoke table1; insert into table1 values (1), (2), (3), (4); select * from table1; exit;
Assuming no errors, you're installation has been successful. You can start working on your modifications.