Page tree
Skip to end of metadata
Go to start of metadata

This page describes how to build Impala from source and how to configure and run Impala in a single node development environment. 

Prerequisites and Requirements

Hardware Requirements

  • CPU must support at least SSSE3
  • Minimum memory: 16GB
  • Hard disk space: 120GB (for the test data)

Supported Operating Systems

Linux

  • Ubuntu 14.04, 16.04

OS X

Not supported

Windows

Not supported

Installing and Configuring Impala

  1. Install prerequisites


    (warning) Instead awleblang/impala-setup, as of Dec 2017, we recommend https://github.com/apache/impala/blob/master/bin/bootstrap_development.sh to setup your Ubuntu 16.04 or Ubuntu 14.04 machine. By default, bootstrap_development.sh clones Impala in $HOME/Impala. To build Impala in a different directory, set IMPALA_HOME when running bootstrap_development.sh. For example IMPALA_HOME=`pwd` ./bootstrap_development.sh

    Older instructions:
    git clone https://github.com/awleblang/impala-setup

    cd impala-setup
    chmod +x install.sh
    sudo ./install.sh

     

    If you are running on a platform that is not supported by the above script, you will have to install prerequisites and set up your machine manually. Impala Build Prerequisites has some information that can guide you.

  2. Clone the Impala git repo and change to the new Impala directory.

    git clone https://git-wip-us.apache.org/repos/asf/impala.git Impala
    cd Impala
  3. Set the following environment variables in your `.bashrc` or elsewhere:

    export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
    export IMPALA_HOME=<path to Impala>
    export BOOST_LIBRARYDIR=/usr/lib/x86_64-linux-gnu
    export LC_ALL="en_US.UTF-8"
  4. Build Impala.

    # Build Impala and prepare the local mini-cluster (the first time only)
    ${IMPALA_HOME}/buildall.sh -noclean -skiptests -build_shared_libs -format

    You may hit an error here in bootstrap_toolchain.py if you are building Impala on an operating system for which prebuilt toolchain binaries are not available. In that case you will need to build native-toolchain from scratch then retry this step.

    Want faster builds?

    Doing a full build of Impala can take quite a while! While you wait, check out Tips for Faster Impala Builds.

  5. Start supporting services.

    # This script starts or restarts the local services required for the test cluster
    # including HDFS, HBase, Hive and ZooKeeper. If you hit a problem with this script,
    # check the log files in ${IMPALA_HOME}/cluster_logs/ for clues.
    ${IMPALA_HOME}/buildall.sh -noclean -notests -build_shared_libs -start_minicluster
  6. Start the Impala cluster.

    source ${IMPALA_HOME}/bin/impala-config.sh # You must source this in your shell before most of the below commands will work.
    
    # The following command will start an Impala cluster with 3 Impala demons, one 
    # Statestore and one Catalog demon.
    ${IMPALA_HOME}/bin/start-impala-cluster.py
  7. Check that everything works correctly.

    source ${IMPALA_HOME}/bin/impala-config.sh # If you didn't already source impala-config.sh in this shell
    
    impala-shell.sh -q "SELECT version()"
    Starting Impala Shell without Kerberos authentication
    Connected to localhost:21000
    Server version: impalad version 2.2.0-INTERNAL DEBUG (build 47c90e004aecb928a37b926080098d30b96b4330)
    Query: select version()
    +---------------------------------------------------------------------------------------+
    | version()                                                                             |
    +---------------------------------------------------------------------------------------+
    | impalad version 2.2.0-INTERNAL DEBUG (build 47c90e004aecb928a37b926080098d30b96b4330) |
    | Built on Sun, Mar 22 15:22:57 PDT 2015                                                |
    +---------------------------------------------------------------------------------------+
    Fetched 1 row(s) in 0.05s
  8. Have fun...
  9. Recompile and restart the Impala cluster for your changes to take effect.

    # Rebuild both backend and frontend
    ${IMPALA_HOME}/buildall.sh -skiptests -build_shared_libs -noclean
    
    source ${IMPALA_HOME}/bin/impala-config.sh # If you didn't already source impala-config.sh in this shell
     # Optional: Build the backend only
    ${IMPALA_HOME}/bin/make_[debug|release].sh [-notests]
    # Optional: Build the Java-side frontend only
    cd ${IMPALA_HOME}/fe && mvn clean package dependency:copy-dependencies -DskipTests=true
    # Restart the Impala cluster
    ${IMPALA_HOME}/bin/start-impala-cluster.py

 

 

  • No labels

2 Comments

  1. In my opinion this page is made obsolete by bin/bootstrap_development.sh. Any objection to deleting it?

    1. I think this is still somewhat useful in that it provides some additional explanation of what the build does and workflow after the first full build (it's not great for that, but better than nothing). I'd probably prefer to leave it until we can strip away the unnecessary stuff and make it better focused on steps after bootstrapping.