...
| Panel |
|---|
| Tabs Container |
|---|
| id | build-tabs |
|---|
| direction | horizontal |
|---|
| | Tabs Page |
|---|
| id | macos |
|---|
| title | MacOS (10.10+) |
|---|
| Install dependencies on MAC (with xcode installed)| Code Block |
|---|
|
brew install protobuf protobuf-c Gsasl openssl boost thrift json-c ccache snappy libyaml libevent python
brew tap brona/iproute2mac
brew install iproute2mac
brew install postgresql
brew install cmake
sudo pip install pygresql==4.0 (make sure specified version 4.0)
sudo pip install unittest2 pycrypto lockfile paramiko psi
sudo pip install http://sourceforge.net/projects/pychecker/files/pychecker/0.8.19/pychecker-0.8.19.tar.gz/download
sudo pip install http://darcs.idyll.org/~t/projects/figleaf-0.6.1.tar.gz
brew uninstall postgresql
|
OS requirement- Use a text editor to edit the /etc/sysctl.conf file. Add or edit each of the following parameter definitions to set the required value.
kern.sysv.shmmax=2147483648 kern.sysv.shmmin=1 kern.sysv.shmmni=64 kern.sysv.shmseg=16 kern.sysv.shmall=524288 kern.maxfiles=65535 kern.maxfilesperproc=65536
- Restart to apply the change
|
| Tabs Page |
|---|
| id | tab-yum |
|---|
| title | Redhat/Centos 7.X |
|---|
| Install dependencies on CentOS7.xDependencies(CentOS7 user can follow easy steps provided by Zhanwei Wang) | Code Block |
|---|
| curl -L "https://bintray.com/wangzw/rpm/rpm" -o /etc/yum.repos.d/bintray-wangzw-rpm.repo
yum install -y epel-release
yum makecache
yum install -y man passwd sudo tar which git mlocate links make bzip2 net-tools \
autoconf automake libtool m4 gcc gcc-c++ gdb bison flex cmake gperf maven indent \
libuuid-devel krb5-devel libgsasl-devel expat-devel libxml2-devel \
perl-ExtUtils-Embed pam-devel python-devel libcurl-devel snappy-devel \
thrift-devel libyaml-devel libevent-devel bzip2-devel openssl-devel \
openldap-devel protobuf-devel readline-devel net-snmp-devel apr-devel \
libesmtp-devel xerces-c-devel python-pip json-c-devel libhdfs3-devel \
apache-ivy java-1.7.0-openjdk-devel \
openssh-clients openssh-server
yum install -y postgresql-devel
pip --retries=50 --timeout=300 install pg8000 simplejson unittest2 pycrypto pygresql pyyaml lockfile paramiko psi
pip --retries=50 --timeout=300 install http://darcs.idyll.org/~t/projects/figleaf-0.6.1.tar.gz
pip --retries=50 --timeout=300 install http://sourceforge.net/projects/pychecker/files/pychecker/0.8.19/pychecker-0.8.19.tar.gz/download
yum erase -y postgresql postgresql-libs postgresql-devel |
OS requirement kernel.shmmax = 1000000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 0
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 200000
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1281 65535
net.core.netdev_max_backlog = 200000
vm.overcommit_memory = 2
fs.nr_open = 3000000
kernel.threads-max = 798720
kernel.pid_max = 798720
# increase network
net.core.rmem_max=2097152
net.core.wmem_max=2097152- Execute the following command to apply your updated /etc/sysctl.conf file to the operating system configuration:
sysctl -p
- Use a text editor to edit the /etc/security/limits.conf file. Add the following definitions in the exact order that they are listed
* soft nofile 2900000
* hard nofile 2900000
* soft nproc 131072
* hard nproc 131072 |
| Tabs Page |
|---|
| title | Redhat/Centos 6.x |
|---|
| class | tabs-compile-dependencies |
|---|
| Build dependencies yourself ( tested on redhat 6.x).Dependencies There are several dependencies (see the following table) you must install before building HAWQ. To build Apache HAWQ, gcc and some dependencies are needed. The libraries are tested on the given versions. Most of the dependencies can be installed through yum. Other dependencies should be installed through the source tarball. Typically you can use "./configure && make && make install" to install from source tarball. Libraries that must be installed using source tarball.
Libraries that can be installed through yum. | Name | Versoin |
|---|
| epel-release | 6-8 | | make | 3.81
| | gcc | 4.4.7
| | gcc-c++ | 4.4.7
| gperf
| 3.0.4
| snappy-devel
| 1.1.3
| bzip2-devel
| 1.0.6
| python-devel
| 2.6.2
| libevent-devel
| 1.4.6
| krb5-devel
| 1.11.3
| libuuid-devel
| 2.26.2
| libgsasl-devel
| 1.8.0
| libxml2-devel
| 2.7.8
| zlib-devel
| 1.2.3
| readline-devel
| 6
| openssl-devel
| 0.9.8
| bison-devel
| 2.5
| apr-devel
| 1.2.12
| libyaml-devel
| 0.1.1
| flex-devel
| 2.5.35
|
OS requirement kernel.shmmax = 1000000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 0
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 200000
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1281 65535
net.core.netdev_max_backlog = 200000
vm.overcommit_memory = 2
fs.nr_open = 3000000
kernel.threads-max = 798720
kernel.pid_max = 798720
# increase network
net.core.rmem_max=2097152
net.core.wmem_max=2097152- Execute the following command to apply your updated /etc/sysctl.conf file to the operating system configuration:
sysctl -p
- Use a text editor to edit the /etc/security/limits.conf file. Add the following definitions in the exact order that they are listed
* soft nofile 2900000
* hard nofile 2900000
* soft nproc 131072
* hard nproc 131072 |
| Tabs Page |
|---|
| id | docker-tab |
|---|
| title | Docker (Redhat/Centos 7.X) |
|---|
| Build with Prebuilt Docker Image Probably the simplest way to get started with the build is starting with the community developed docker image with all the project dependencies pre-installed. To use the docker image follow the steps on: https://hub.docker.com/r/mayjojo/hawq-devel/ |
|
|
Install
...
Hadoop
Please follow the steps here: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html
HDFS is a must, but YARN is optional. YARN is only needed when you want to use YARN as the global resource manager.
Get the HAWQ code and Compile
...
- To install Apache HAWQ, run command
Test In HAWQ
Init/Start/Stop Apache HAWQ
- Before initializing HAWQ, you need to install HDFS and make sure it works
- Unit test. To do unit test, go to the src/backend and run unittest.
cd src/backend
make unittest-check
- Installcheck-good test. After installing HAWQ, please ensure HDFS work before initializing HAWQ.
- source /install/dir/greenplum_path.sh
- hawq init cluster
make installcheck-good
Install YARN (Optional)
If you want to integrate with YARN for resource management, you need to install YARN first.
Init and Start/Stop Apache HAWQ
- Before initializing HAWQ, you need to install HDFS and make sure it works
- source /install/dir/greenplum_path.sh
- hawq init cluster (after initialization, HAWQ is started by default)
- Now you can stop/restart/start the cluster by using: hawq stop/restart/start cluster
Connect and Run basic queries
- psql -d postgres
- create table t ( i int );
- insert into t values(1);
- select * from t;
...
- (after initialization, HAWQ is started by default)
- Now you can stop/restart/start the cluster by using: hawq stop/restart/start cluster
Connect and Run basic queries
- psql -d postgres
- create table t ( i int );
- insert into t values(1);
- select * from t;
Test In HAWQ
- Unit test. To do unit test, go to the src/backend and run unittest.
cd src/backend
make unittest-check
- Installcheck-good test. After installing HAWQ, please ensure HDFS work before initializing HAWQ.