DUCC Project Documentation
Tutorials
Quick Start
Audience: DUCC Users/Administrators
Instructions on how to fetch, build and install DUCC 1.0 for single host system (my laptop!)
See DUCC Project Documentation for further details.
The following steps were performed on Red Hat Enterprise Linux Workstation release 6.4 (Santiago)
Prerequisites
- Install svn
- Install mvn
- Install jdk (1.6 or better)
export JAVA_HOME=~/apps/java/java-home - Install latex
- Install ant
Start sshd
[degenaro]$ sudo su root [root]# chkconfig sshd on [root]# /sbin/service sshd restart [root]# exit [degenaro]$
Create user ducc
[degenaro]$ sudo adduser ducc
Permit universal read+execute access to home directory for users degenaro and ducc
[degenaro]$ sudo su root [root]# cd /home [root]# chmod 755 degenaro [root]# chmod 755 ducc [root]# exit [degenaro]$
Enable passwordless ssh for user ducc
[degenaro]$ sudo su ducc [ducc]$ cd [ducc]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/ducc/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/ducc/.ssh/id_rsa. Your public key has been saved in /home/ducc/.ssh/id_rsa.pub. The key fingerprint is: 4f:4a:81:60:c9:a5:d8:88:80:f3:6c:a1:ef:31:8b:8e ducc@hal9000.apache.com The key's randomart image is: +--[ RSA 2048]----+ |o .oo. | |+..=oo . | |.=o.o . . | |. + . | | o S . | | + . + | | o + . . | |o o | |E. | +-----------------+ [ducc]$ cd .ssh [ducc]$ cat id_rsa.pub >> authorized_keys2 [ducc]$ chmod 600 * [ducc]$ exit [degenaro]$
Insure JAVA_HOME is set, and PATH contains LaTeX
Sample code to set JAVA_HOME
.bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
export JAVA_HOME=~/apps/java/java-home
export PATH=$PATH:/usr/local/tex
Fetch DUCC from svn
[degenaro]$ svn co https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk ~/svn/ducc/trunk
Build DUCC jars
[degenaro]$ cd ~/svn/ducc/trunk [degenaro]$ mvn clean install -Pbuild-duccdocs
Install
[degenaro]$ cd target [degenaro]$ sudo su ducc [ducc]$ tar -zxf uima-ducc-1.0.0-SNAPSHOT-bin.tar.gz -C ~ducc [ducc]$ cd ~ [ducc]$ ln -s apache-uima-ducc-1.0.0-SNAPSHOT ducc_install [ducc]$ cd ~ducc/ducc_install/admin [ducc]$ ./ducc_post_install
Check
[ducc]$ cd ~ducc/ducc_install/admin [ducc]$ ./check_ducc -s [ducc]$ exit [degenaro]$
Optional: enable ducc_ling and login to Web-server via Linux authentication
[degenaro]$ sudo su root [root]$ mkdir /local [root]$ mkdir /local/ducc [root]$ mkdir /local/ducc/bin [root]$ chown ducc.ducc /local/ducc [root]$ chown ducc.ducc /local/ducc/bin [root]$ chmod 700 /local/ducc [root]$ chmod 700 /local/ducc/bin [root]$ cp ~ducc/ducc_install/duccling/src/ducc_ling /local/ducc/bin [root]$ chown root.ducc /local/ducc/bin/ducc_ling [root]$ chmod 4750 /local/ducc/bin/ducc_ling [root]$ exit [degenaro]$
[degenaro]$ sudo su ducc
[ducc]$ cd ~ducc/ducc_install/resources
1. edit ducc.properties
2. locate: ducc.ws.login.enabled = false
3. modify: ducc.ws.login.enabled = true
4. locate: ducc.agent.launcher.ducc_spawn_path=${DUCC_HOME}/admin/ducc_ling
5. modify: ducc.agent.launcher.ducc_spawn_path=/local/ducc/bin/ducc_ling
6. save
Start
[ducc]$ cd ~ducc/ducc_install/admin [ducc]$ ./start_ducc -s [ducc]$ exit [degenaro]$
ducc-mon System Daemons
Start your favorite browser (firefox, chrome, opera) and visit the ducc-mon system daemons page.
http://localhost:42133/system.daemons.jsp
Submit a test Job
[degenaro]$ cd ~ducc/ducc_install/bin [degenaro]$ export PATH=$PATH:$PWD [degenaro]$ cd ~ [degenaro]$ mkdir ducc [degenaro]$ cd ducc [degenaro]$ cp -pR ~ducc/ducc_install/examples . [degenaro]$ cd examples/simple [degenaro]$ ducc_submit -f 1.job Job 1 submitted [degenaro]$
ducc-mon Jobs
Visit the ducc-mon jobs page.
http://localhost:42133/jobs.jsp
End.
Install WebServer Demo
Audience: DUCC Developers
Copy DUCC binary tar-ball to installation host
[degenaro]$ scp uima-ducc-1.0.0-bin.tar.gz degenaro@uima-ducc-vm.apache.org:~degenaro
Login to installation host
Note: Developers who are presently unable to login should request access via uima-dev mailing list
[degenaro@oc0883280806 Downloads]$ ssh degenaro@uima-ducc-vm.apache.org ... degenaro@uima-ducc-vm:~$
Stop ducc (if running)
degenaro@uima-ducc-vm:~$ cd ~/ducc/install/admin degenaro@uima-ducc-vm:~/ducc/install/admin$ ./stop_ducc -a ... .Stopping broker: localhost .. FINISHED
Install new code such that ~/ducc/install is the $DUCC_HOME. For example, extract the downloaded zip to the versions folder...
degenaro@uima-ducc-vm:~/ducc/versions$ ls apache-uima-ducc-1.0.0 apache-uima-ducc-1.0.0.rc4
...and update the symbolic link
degenaro@uima-ducc-vm:~/ducc$ ls -atl total 12 drwxr-xr-x 3 degenaro degenaro 4096 Jan 16 16:32 . drwxr-xr-x 18 degenaro degenaro 4096 Jan 16 16:04 .. drwxr-xr-x 4 degenaro degenaro 4096 Jan 16 14:48 versions lrwxrwxrwx 1 degenaro degenaro 32 Jan 7 18:48 install -> versions/apache-uima-ducc-1.0.0/
Run ducc_post_install
degenaro@uima-ducc-vm:~/ducc/install/admin$ ./ducc_post_install Python version: 2.7.3 (default, Sep 26 2013, 20:03:06) [GCC 4.6.3] Ducc head is configured as uima-ducc-vm Initial /home/degenaro/ducc/versions/1.0.0/rc10/apache-uima-ducc-1.0.0/resources/ducc.nodes created. Initial nodepool /home/degenaro/ducc/versions/1.0.0/rc10/apache-uima-ducc-1.0.0/resources/jobdriver.nodes created. ActiveMQ is automanaged on node tcp://uima-ducc-vm:61617?wireFormat.maxInactivityDuration=0&jms.useCompression=true&closeAsync=false Initialized property "ducc.jvm" to /home/degenaro/local/sun/jdk1.7.0_45/bin/java JAVA: java version "1.7.0_45" JAVA: Java(TM) SE Runtime Environment (build 1.7.0_45-b18) JAVA: Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) Java version: "1.7.0_45" Java is verified. WARNING WARNING The default ssl pw of quackquack is configured. For security, change the property ducc.ws.port.ssl.pw to a secure password in ducc.properties and rerun ducc_post_install. WARNING None keystore created. Web server keystore generated from ducc.properties DUCC book installed into webserver root cc -g ducc_ling.c -o ducc_ling Initial ducc_ling is installed. See the installation guide for multi-user setup.
Install local modifications
degenaro@uima-ducc-vm:~/ducc.local-mods$ ls admin ducc.py degenaro@uima-ducc-vm:~/ducc.local-mods$ cp -p admin/ducc.py ~/ducc/install/admin/ degenaro@uima-ducc-vm:~/ducc.local-mods$ ls resources/ ducc.properties degenaro@uima-ducc-vm:~/ducc.local-mods$ cp -p resources/ducc.properties ~/ducc/install/resources/
Launch DUCC
degenaro@uima-ducc-vm:~/ducc/install/admin$ ./start_ducc -s
ENV: Java is configured as: /home/degenaro/usr/bin/java
...
uima-ducc-vm
Single user mode: bypassing ducc_ling checks.
DUCC Agent started PID 10601
All threads returned
Create user logs directory
degenaro@uima-ducc-vm:~$ cd ~/ducc/user/logs/1.0.0 degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0$ mkdir rc10 degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0$ cd rc10 degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0/rc10$ mkdir logs degenaro@uima-ducc-vm:~/ducc/user/logs/1.0.0/rc10$ cd ~/ducc degenaro@uima-ducc-vm:~/ducc$ rm logs degenaro@uima-ducc-vm:~/ducc$ ln -s user/logs/1.0.0/rc10/logs logs
Create and populate user examples directory
degenaro@uima-ducc-vm:~$ cd ~/ducc/user/examples/1.0.0 degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0$ mkdir rc10 degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0$ cd rc10 degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0/rc10$ mkdir examples degenaro@uima-ducc-vm:~/ducc/user/examples/1.0.0/rc10$ cd ~/ducc degenaro@uima-ducc-vm:~/ducc$ rm examples degenaro@uima-ducc-vm:~/ducc$ ln -s user/examples/1.0.0/rc10/examples examples degenaro@uima-ducc-vm:~$ cd ~/ducc/examples degenaro@uima-ducc-vm:~/ducc/examples$ cp -pR ~/ducc/install/examples/* .
Submit a test Job
[degenaro]$ cd ~/ducc/install/bin [degenaro]$ export PATH=$PATH:$PWD [degenaro]$ cd ~/ducc/examples/simple [degenaro]$ ducc_submit -f 1.job Job 1 submitted [degenaro]$
Visit uima-ducc-vm.apache.org:42133
End.