Running Hama Over InfiniBand
- Java 7+ required.
Create an SDP configuration file
Each BSP peer (task) will have its own port number for communicating with others. For example, if you set set bsp.peer.port
to 61000 and set bsp.tasks.maximum
to 5, BSP peers will use the port range from 61000 to 61005.
Wiki Markup |
---|
To enable SDP^\[1\]^ support for BSP peers, create a configuration file like the following sample: |
No Format |
---|
[edward@192.168.10.1 ~]# cat {$HAMA_HOME}/conf/sdp.conf # Use SDP when binding to bind 192.168.10.1 61000-61010 # Use SDP when connecting to connect 192.168.10.0/24 61000-61010 |
NOTE: This configuration file must reside on every node.
Set the system property that specifies the location of the configuration file
And then, set the location of the configuration file by adding -Dcom.sun.sdp.conf
to the bsp.child.java.opts
property as following:
No Format |
---|
<property> <name>bsp.child.java.opts</name> <value>-Dcom.sun.sdp.conf={$HAMA_HOME}/conf/sdp.conf -Xmx1024m</value> <description>Java opts for the groom server child processes. The following symbol, if present, will be interpolated: <at:var at:name="taskid" /> is replaced by current TaskID. Any other occurrences of '@' will go unchanged. For example, to enable verbose gc logging to a file named for the taskid in /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of: -Xmx1024m -verbose:gc -Xloggc:/tmp/<at:var at:name="taskid" />.gc The configuration variable bsp.child.ulimit can be used to control the maximum virtual memory of the child processes. </description> </property> |
If everything is OK, you will see the following logs.
No Format |
---|
CONNECT to 192.168.10.2:61005 (socket converted to SDP protocol) CONNECT to 192.168.10.5:61002 (socket converted to SDP protocol) CONNECT to 192.168.10.4:61003 (socket converted to SDP protocol) CONNECT to 192.168.10.1:61006 (socket converted to SDP protocol) ... |