Child pages
  • How to profile Tez with Yourkit
Skip to end of metadata
Go to start of metadata
To profile DAGAppMaster, we can attach Yourkit agent to it by following these steps:
  1. Download Yourkit tar on each node of your cluster (since you don't know where AM will be launched) and extract files. 
  2. Add following property in tez-site.xml:
<property>
	<name>tez.am.launch.cmd-opts</name>
	<value>-agentpath:<absolute path of libyjpagent.so>=<yourkit options></value>
</property>

libyjpagent.so can be found in <Yourkit-dir>/bin/<architecture>/libyjpagent.so. 

Yourkit options can be found by running

java -agentpath:<path of libyjpagent.so>=help <some-java-class>

 

If you are capturing snapshot of DAGAppMaster, snapshot files can be found under your home dir in 'Snapshot' dir. If you submit YARN job as user yarn, you need go to /home/yarn/Snapshots.

 

Profiling task containers is similar to profiling DAGAppMaster, the only difference is you should use property 'tez.task.launch.cmd-opts' (for all tasks) or 'tez.task-specific.launch.cmd-opts' (for specific task) in tez-site.xml.

  • No labels