To profile DAGAppMaster, we can attach Yourkit agent to it by following these steps:
  1. Download Yourkit tar on each node of your cluster (since you don't know where AM will be launched) and extract files. 
  2. Add following property in tez-site.xml:
	<value>-agentpath:<absolute path of>=<yourkit options></value>
</property> can be found in <Yourkit-dir>/bin/<architecture>/ 

Yourkit options can be found by running

java -agentpath:<path of>=help <some-java-class>


If you are capturing snapshot of DAGAppMaster, snapshot files can be found under your home dir in 'Snapshot' dir. If you submit YARN job as user yarn, you need go to /home/yarn/Snapshots.


Profiling task containers is similar to profiling DAGAppMaster, the only difference is you should use property 'tez.task.launch.cmd-opts' (for all tasks) or 'tez.task-specific.launch.cmd-opts' (for specific task) in tez-site.xml.

  • No labels