Child pages
  • How to profile Tez with Yourkit
Skip to end of metadata
Go to start of metadata
To profile DAGAppMaster, we can attach Yourkit agent to it by following these steps:
  1. Download Yourkit tar on each node of your cluster (since you don't know where AM will be launched) and extract files. 
  2. Add following property in tez-site.xml:

libyjpagent.so can be found in <Yourkit-dir>/bin/<architecture>/libyjpagent.so. 

Yourkit options can be found by running

 

If you are capturing snapshot of DAGAppMaster, snapshot files can be found under your home dir in 'Snapshot' dir. If you submit YARN job as user yarn, you need go to /home/yarn/Snapshots.

 

Profiling task containers is similar to profiling DAGAppMaster, the only difference is you should use property 'tez.task.launch.cmd-opts' (for all tasks) or 'tez.task-specific.launch.cmd-opts' (for specific task) in tez-site.xml.

  • No labels