This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • How to Diagnose Tez App

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Profiling in Tez

It is possible to profile specific tasks in specific vertices in Tez. Examples are given below.

Example A: Use Xprof. Add "-Xprof" to tez.task.launch.cmd-opts, e.g.  tez.task.launch.cmd-opts="-Xprof".

Example B. Use yourkit on specific tasks via "tez.task-specific.launch.cmd-opts.list" and "tez.task-specific.launch.cmd-opts" options can be used for this. Examples are given below.. If your cluster doesn't have yourkit installed, you can config "tez.lib.uris" and "tez.lib.uris.classpath" so that the yourkit tar.gz is localized by yarn and added to the container classpath. With the below configuration, yourkit snapshot files will be created under yarn container log directory based on vertex name and task index. Then you can use "yarn logs" command to download the container log and exact the snapshot portion from the log.

  • tez.task-specific.launch.cmd-opts.list
    • Specifies the tasks in different vertices where additional options have to be specified
    • Examples:
      • tez.task-specific.launch.cmd-opts.list=“M5[0]" - Specifies task 0 in vertex M5
      • tez.task-specific.launch.cmd-opts.list=“Map10[5,20]" - Specifies task 5,20 in vertex Map 10
      • tez.task-specific.launch.cmd-opts.list=“M5[]" - Specifies all tasks in M5
      • tez.task-specific.launch.cmd-opts.list=“M5[1:3,10]" - Specifies 1,2,3,10 tasks in M5
        • However, defining partial ranges like "M5[:3]" is not yet supported.
      • tez.task-specific.launch.cmd-opts.list=“M5[0];V2[10]" - Specifies task 0 in vertex M5 and task 10 in vertex V2

  • tez.task-specific.launch.cmd-opts
    • Specifies the additional task specific JVM launch options that need to be added.
    • __VERTEX_NAME__ and __TASK_INDEX__ can be specified in the option, which would be replaced at runtime.
    • Examples:
      • tez.task-specific.launch.cmd-opts="-agentpath:/opt/yourkit/yjp-2016.02-b46-linux.tar.gz/yjp-2016.02/bin/linux-x86-64/libyjpagent.so=disablej2eedisablej2ee,tracing,alloceach=1000,onexit=snapshot,tracing_settings_path=/tmp/walltime.txt, dir=/tmp/sampling,disabletracing,disablealloc,periodicperf=60,dir=<LOG_DIR>/__VERTEX_NAME__/__TASK_INDEX__"
  • tez.lib.uris

    • Add the yourkit tar.gz HDFS path to tez.lib.uris. Example: tez.lib.uris=/lib/tez-0.9.0-SNAPSHOT-minimal.tar.gz,/lib/yjp-2016.02-b46-linux.tar.gz

  • tez.lib.uris.classpath

    • tez runtime tar.gz is defined in tez.lib.uris, config the classpath for the tez runtime tar.gz via tez.lib.uris.classpath=tez-0.9.0-SNAPSHOT-minimal.tar.gz/*,tez-0.9.0-SNAPSHOT-minimal.tar.gz/lib/.