DiagnosisTool

Backgroud

When you want to get assistance from community or a Kylin expert, instead of sending he/her some screenshots which often contains partial information, sharing a diagnosis package maybe a better choice because almost everything which is important, including metadata, logs file, configuration files, yarn logs, hadoop components's version will collected in one click.

Diagnosis package

We divided diagnosis packages into two kind, Project Diagnosis Package and Job Diagnosis Package.
org.apache.kylin.tool.DiagnosisInfoCLI is for get a Project Diagnosis Package, and org.apache.kylin.tool.JobDiagnosisInfoCLI is for get a Job Diagnosis Package.
If you have any issue/failure when building a segment, please choose using the Job Diagnosis Package; otherwise, please choose the Project Diagnosis Package.

Content

[root@cdh-master diag_all_info]# tree -L 1
.
├── client                         ---> Operation System Level Information, and version of Hadoop Component
├── commit_SHA1                    ---> Which commit did you binary built from
├── conf                           ---> All configuration files from Kylin itself
├── hbase                          ---> HBase information, including HBase related HDFS files and HTable metadata, jmx and configuration
├── info                           ---> Metadata of this diagnosis package
├── jobs                           ---> Building Job 's output and parameters 
├── kylin_env                      ---> Version of Kylin
├── logs                           ---> The logs of Kylin and Tomcat
├── metadata                       ---> Backup of Kylin's metadata
├── spark                          ---> Spark's configuration files
└── yarn                           ---> Logs collected by yarn logs -applicationId xxx
8 directories, 3 files

How to use

There are two ways to get a diagnosis package, Web UI or Command Line. Please check http://kylin.apache.org/blog/2016/06/10/diagnosis-tool/ if you cannot find the right place in web page.

If you want to choose Command Line way, here is some examples(works after KYLIN-4425 - Getting issue details... STATUS  was introduced.)

# If you want to collected the most complete information for all project, you should choose this one. 
sh bin/diag.sh org.apache.kylin.tool.DiagnosisInfoCLI -project -all \
    -destDir diag_all_info


# If you want to collected most complete information for a specific project, you should choose this one.
sh bin/diag.sh org.apache.kylin.tool.DiagnosisInfoCLI -project learn_kylin \
    -destDir diag_learn_kylin


# If you only want to collect related metadata, logs files and configuration files, you should choose this one.
sh bin/diag.sh org.apache.kylin.tool.DiagnosisInfoCLI -project learn_kylin \
    -destDir diag_learn_kylin_one_day \
    -includeHBase false \
    -includeJobs false \
    -period 1
 

# If you have some building job failed but cannot find the cause, you may should try this. Yarn logs will be collected.
sh bin/diag.sh org.apache.kylin.tool.JobDiagnosisInfoCLI \
    -jobId 35ebd64b-d56c-7236-6bb6-58d63160b49d \
    -destDir diag_job

Tips

1. When you click the button in Web UI to download a diagnosis package and wait too much time, you may want find if there anything go wrong, please check the log content in $KYLIN_HOME/logs/kylin.out (that is the stand output). If you want to let the output print to other place, please consider modify $KYLIN_HOME/conf/kylin-tools-log4j.properties.
2. If you find it difficult to download a diagnosis package from web page, or maybe you want set some custom filter condition to reduce the size of diagnosis package, please consider using the CLI way, it is somehow flexible.
3. By default, DiagnosisInfoCLI will only collected logs files of recent three days, if you need to change it please use period option.

Disclaimer

All private information such as address, username or password will be kept as is and will not be hidden or deleted. Please be careful if you share a diagnosis package from a production Kylin instance.