DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.

Apache Kylin

Space shortcuts

Apache Kylin : Analytical Data Warehouse for Big Data

Page tree

Versions Compared

Old Version 26

changes.mady.by.user Xiaoxiang Yu

Saved on

compared with

New Version 27

changes.mady.by.user Xiaoxiang Yu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Advanced Tables - Table Plus
sortDescendingtrue
sortColumnPriority
autoNumberSorttrue
allowExporttrue
displayDataFiltertrue
sortIcontrue


NONONOtrueNOYESYESYESNONONONOtitleYEStitleNO
PropertyRequiredPriority
Importance
DatatypeConfiguration LevelDefaultDescriptionVersionReference
kylin.engine.spark.build-class-name
Status
subtletrue
title
 Status
subtletrue
titleMinor
String
 Status
subtletrue
titleProcess
org.apache.kylin.engine.spark.job.CubeBuildJob
For developer only. The className use in spark-submit.
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.engine.spark.cluster-info-fetcher-class-name
 Status
subtletrue
title
 Status
subtletrue
titleMinor
String
 Status
subtletrue
titleProcess
org.apache.kylin.cluster.YarnInfoFetcher
For developer only. Fetch yarn information of spark job
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.engine.spark-conf.XXX
 Status
subtletrue
title
 Status
subtletrue
titleMinor
String
 Status
subtletrue
titleProcess
NullSpark configurations want to override for build job like "spark.driver.cores". If don't set these spark properties, kylin will automaticly adjust these properties before submitting build job. 
 Status
subtletrue
colourBlue
title4.0.0-alpha
Adaptively-adjust-spark-parameters
kylin.storage.provider
 Status
subtletrue
title
NO
status
subtletrue
titleMinor
String
 Status
subtletrue
titleProcess
org.apache.kylin.common.storage.DefaultStorageProvider
The content summary objects returned by different cloud vendors are not the same, so need to provide targeted implementation.
You can refer to this to learn more : org.apache.kylin.common.storage.IStorageProvider
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.engine.spark.merge-class-name
 Status
subtletrue
title
NO
 Status
subtletrue
titleMinor
String
 Status
subtletrue
titleProcess
org.apache.kylin.engine.spark.job.CubeMergeJob
For developer only. The className use in spark-submit
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.engine.spark.task-impact-instance-enabled
 Status
subtle
titleNO

Boolean
 Status
subtletrue
titleProcess
trueCheck kylin.engine.spark.task-core-factor. If kylin.engine.spark.task-impact-instance-enabled is set to true and kylin.engine.spark-conf.spark.executor.instances is not set, Kylin will calculate spark.executor.instances for Build Engine.
 Status
subtletrue
colourBlue
title4.0.0-alpha
Adaptively-adjust-spark-parameters

kylin.engine.spark.task-core-factor
 Status
subtletrue
colourYellow
title
Medium
Integer
 Status
subtletrue
titleProcess
3


kylin.engine.driver-memory-base
 Status
subtletrue
colourYellow
title
Medium
Integer
 Status
subtletrue
titleProcess
1024Auto adujst spark.driver.memory for Build Engine if kylin.engine.spark-conf.spark.driver.memory is not set.



 Status
subtletrue
colourBlue
title4.0.0-alpha
Adaptively-adjust-spark-parameters
kylin.engine.driver-memory-strategy
 Status
subtletrue
colourYellow
title
Medium
Array
 Status
subtletrue
titleProcess
{"2", "20", "100"}



kylin.engine.driver-memory-maximum
 Status
subtletrue
colourYellow
title
Medium
Integer
 Status
subtletrue
titleProcess
4096


kylin.engine.persist-flattable-threshold
 Status
subtletrue
colourYellow
title
Medium
Integer
 Status
subtletrue
titleProcess
1If the number of cuboids which will be build from flat table is bigger than this threshold, the flat table will be persisted into $HDFS_WORKING_DIR/job_tmp/flat_table for saving more memory.
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.snapshot.parallel-build-timeout-seconds
 Status
subtletrue
title
 Status
subtletrue
colourRed
titleMAJOR

 Status
subtletrue
titleProcess
3600
To improve the speed of snapshot build.

 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.snapshot.parallel-build-enabled
 Status
subtletrue
title
 Status
subtletrue
colourRed
titleMAJOR
Boolean
 Status
subtletrue
titleProcess
true





 Status
subtletrue
titleProcess




kylin.spark-conf.auto.prior
 Status
subtletrue
title
 Status
subtletrue
titleMinor
Boolean
 Status
subtletrue
titleProcess
true If need to adjust spark parameters adaptively.
 Status
subtletrue
colourBlue
title4.0.0-alpha
Adaptively-adjust-spark-parameters
kylin.engine.submit-hadoop-conf-dir
 Status
subtletrue
colourYellow
YES
 Status
subtletrue
colourRed
titleMAJOR
String
 Status
subtletrue
titleProcess
/etc/hadoop/conf
Set HADOOP_CONF_DIR for spark-submit.
 Status
subtletrue
colourBlue
title4.0.0-alpha

kylin.storage.columnar.shard-size-mb
 Status
subtletrue
colour
Yellow
titleYES
 Status
subtletrue
colourRed
titleMAJOR
Integer
 Status
subtletrue
colourRed
titleCube
128The size of each parquet partition file of cuboid
 Status
subtletrue
colourBlue
title4.0.0-alpha
ShardBy
kylin.storage.columnar.shard-rowcount
 Status
subtletrue
colour
Yellowtitle
 Status
subtletrue
colourRed
titleMAJOR
Long
 Status
subtletrue
colourRed
titleCube
2500000
The max rows of each parquet partition file of cuboid

kylin.storage.columnar.shard-countdistinct-rowcount
 Status
subtletrue
colour
Yellow
YES
 Status
subtletrue
colourRed
titleMAJOR
Long
 Status
subtletrue
colourRed
titleCube
1000000The number rows of each parquet partition file of cuboid when the shard column is distinct column.

kylin.query.spark-engine.join-memory-fraction
 Status
subtletrue
colourYellow
title
Medium
Double
 Status
subtletrue
titleProcess
0.3Limit memory used by broadcast join.
 Status
subtletrue
colourBlue
title4.0.0-alpha


...