Code Analysis

livy integration

Griffin uses livy as job-submitter of spark, all jobs will be submitted via livy client.

In code base, we wrap livy client into . leverages to quartz scheduler and call livy client to submit jobs

DSL parser

Griffin gives a new sql-like language to define data quality computation, there is an implementation in measure project.

GriffinDslParser.scala is responsible for parsing rules in data quality.


Data sink is a term used to describe a computer or any other medium capable of receiving data.

Sink.scala is an abstract data sink type in griffin which deals with all data processed.