The following features are targeted by Flume NG (FLUME-728). This list is still mutable. That said, the goal is to product a working core and incrementally build out the features necessary for robust data ingestion. A lack of a JIRA may indicate that work has not yet begun on a given feature.
Feature |
JIRA |
---|---|
General Features |
|
Source, Sink, Channel, Event Interfaces |
|
Source and Sink Runners (event-driven and pollable) |
|
Input and Output Format Interfaces / Serialization |
|
Configuration Interfaces |
|
Metrics collection infrastructure |
|
JMX mbean support |
|
Configuration |
|
File-based configuration system |
|
Centrally managed ZK based configuration system |
|
Channels / Reliability Features |
|
In-memory "best effort" channel |
|
JDBC-based durable channel |
FLUME-760 |
File-based durable channel |
|
Sources |
|
Avro RPC source |
|
Thrift RPC source |
|
Scribe compatibility source |
|
Exec source |
|
Tail source |
|
Sinks |
|
HDFS sink |
FLUME-761 |
FS sink |
|
HBase sink |
|
Specialized Adapters / Integration |
|
log4j -> avro source appender |
|
TODO