Status

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Flink already has quite a big amount of documentation, which is not always easy to find. We do still need more documentation around many aspects of the system, which will make it even tougher to find the appropriate documentation.

The lack ov information accessibility is due to two issues:

• Documentation is not well grouped into categories according to what is relevant in different steps of the development and deployment process
  
  
• The current layout has not enough top-level navigation menus to guide people that look for docs. Experience shows that anything hidden in a larger document linked from a sub-menu is overlooked by many people.

Proposed Changes

The proposal is to re-arrange the documentation into 5 categories.

Each category can have it’s own landing page and have its different parts as a top-level menu.

Concepts

• Overview: Similar to the current start page, stack and brief description
• Concepts: Similar to what we currently have
• Architecture: Process model (JM / TM / Client / …)
• Project Structure: Core Maven artifacts, what needed when, relationships
• Semantics

Setup & Operations

• Setup: Downloading, Building from Source, Hadoop Versions / Scala Versions
• Deployment Options: Yarn, Mesos, Standalone, etc
• Configuration
• Failure & Recovery Model
• Security Model
• Operations: Version Upgrades, etc

Application Development

• Quickstarts 
• APIs: Streaming, Batch, Table, SQL
• Event Time and Windowing Semantics
• Libraries
• State Backends
• Connectors, End-to-end Exactly-once
• Data Types: Types, Serialization (Custom, Kryo, Avro), Lambdas, Hints, Extensibility

Testing & Debugging

• Testing and test utilities
• Debugging
• Monitoring: Web Frontend, Metrics
• Tuning: Memory, CPU, GC

Internals

• how does it work

Rejected Alternatives

(none so far)