This page should help to collect feature requests from users / or any other improvements for existing features. We can then use this collection to discuss them on the mailing list - especially when needing more in-depth conceptual work.
This page should not replace Jira to track issues or any discussions on the mailing list.
Features/Improvements in focus:
Prio | Feature/Improvement | Jira Issue | Created by | Discussed on mailing list | Comment |
---|---|---|---|---|---|
1 | Python Wrapper | Patrick | yes | A new python wrapper for creating data processors | |
2 | Platform Services | API to interact with live/historical data, pipelines and pipeline elements | |||
3 | Pipeline Monitoring | Dominik | Get simple pipeline monitoring info, e.g., processed events per processor/sink, lag |
Loose collection:
- Edge Deployment: Allow advanced deployment options, e.g. to assign pipeline elements (standalone) to individual nodes
- StreamPipes Client: Define/create pipelines from code, which are automatically deployed in StreamPipes
- Fault Tolerance: Better support for failure handling/resiliency/ state management and state recovery
- Pipeline Monitoring/Statistics: Inspect current pipeline execution state, receive statistics (e.g., processed messages) and see errors
- Unified Data Visualization: Use common API for live + historic data visualization (e.g. only using time-series DB for historical data and polling to retrieve near realtime event)
- Event/Configuration Preview: View current events in Pipeline Editor with application logic applied based on current configuration of static properties of pipeline elements up to this point
- Pipeline adaptation: Manipulate pipeline configurations at runtime, e.g. static properties etc, either by an API or via trigger in visualizations (Visual Analytics), e.g. slider to set new active threshold value
- Pipeline Triggers: A pipeline can either be started manually or via a trigger. The latter can be based on a user-defined scheduled time/day, e.g. no production during night shift or based on an API call.
- Virtual Streams - replace data lake and dashboard sinks with a "virtual stream" that is also shown in the "data streams" section. Dashboard and data lake would then simply operate on the notion of streams instead of pipelines.