- Provide the ability for reference-able groups of processors and relationships which are as natural to use as a processor.
Background and strategic fit
Process Groups enable powerful composition of processors and relationships with specific input ports and output ports. However, they live exclusively at a specific section of the dataflow and the only way to send data to them is by way of explicit relationship from a source component to an input port of that process group. Further, the only way to get data out of them is by direct connection from one their output ports to a given component. Thus, if you want to be able to reuse that same instance of a process group for multiple flows you have to keep the necessary context around it by using something like UpdateAttribute on input to tag data and RouteOnAttribute afterwards to route on that tag. This can be awkward and can lead to dataflows being more complicated than necessary. As a result rather than leveraging a single process group for a variety of dataflows folks tend to create templates of that group and have many instances of it. This in turn can cause wasted resources.
So, this concept of 'reference-able process groups' means that users could instead refer to a special type of Process Group by perhaps dragging a reference to that Process Group onto their flow wherever they need to. They would establish connections to its input port just like they would when connecting to a processor and they would establish connections from its output ports to other components just like you would with processor relationships. In essence what this means is that you can compose 'new processors' simply be creating reference-able Process Groups. This then becomes like a function reference mechanism. This could greatly reduce the complexity of the flow visually. These groups would be special in a couple of ways. First when sending data to them and in directing their output the framework itself would automatically keep track of its context of use. Users would just connect to and from them naturally and even though it is a single Process Group being referenced the framework will know where in the actual flow data comes from and goes to. Second, these special Process Groups would not really live at any specific point in the dataflow.
- This concept will require some significant consideration for User Experience to avoid confusing the user about which type of Process Group they have.
User interaction and design
Below is a list of questions to be addressed as a result of this requirements document: