Date: Thu, 28 Mar 2024 10:35:09 +0000 (UTC) Message-ID: <643439600.67892.1711622109502@cwiki-he-fi.apache.org> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_67891_311272669.1711622109501" ------=_Part_67891_311272669.1711622109501 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
Note: This document is considered a living document and will evo= lve as the community guides the development of the effort moving forward. T= he initial content is based heavily around the Java agent and scope will la= ter grow to increase other agents.
Given the commonalities between the MiNiFi effort and that of NiFi, a si= milar structure will be provided for the Java version, as a Maven project, = comprised of:
The following guidelines represent a list of guidelines tha=
t may be useful for MiNiFI C++ development and design. These stem from the =
environments which
are potential targets. Their basis stems from typical C++ development=
practices and design goals that lend toward maximizing development efforts=
.
Key | Summary | T | Created | Updated | Due | Assignee | Reporter | P | Status | Resolution |
---|
Agents will have a defined taxonomy and capabilities associated wi= th them. These properties will aid in the agent being able to communicate w= hat items are possible and aid flow designers in the process of creating fl= ows for various agent classes. Said capabilities will be communicated with = a manager for the sake of understanding what is possible with v= arious agents. Capabilities and capacities may change over time and this in= formation will be continually registered with associated systems
Longer term, agents should be able to convey their capabilities as a res= ult of items such as environment, version of software, networking, and hard= ware for establishing configuration of flow and collected data from a manag= er perspective.
Primarily handles the bootstrapping of the process and the configuration= of the JVM which is monitoring and controlling the flow process. Thi= s will receive configuration changes and affect the associated flow process= to provide these updates.
Key | Summary | T | Created | Updated | Due | Assignee | Reporter | P | Status | Resolution |
---|
The FlowFile format has been the core serialization format o= f NiFi and provides structure that allow for ease of files traversing a giv= en flow and exploit pass by reference semantics in routing operations. &nbs= p;Of interest is the handling of information with the core FlowFile format = as metadata is transmitted from the agent to a receiving node/system.  = ;This may be out of band or as an augmentation to the FlowFIle format. &nbs= p;
Key | Summary | T | Created | Updated | Due | Assignee | Reporter | P | Status | Resolution |
---|
Provides a means for introducing data into the system and currentl= y maps data to existing processors in the system. Given the desire to make = use of existing libraries and functionalities when developing the initial a= gent offering, focus will be provided to the core use cases, mapping to exi= sting processors, this would be comprised of:
Egress is viewed as high level terminology for getting data from an agen= t to an associated system. The complexity and needs for this functionality = may vary across environments and may have complex networking schemes requir= ed
For the existing proof of concept and for establishing an agent to make = larger architectural decisions, the Java agent can make use of the existing Site to Site protocol and fun= ctionality to communicate with an endpoint system.
Key | Summary | T | Created | Updated | Due | Assignee | Reporter | P | Status | Resolution |
---|
A lightweight process, capable of being constructed for acquiring = information from a host system(s) and providing this information to another= system for consumption. This process provides provenance, a directed graph= of processing, and extensibility to map to various data formats, schemas, = and protocols.
Functionality that a given agent is able to perform. In some conte= xts, this may be communicating with specific devices, handling a certain na= ture or complexity of data, compute power, or serving specific roles in the= data ingress/egress process from generation to consumer
An aggregation of one or more capabilities that allows specific ag= ents to carry out a given processing graph. For example, a high-level view = of a "File Forwarder" class would require the capability to both interact w= ith the file system to get files and additionally have one or more egress m= ethods to return information to a desired consumer
A generic term for providing information from an agent to one or m= ore consumers. In simplest form, this is a direct line through networking t= o send data to a desired target. In more complex environments, this may req= uire an n-hop network relying on several other agents to relay the data thr= oughout the network traversal.