Date: Tue, 19 Mar 2024 10:31:26 +0000 (UTC) Message-ID: <2000939454.56262.1710844286170@cwiki-he-fi.apache.org> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_56261_840243709.1710844286170" ------=_Part_56261_840243709.1710844286170 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
NiFi is a dataflow system based on the concepts of flow-based programmin= g.
Ni{{`Fi supports powerful and scalable directed graphs of data routing, = transformation, and system mediation logic. Some of the high-level capabili= ties and objectives of Ni}}`Fi include:
Reliable and effective dataflow between systems can be difficult whether= you're running scripts on a laptop or have a massive distributed computing= system operated by numerous teams and organizations. As the volume and rat= e of data grows and as the number of systems, protocols, and formats increa= se and evolve so too does the complexity and need for greater insight and a= gility. These are the dataflow challenges that NiFi was built to tackle.
Ni{{`Fi is designed in a manner consistent with the core concepts descri= bed in flow-based programming as originally documented by J. Paul Morrison = in the 1970s. http://www.jpaulmorrison.com/fbp/ This model = lends itself well to visual diagramming, concurrency, componentization, tes= ting, and reuse. In addition to staying close to the fundamentals of flow-b= ased programming, Ni}}`Fi provides integration system specific features suc= h as: guaranteed delivery; back pressure; ability to gracefully handle back= logs and data surges; and an operator interface that enables on-the-fly dat= a flow generation, modification, and observation.
Ni{{`Fi provides a reliable, scalable, manageable and accountable platfo= rm for developers and technical staff to create and evolve powerful data fl= ows. Such a system is useful in many contexts including large-scale enterpr= ise integration, interaction with cloud services and frameworks, business t= o business, intra-departmental, and inter-departmental flows. Ni}}`Fi fits = well within the Apache Software Foundation (ASF) family as it depends on nu= merous ASF projects and integrates with several others. We also anticipate = developing extensions for several other ASF projects such as Cassandra, Kaf= ka, and Storm in the near future.
An integration platform is only as good as its ability to integrate syst= ems in a reliable, timely, and repeatable manner. The same can be said of i= ts ability to attract talent and a variety of perspectives as integration s= ystems by their nature are always evolving. We will actively seek help and = encourage promotion of influence in the project through meritocracy.
Over the past several years, Ni{{`Fi has developed a strong community of= both developers and operators within the U.S. government. In open sourcing= Ni}}`Fi we plan to grow the community to a broader base of industries and = will work to align the interaction of our existing community.
The initial core developers are employed by the National Security Agency= and defense contractors. We will work to grow the community among a more d= iverse set of developers and industries.
From its inception, Ni{{`Fi was developed with an open source philosophy=
in mind and with the hopes of eventually being truly open sourced. The Apa=
che way is consistent with the approach we have taken to date. The ASF clea=
rly provides a mature and effective environment for successful development =
as is evident across the spectrum of well-known projects. Further, Ni}}Client, Jakarta and Maven. We =
also anticipate extensions and dependencies with several more ASF projects,=
including Accumulo, Avro, Casandra, HBase, JClouds, Storm, Kafka, Thrift, =
Tika, and others. This existing alignment with Apache and the desired commu=
nity makes the Apache Incubator a good fit for Ni
`Fi.
Risk of orphaning is limited though it is important to grow the communit= y. The project user and developer base is substantial, growing, and there i= s already extensive operational use of NiFi.
The initial committers to NiFi have limited experience with true open so= urce software development. However, despite the project origins being from = closed source development we have modeled our behavior and community develo= pment on The Apache Way to the greatest extent possible. This environment i= ncludes widely accessible source code repositories, published artifacts, ti= cket tracking, and extensive documentation. We also encourage contributions= and frequent debate and hold regular, collaborative discussions through e-= mail, chat rooms, and in-person meet-ups. We are committed to the ideals of= open source software and will eagerly seek out mentors and sponsors who ca= n help us quickly come up to speed.
The initial committers of NiFi come from a limited set of entities thoug= h we are committed to recruiting and developing additional committers from = a broad spectrum of industries and backgrounds.
We expect NiFi development to continue on salaried time and through volu= nteer time. The initial committers are paid by their employers to contribut= e to this project. We are committed to developing and recruiting participat= ion from developers both salaried and non-salaried.
As described in the alignment section, NiFi is already heavily dependent= on other ASF projects and we anticipate further dependence and integration= with new and emerging projects in the Apache family.
We respect the laudable Apache brand and that is certainly a factor in t= he decision to propose Ni{{`Fi for the Apache Incubator. The ASF is a natur= al home for Ni}}`Fi given our existing dependency and alignment with ASF pr= ojects. We intend to provide a great deal of energy and capability to the A= SF through this project. We will be sensitive to and respectful of any over= use of the Apache brand and ensure our focus remains on how we benefit the = Apache community.
At this time there is no NiFi documentation on the web. However, we have= extensive documentation included within the application that details usage= of the many functions. Using incubator INFRA we will be rapidly expanding = the available documentation to cover things like installation, developer gu= ide, frequently asked questions, best practices, and more.
NiFi has been in active development since late 2006 with contributions f= rom dozens of developers and feedback from hundreds of users and developers= . The core codebase is written in Java and includes detailed Javadocs and f= eature documentation.
Previously referred to as Niagarafiles, the Ni{{`Fi code and documentati= on materials will be submitted by the National Security Agency. Ni}}`Fi has= been developed by a mix of government employees and private companies unde= r government contract. Material developed by the government employees is in= the public domain and no U.S. copyright exists in works of the federal gov= ernment. For the contractor developed material in the initial submission, t= he U.S. Government has sufficient authority to open source per DFARS 252.22= 7-7014. NSA has submitted the Software Grant Agreement and Corporate Contri= butor License Agreement to the Apache Software Foundation.
We have at least one dependency on an LGPL library which we will promptl= y address. Otherwise, we believe all current dependencies are compatible wi= th the ASF guidelines. Our dependency licenses come from the following lice= nse styles: Apache v 2.0, BSD, Public Domain, Eclipse Public v1, MIT, CDDL = v1.
Consistent with http://www.apache.org/licenses/exports=
/ we believe Ni{{`Fi is classified as ECCN 5D002. Ni}}Fi doesn't =
implement any cryptographic algorithms but is designed to use algorithms pr=
ovided by Oracle Java Cryptographic Extensions, Bouncy
`Castle, and J=
Craft, Inc. These cryptographic algorithm providers are used to support SSL=
, SSH/SFTP, and the encryption and decryption of sensitive properties. In t=
he event that it becomes necessary we will engage with appropriate Apache m=
embers to ensure we file any necessary paperwork or clarified any cryptogra=
phic export license concerns.
Ni{{`Fi requests use of Git for source control (git://git.apache.org/nif= i.git). We request a writeable Git repo for Ni}}`Fi with mirroring to be se= tup to Github through INFRA. We request sponsor Benson Margulies (bimarguli= es) to assist with creating the INFRA ticket for this.
JIRA NiFi (NIFI)
We request the Apache Incubator to sponsor this project.