You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 20 Next »

Version 0.5.0

Version 0.5.0 of Apache NiFi is in the process of being released. It will include several exciting new capabilities to include new processors, exciting new developer tools for building and testing processors. Stability and performance continue to be a priority and a long list of bugs were identified and resolved!

Highlights of the 0.5.0 release includes:

  • New Application Features
    • Data inspection: We've greatly enhanced the ability to interact with data and inspect it as it is flowing through NiFi. 
  • Things to make NiFi Development Better and Easier
    • State Management: As many developers are aware, keeping state in a Processor was often "up to you".  This extension to the framework addresses this by adding state management as a core feature. In addition, many existing processors which kept state were modified to take advantage of this new capability. 
    • Testing Improvements: We've improved the performance of many unit tests,  added support for groovy for unit tests, the ability to test in unit tests, as well as better support for integration testing 
  • Improvements to Existing Capabilities
    • RELP Support for Syslog: In addition to adding support for RELP (Reliable Event Logging Protocol) for transporting syslog messages, emphasis was put on improving the framework to enable even more extensions in future releases. 
    • S3 Improvements: We've pushed past the 5G max upload limitation of the previous PutS3Object processor by adding Multipart Upload support. Additionally, we added a couple authorization capabilities, like expression language support for keys, and an exciting new controller service created initially for fetching from buckets access across accounts.
    • Hive Support: The Kite bundle now supports sending data directly into Hive tables
    • Encryption: A broad set of enhancements were made to improve encryption and decryption of content and associated user documentation.
  • New Extensions
    • Script Execution: We've dramatically enhanced the ability to operate on data flowing through NiFi by adding support for launching scripts in a flow. We added a broad set of scripting languages - you can launch JRuby, Groovy, JavaScript, Lua, or Jython scripts with access to FlowFile data, which will allow much more dynamism and quick reaction capability in your data flows.
    • Support for sending Riemann Events: NiFi now supports sending events to the Riemann event stream processing system. 
    • Elasticsearch: Connectivity to the popular search engine was added to this release
    • Avro Schema Inference: Support was added allowing the inference of an Avro schema from JSON and CSV data flowing through NiFi.
    • AMQP support: NiFi now supports sending and receiving from AMQP (Advanced Message Queueing Protocol) based messaging systems

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12334158

Known Issues with this release:

  • NIFI-1497 - Getting issue details... STATUS : An issue with Access Tokens may cause Viewing Content and Custom UIs to fail

Version 0.4.1

Version 0.4.1 of Apache NiFi is an incremental release addressing several bugs and providing a few minor improvements over the 0.4.0 release.

Highlights of 0.4.1 release include:

  • Bugs addressed
    • Site-To-Site: If the remote system was applying back-pressure and the sending system attempted to stop the connection the sending system flow configuration could hang.  If one of the nodes in a cluster being delivered to went off-line it could cause site-to-site to stop delivering to the other nodes in certain conditions.  The automatic account request mechanism was broken.
    • Flow File Ordering: Ordering based on timestamp of flow file entry has been augmented with a sequence counter to provide better ordering precision.
    • Run duration: Only shown now when the processor supports the batching mechanism of run duration.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12334375

Version 0.4.0

Version 0.4.0 of Apache NiFi is a provides substantial improvements in functionality and usability, as well as providing some stability and performance improvements and bug fixes.

Highlights of 0.4.0 release include:

  • New Application Features
    • Multiple Authentication Mechanisms: NiFi now supports multiple Authentication Mechanisms! No longer is NiFi tied to being either non-secure or security based on two-way SSL but now can provide
      User Authentication via LDAP. This was a significant undertaking, but has paved the way to far more easily provide new Authentication Mechanisms. Future releases
      will include additional mechanisms, such as Kerberos.
    • Drop FlowFiles from Queue: Users are now able to right-click on a Connection and drop the FlowFiles in the queue, rather than relying on FlowFile Expiration to remove unwanted FlowFiles.
  • Usability Improvements
    • Explicit Processor Connectivity: Processors that do not expect incoming data will no longer allow incoming Connections. Attempting to draw a connection to a "Source Processor" will show the
      connection line as a red, dotted line, and will not allow the Connection to be made. Likewise, Processors that require input to perform work will be invalid until they have an incoming Connection.
      Some Processors may accept incoming data or run without any incoming data. These Processors will be valid regardless of whether or not they have incoming connections.
    • Getting Started Guide: Getting Started Guide is added to the 'help' screen of the application. This guide provides an introduction to NiFi terms, introduces the key concepts of NiFi and
      discusses how to work with FlowFiles and their Attributes. This is similar in concept to the User Guide but is far less verbose and explains concepts at a higher level.
    • Provenance Fetch Event: A new Provenance Event Type (FETCH) was added. This Event Type is used to indicate that an existing FlowFile's contents were modified as a result of obtaining
      data from an external resource. This is in contrast to a RECEIVE event, which is used to indicate that a FlowFile entered the system as a result of obtaining data from an external resource.
  • New Extensions
    • RouteText: Allows user to easily establish queries against textual data. Each line of a FlowFile is matched against the specified rules and routed according to the rules (potentially many lines of text
      are included in each output FlowFile). Also supports grouping of textual data so that FlowFiles that are output do not contain text from two different groups.
    • TailFile: Allows user to "tail" a file, consuming data from the end of the file as it is written by another process. This is typically used to consume data from log files as it is written. Processor will pick up
      where it left off, even if NiFi is restarted and log files roll over.
    • ListenSyslog / PutSyslog: Listens for incoming Syslog events over UDP or TCP and sends events to Syslog.
    • ListFile / FetchFile: Performs a listing of files in a given directory and fetches those files. These processors differ from GetFile in that the ListFile processor keeps state about files that have been consumed,
      so that the file can be ingested only once without deleting the source file. Additionally, if the directory being monitored exists in a mounted volume, the state can be shared across the cluster so that a new
      primary node can pick up where the previous processor left off, and the listing can also be shared across the cluster, distributing the work of pulling in and processing the files.
    • ListSFTP / FetchSFTP: Performs a listing of files on an SFTP server and fetches those files. Similarly to ListFile / FetchFile, state can be distributed across the cluster so that multiple nodes can perform the
      work in parallel.
    • DeleteS3Object: Removes an Object from Amazon S3.
    • PutHBaseCell / GetHBase / PutHBaseJSON: Allows users to put the contents of a FlowFile to HBase and listen for changes to an HBase table, automatically pulling in the rows that are added/updated.
    • GetAzureEventHub / PutAzureEventHub: Send the contents of FlowFiles as Events to Microsoft Azure Event Hub or listen for incoming Events on an Event Hub and create FlowFiles for those Events.
    • GetCouchbaseKey / PutCouchbaseKey: Send the contents of FlowFiles to Couchbase or fetch the contents of a record from Couchbase.
    • AttributesToJSON: Easily form a JSON document (as the contents of a FlowFile or as an Attribute) from a user-defined set of FlowFile Attributes.
    • SplitAvro: Splits a FlowFile that consists of many Avro records into individual FlowFiles, each containing a smaller number of Avro records.
    • ExtractAvroMetadata: Extracts the metadata from the header of an Avro file and adds the metadata to the FlowFile as a set of Attributes.
    • Image Content Viewer: When users look at the details of a Provenance Event, in the Content tab, if the View button is clicked, and the contents of the FlowFile are an image, that image will be rendered in
      the UI, rather than indicating that no viewer is available for this content type.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333070

Version 0.3.0

Highlights of 0.3.0 release include

  • Performance improvements in handling large volumes of small files.
  • Performance improvements in Provenance repositories.
  • Added Reporting Task for ApacheTM AmbariTM.
  • Improved stability of nifi bootstrap.
  • Added Processors for working with images.
  • Support for interacting with Kerberos enabled Hadoop clusters
  • Added additional Avro capabilities - merging datafiles & converting to json
  • Added Processors for performing INSERT, UPDATE, DELETE statements against relational databases
  • Added Processors that run ApacheTM FlumeTM sources and sinks.
  • Added Processors to integrate with Amazon Web Services (S3, SNS, and SQS).
  • Archival of content is now enabled by default.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329653

Version 0.2.1

Highlights of 0.2.1 release include

  • Removed all appropriate references to Incubator as NiFi is now a TLP.
  • Fixed a bug that appeared in 0.2.0 that caused new users to a NiFi instance not to be prompted to create an account.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333089

NiFi NAR Maven Plugin Version 1.1.0

Highlights of 1.10 release include

  • Created a new plugin that allows developers to obtain a listing of transitive dependencies that are provided through a NAR dependency.

A full list of issues that were resolved can be found at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333301

 

 

  • No labels