| Table of Contents | ||
|---|---|---|
|
Version 2.0.0-M4
Version 2.0.0-M4 is the fourth milestone version of Apache NiFi 2.0.0.
Release Date: July 1, 2024
Jira Release Notes for 2.0.0-M4
Please refer to the Deprecated Components and Features documentation for items removed in this version.
Migrating Deprecated Components and Features for 2.0.0 includes current progress on the steps necessary for migrating certain components and features.
New Capabilities
- Completely rebuilt user interface featuring system or user selectable dark mode!
- Support for Kafka 3 in for both consumption and publishing with Kafka.
- Now able to split binary Packet Capture (PCAP) with SplitPCAP
- Microsoft Excel XLSX files can be split to individual sheets with SplitExcel.
- This is a good example of the increasingly common usage of NiFi in the wild to capture and transform unstructured or semi-structured data and deliver it to systems such as databases, vector stores, and more.
- DeleteFile allows for removal of local files
- The List/Fetch pattern was the beginning of the movement and Delete* components help complete it. NiFi is increasingly being used not just for the traditional continuous and automated capture of data streams but also for more change data capture/replication scenarios.
- New interface for Python extensions supporting components which source new data called FlowFileSource
Notable Improvements
- Added Python Processor support to MiNiFi framework
- Added App Installation Token support to GitHub FlowRegistryClient
- Improved framework keystore and truststore handling to support dynamic reloading for cluster communication
- Changed from RSA to Ed25519 for signing and verification of Application Bearer Tokens
Breaking Changes
- Moved nifi-python-extensions to new Git repository
- Removed support for application property protection and encrypt-config
- Removed REST API support for Apache Knox SSO authentication
- Removed REST API support for Kerberos SPNEGO authentication
NiFi NAR Maven Plugin Version 2.0.0
Version 2.0.0 of Apache NiFi NAR Maven Plugin is a bug fix and dependency upgrade release.
Release Date: May 29, 2024
Highlights of the NAR Maven Plugin 2.0.0 release include:
- Add a flag to skip documentation generation
- Add Parameter Providers and Flow Analysis Rules as extension types
- Attach extension-manifest.xml to build as an artifact
- Dependencies housekeeping and move to Java 21 and Maven 3.9.6 to align with NiFi 2
A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354745
Version 2.0.0-M3
Version 2.0.0-M3 is the third milestone version of Apache NiFi 2.0.0.
Release Date: May 16, 2024
Jira Release Notes for 2.0.0-M3
Please refer to the Deprecated Components and Features documentation for items removed in this version.
Migrating Deprecated Components and Features for 2.0.0 includes current progress on the steps necessary for migrating certain components and features.
New Capabilities
- The new Apache NiFi UI is included and available at the /nf context path!
- The current stable and pre-existing nifi UI remains available at https://host:port/nifi
- The new UI is available at https://host:port/nf
- This is an entirely newly implemented UI that remains faithful to most of the existing user experience but cleans things in up in various ways. It provides a far more modern stack to build on and includes various niceties such as a dark mode - to all those enterprises around the world that have been using NiFi in SOC or other dark rooms for a long time - we're sorry it took so long but you'll love the new dark theme. Once the new UI has all the critical bits to be fully GA we will remove the old one and the only context path will be 'nifi'.
- Python components can now be loaded from a NAR with included dependencies. This allows for strong development processes which leverage pip or anacodona but keeps secure practices in place for deployment where such processes may be blocked.
- OpenID Connect integration now supports the OAuth 2.0 Client Credentials Flow for programmatic access. The Authorization Code Flow now supports Proof Key for Code Exchange for enhanced security with browser-based authentication.
- New RecordReader and support for Protobuf.
- Support for OpenSearch to both publish and query vectors/embeddings.
- New capability allows usage of a Flow Registry client where the 'registry' is simply a Github Repo. The can greatly simplify the SDLC processes and this mechanism now supports branches!
Key Improvements
- The Python integration is really solid and tested both at scale and for behaviors over long duration. Logging behaviors and packaging options all improved.
- We have upgraded nearly all major dependencies and thousands of minor dependencies to their latest available versions and removed components for which dependencies are no longer actively maintained.
Breaking Changes
- Removed separate distribution of Kafka Connect modules from external modules
- Removed Apache Accumulo 2 Processors and Services
- Removed Apache Atlas Reporting Task
- Removed Apache Hive 3 Processors and Services
- Removed Apache Ranger Plugin
- Removed Apache Solr Processors
- Removed Cassandra 3 Processors and Services
- Removed ConvertAvroToJSON Processor
- Removed Couchbase Processors and Services
- Removed Datadog Reporting Task
- Removed ExecuteStateless Processor
- Removed InvokeAWSGatewayApi Processor
- Removed Spark Livy Components
Version 2.0.0-M2
Version 2.0.0-M2 is the second milestone version of Apache NiFi 2.0.0.
This version includes a number of new features and substantial dependency upgrades, as well as several breaking changes.
Release Date: January 29, 2024
Jira Release Notes for 2.0.0-M2
New Components
- Controller Services
- DatabaseTableSchemaRegistryService
- StandardKustoIngestService
- StandardJsonSchemaRegistry
- ZendeskRecordSink
- Processors
- CalculateParquetOffsets
- CalculateParquetRowGroupOffsets
- FilterAttribute
- PublishSlack
- PutMongoBulk
- PutAzureDataExplorer
- PutZendeskTicket
Notable Upgrades
- Spring Framework 6
- Jetty 12
- Jakarta Servlet API 6
- Jakarta XML Binding 4
- Swagger 2 annotations
- OpenAPI 3.0 REST API specification
Breaking Changes
- Removed MiNiFi C2 Server modules and Docker image configuration
- Relocated JoltTransformJSON and JoltTransformRecord from nifi-standard-nar to nifi-jolt-nar
- Removed InfluxDB Processors
- Removed Bootstrap Notification Services
- Moved from SimpleDateFormat to DateTimeFormatter for date and time parsing and formatting
- Moving from Servlet API 3 and Servlet API 6 requires custom user interface extensions to be updated and recompiled
Version 2.0.0-M1
Version 2.0.0-M1 is the first milestone version of Apache NiFi 2.0.0. This version includes over 900 issues resolved, including new features as well as numerous improvements and bug fixes.
Please refer to the Deprecated Components and Features documentation for items removed in this version.
Migrating Deprecated Components and Features for 2.0.0 includes current progress on the steps necessary for migrating certain components and features.
Release Date: November 25, 2023
Jira Release Notes for 2.0.0-M1
New Features and Changes for 2.0.0-M1
- Initial version of native Python API for Processors
- Stateless Execution mode for Process Groups
- Flow Analysis Rules API
- Kubernetes-based Leader Election and State Management extensions
- Python-based Processors for interacting with ChatGPT and Vector Databases
- ListenOTLP Processor for collecting OpenTelemetry
- ListenSlack and ConsumeSlack Processors for handling messages from Slack
- EncryptContentAge and DecryptContentAge Processors supporting age-encryption.org specification
- Schema Registry Services for Amazon Glue and Apicurio
- Parameter Provider for 1Password Vault
- YamlTreeReader for YAML as Records
- PackageFlowFile Processor for writing file streams and attributes as FlowFile Version 3
- Migrated from H2 Database Engine to JetBrains Xodus for storing Flow Configuration History
- Moved several modules to optional build profiles, requiring separate NAR download from the Central Repository
- nifi-hadoop-nar
- nifi-hadoop-libraries-nar
- nifi-hbase-nar
- nifi-hbase2_client-service-nar
- nifi-kudu-nar
- nifi-parquet-nar
- nifi-solr-nar
Version 1.27.0
Version 1.27.0 is feature improvement and bug fix release, including over 100 issues resolved.
Release Date: July 8, 2024
New features
- Add completion strategy to FetchSmb
- Add SplitExcel Processor
Notable Changes
- Streamline using single parameter context for nested PGs
- Add Endpoint Override Property to PutSNS
- CLI command to upgrade all instances of a versioned flow
- Deprecate InvokeAWSGatewayApi Processor
- Deprecate Accumulo Components for Removal
- Deprecate Apache Knox SSO Integration for Removal
- Deprecate Kerberos SPNEGO Authentication for Removal
- Deprecate Property Protection and Encrypt Config Tools
Version 1.26.0
Version 1.26.0 is feature improvement and bug fix release, including over 100 issues resolved.
Release Date: May 6, 2024
New Components
- Controller Services
- Record Reader for Protobuf data
Notable Changes
- Support for password-protected sheets in ExcelReader
- Add No Tracking Strategy to ListS3
- Support dynamic host/port in Set/Get SNMP processors
- Add DFS setting to smb processors
- Many improvements around Kerberos authentication handling in processors
- Deprecate Spark Livy components for removal
- Deprecate Solr components for removal
- Deprecate Apache Ranger integration for removal
- Deprecate Hive 3 components for removal
- Deprecate ConvertAvroToJSON processor for removal
- Many dependency upgrades
Version 1.25.0
Version 1.25.0 is feature improvement and bug fix release, including over 100 issues resolved.
Release Date: January 29, 2024
New Components
- Controller Services
- DatabaseTableSchemaRegistryService
- StandardJsonSchemaRegistry
- ZendeskRecordSink
- Processors
- CalculateParquetOffsets
- CalculateParquetRowGroupOffsets
- FilterAttribute
- PublishSlack
- PutZendeskTicket
Notable Changes
- Added Component State strategy for ConsumeAzureEventHub Checkpointing
- Corrected automated migration of Flow Configuration History from NiFi 1.23.2 and earlier versions
- Deprecated Bootstrap Notification Services for removal in NiFi 2.0.0-M2
- Deprecated MiNiFi C2 Server for removal in NiFi 2.0.0-M2
- Upgraded Logback to 1.3.14
- Upgraded SSHJ to 0.38.0
- Upgraded JLine to 3.25.0
- Upgraded JSON Path to 2.9.0
Version 1.24.0
Version 1.24.0 of Apache NiFi is a feature improvement and bug fix release, including over 270 issues resolved.
Release Date: November 27, 2023
New Features and Changes for 1.24.0
- ListenOTLP Processor for collecting OpenTelemetry
- EncryptContentAge and DecryptContentAge Processors supporting age-encryption.org specification
- YamlTreeReader for YAML as Records
- PackageFlowFile Processor for writing file streams and attributes as FlowFile Version 3
- Migrated from H2 Database Engine to JetBrains Xodus for storing Flow Configuration History
- Deprecated Jython Script Engine for Scripted Processors
Version 1.23.2
Version 1.23.2 of Apache NiFi is a bug fix release. NiFi 1.23.0 and 1.23.1 should not be used.
Release Date: August 22, 2023
Highlights of the 1.23.2 release include:
- Corrected Repository corruption related to handling empty FlowFiles
Version 1.23.1
Version 1.23.1 of Apache NiFi is a bug fix and dependency upgrade release.
Release Date: August 18, 2023
Highlights of the 1.23.1 release include:
- Corrected ListHDFS closing of File System resources
- Corrected ListenGRPC with TLS on Java 17
- Corrected NiFi Registry Git Provider integration with SSH
- Added Proxy Configuration support for Azure Storage Credentials
- Incremental dependency upgrades including Logback 1.3.8, Bouncy Castle 1.76, and Netty 4.1.96
A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353541
Version 1.23.0
Version 1.23.0 of Apache NiFi is an improvement, bug fix and security focused release.
Release Date: July 28, 2023
Highlights of the 1.23.0 release include:
- Added Record Reader for Excel XLSX Files and RemoveRecordField Processor
- Added Service for Amazon Glue Schema Registry
- Upgraded Spring Framework, Spring Security, Bouncy Castle, and many other libraries for framework and extension components
- Moved Riemann and HBase 1.1.2 NAR bundles to optional build profiles
- Deprecated RethinkDB components
A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353320
NiFi NAR Maven Plugin Version 1.5.1
Version 1.5.1 of Apache NiFi NAR Maven Plugin is a bug fix and dependency upgrade release.
Release Date: June 14, 2023
Highlights of the NAR Maven Plugin 1.5.1 release include:
- NAR Plugin extension docs not correctly resolving overridden versions from system properties.
- NAR Plugin not correctly using remote repositories.
A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353009
Version 1.22.0
Version 1.22.0 of Apache NiFi is an improvement, bug fix and security focused release.
Release Date: June 11, 2023
Highlights of the 1.22.0 release include:
- MiNiFi agents can now talk to C2 servers using reverse proxy/load balancers.
- New processor to support modifying the compression algorithm of content which still incurs the CPU hit substantially improves IO by avoiding writing and reading the intermediary decompressed form.
- New processor for Azure Queue Storage using Azure SDK 12.
- Put Database Record now allows for Upserts.
- Deprecated additional components and features in preparation toward an Apache NiFi 2.0.
- Upgraded numerous dependencies due to potential vulnerabilities and latest stable lines.
- Numerous bug fixes to processors but also core NiFi framework behavior
- A full list of issues that were resolved can be found at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353069
Version 1.21.0
Version 1.21.0 of Apache NiFi is an improvement, bug fix and security focused release.
...