Summary

In order to improve security, reduce complexity, and provide a solid foundation for new features, NiFi 2.0 should focus primarily on technical debt reduction.

The release should not exclude new features, but should prioritize technical debt reduction together with a straightforward upgrade path for current installations.

Approval

The Apache NiFi Project Management Committee voted to adopt these release goals on 2022-12-15.

Background

Following the principles of Semantic Versioning, a major release provides the opportunity to introduce breaking changes.

The NiFi 2.0 Release Goals thread on the Developers mailing list provides additional background.

For the purpose of scoping the release to achievable goals, significant new features should be considered in separate proposals.

Primary Goals

  1. Remove Java 8 support and require Java 11
  2. Remove deprecated components
  3. Remove deprecated component properties
  4. Remove components integrating with unmaintained services
  5. Remove compatibility classes and methods
  6. Remove flow.xml.gz in favor of flow.json.gz
  7. Remove duplicative features
  8. Upgrade internal Java API references
  9. Reorganize standard components
  10. Implement migration tools for upgrading flows

Remove Java 8 Support and Require Java 11

Multiple important project dependencies have deprecated or removed support for Java 8.

The Developers mailing list thread discussing a strategy for dropping Java 8 support provides additional background.

Subsequent discussion on the Developers mailing list indicated a general preference for making Java 17 the minimum required version. NIFI-11717 changed the minimum required version from Java 11 to Java 17.

Remove Deprecated Components

Removing deprecated components includes Processors and Controller Services marked as unsupported or superseded by better alternatives.

Remove Deprecated Component Properties

Removing deprecated component properties maintains existing Processors while eliminating duplicative configuration options.

  • PGP properties in EncryptContent and associated capabilities now implemented in EncryptContentPGP and DecryptContentPGP
  • Keytab properties directly in processors that support Kerberos, as well as KeytabCredentialService, replaced with KerberosUserService

Remove Components integrating with Unmaintained Services

Multiple components and associated bundles support integration with services that are no longer maintained. This includes products that have released new major versions.

Remove Compatibility Classes and Methods

Multiple classes and methods supporting legacy capabilities should be removed.

  • PersistentProvenanceRepository
  • StandardRecordWriter
  • NiFiLegacyCipherProvider and related encryption methods
  • supportsExpressionLanguage() with no arguments
  • Custom InputStream and OutputStream classes

Remove flow.xml.gz in favor of flow.json.gz

The internal persistent flow configuration should be consolidated to use flow.json.gz.

  • NiFi 1.16.0 introduced flow.json.gz for storing the flow configuration using Versioned Component model classes
  • NiFi, NiFi Stateless, and NiFi Registry use the JSON Flow Definition representation of Versioned Components
  • Eliminating the flow.xml.gz reduces the overhead of persisting configuration and synchronizing multiple representations

Remove Duplicative Features

Duplicative features should be removed to provide clear patterns of recommend flow design.

  • Remove XML Templates in favor of JSON Flow Definitions
  • Remove Variable Registry in favor of Parameter Contexts

Upgrade Internal Java API References

Migrating internal usage of java.util.Date to java.time classes provides greater precision in parsing and formatting.

  • Refactor Record-oriented components to use DateTimeFormatter and java.time classes for date and time handling
  • Review and refactor references to deprecated and outdated Java reflection and collection methods

Reorganize Standard Components

The the size and scope of Standard processors and Standard Services NAR bundles should be reduced.

  • Move components with specialized dependencies from Standard Processors to separate bundles
    • SFTP Processors
    • Jolt Transform Processors
    • Jetty HTTP Processors
    • JSON Processors
    • Netty-based Processors

Implement Migration Tools for Upgrading Flows

Migration documentation and tools should streamline the process for upgrading to the next major version.

  • Implement automated migration where possible to remap properties and features
  • Implement migration tools for manual conversion of XML Templates to JSON Flow Definitions
  • Create documentation for manual steps necessary where programmatic migration cannot be implemented
  • NiFi 2.0 should be capable of starting with ghosted components for removed Processors or Controller Services
  • No labels