Release Notes & Migration Guidance

Prior to upgrade you should review the Release Notes carefully to ensure that you understand the changes made and the impact they may have on your existing data flows and/or environment.

The instructions below are general steps to follow when upgrading from a 1.x.0 release to another.  Check the Migration Guidance page for specific items that you should be aware of when moving between specific NiFi versions.

Important:

Before you begin

Before you begin, you can perform some tasks to improve the efficiency and success of your upgrade.

Preserving your custom processors

If you have written any custom NARs, first test and validate them in a separate new NiFi release sandbox prior to upgrading.  The verified NARs can be preserved during the upgrade by storing them in a centralized location as follows:

  1. Create a second library directory, called custom_lib.

  2. Move your custom NARs to this new lib directory.

  3. Add a new line to the nifi.properties file to specify this new lib directory:

nifi.nar.library.directory=./lib

nifi.nar.library.directory.custom=/opt/configuration_resources/custom_lib

Preserving your modified NARs

If you have modified any of the default NAR files, an upgrade will overwrite these changes. To preserve the customization:

  1. Identify and save the changes you have made to the default NAR files in your existing instance.

  2. Perform your upgrade to the new NiFi release.

  3. Redo the changes you have made to the NAR files in your upgraded NiFi instance.

Installation

1. Download and extract the new NiFi tar.gz file.

2. For each node in your cluster, install the new NiFi in a directory parallel to your existing NiFi installation.  Assuming you have installed into an opt/ directory, for example:

Host Machine - Node 1

|--> opt/

   |--> existing-nifi

   |--> new-nifi

 

Host Machine - Node 2

|--> opt/

   |--> existing-nifi

   |--> new-nifi


Host Machine - Node 3

|--> opt/

   |--> existing-nifi

   |--> new-nifi


Note:  Make sure that all file and directory ownerships for your new NiFi directories match what you set on the existing directories.

3. On your existing NiFi installation:

4. Use the configuration files from your existing NiFi installation (<installation-directory>/conf) to update the corresponding properties in your new NiFi deployment.

Configuration fileNecessary changes
authorizers.xmlCopy the <authorizer>...</authorizer> configured in the existing NiFi to new the NiFi file.

If you are using the default “file-provider” authorizer, ensure that you copy the users.xml and authorizations.xml files from the existing to the new NiFi.

Configuration best practices recommend creating a separate location outside of the NiFi base directory for storing such configuration files. If you are storing these files in a separate directory, you do not need to move them. Instead, ensure that the new NiFi is pointing to the same files. For example: /opt/nifi/configuration-resources/

bootstrap-notification-services.xml

Update the values based on values from the existing NiFi file.

bootstrap.confUse the existing NiFi bootstrap.conf file to update properties in the new NiFi.
logback.xmlIf you added any custom logging modification to the existing NiFi, make those same changes to the new NiFi.
login-identity-providers.xml

If you used a provider value in the existing NiFi, copy the <provider>...</provider> configuration from the existing NiFi to this file in your new NiFi.

nifi.properties






Use the existing file to populate the same properties in the new NiFi nifi.properties file.

This file contains the majority of NiFi configuration settings, so ensure that you have copied the values correctly.

If you followed NiFi best practices, the following properties should be pointing to external directories outside of the base NiFi installation path.

If the below properties point to directories inside the NiFi base installation path, you must copy the target directories to the new NiFi. Stop your existing NiFi installation before you do this.

nifi.flow.configuration.file=

If you have retained the default value, (./conf/flow.xml.gz), copy flow.xml.gz from the existing to the new NiFi base install conf directory.

Alternately, you can copy to an external location and update the property value to point there.

nifi.flow.configuration.archive.dir=

Same applies as above if you want to retain archived copies of the flow.xml.gz.

nifi.database.directory=

Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external database repository location.

nifi.flowfile.repository.directory=

Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external flowfile repository location.

Warning: You may experience data loss if flowfile repositories are not accessible to the new NiFi.

nifi.content.repository.directory.default=

Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external content repository location.

Your existing NiFi may have multiple content repos defined. Make sure the exact same property names are used and point to the appropriate matching content repo locations. For example:

nifi.content.repository.directory.content1=
nifi.content.repository.directory.content2=

Warning: You may experience data loss if content repositories are not accessible to new NiFi.

Warning: You may experience data loss may if property names are wrong or the property points to wrong content repository.

nifi.provenance.repository.directory.default=

Best practices recommends that you use an external location for each repository. Point new NiFi at same external provenance repository location.

Your existing NiFi may have multiple content repos defined. Make sure exact same property names are used and point to appropriate matching content repo locations. For example:

nifi.provenance.repository.directory.provenance1=
nifi.provenance.repository.directory.provenance2=

Note: You may not be able to query old events if provenance repos are not moved correctly or properties are not updated correctly.

state-management.xml


For the “local-provider” value, verify the location of “state/local” directory.

If you have retained the default location (./state/local), copy the complete directory tree to the new NiFi. The existing NiFi should be stopped if you are copying this directory because it may be constantly writing to this directory while running.

Configuration best practices recommend that you move the state to an external directory like/opt/nifi/configuration-resources/ to facilitate easier upgrading later.

For a NiFi cluster, the “cluster-provider” ZooKeeper “Connect String" should be using the same external ZooKeeper as the existing NiFi installation.

For a NiFi cluster, make sure the “cluster-provider” ZooKeeper "Root Node" matches exactly the value you used in the existing NiFi.
If you are also setting up a new external ZooKeeper, see the ZooKeeper Migrator section of the NiFi System Administrator’s Guide for instructions on how to move ZooKeeper information from one cluster to another and migrating ZooKeeper node ownership.


Note:

5. Start each of your new NiFi instances.

6. Verify that:

7. After confirming your new NiFi instances are stable and working as expected, the old instances can be removed.

Note:  If original NiFi was setup to run as a service, update any symlinks or service scripts to point to the new NiFi version executables.