Release Notes & Migration Guidance
Prior to upgrade you should review the Release Notes carefully to ensure that you understand the changes made and the impact they may have on your existing data flows and/or environment.
The instructions below are general steps to follow when upgrading from a 1.x.0 release to another. Check the Migration Guidance page for specific items that you should be aware of when moving between specific NiFi versions.
Important:
You cannot have nodes with different NiFi versions in the same cluster.
Running multiple NiFi nodes on the same host machine is highly discouraged.
Before you begin
Before you begin, you can perform some tasks to improve the efficiency and success of your upgrade.
Preserving your custom processors
If you have written any custom NARs, first test and validate them in a separate new NiFi release sandbox prior to upgrading. The verified NARs can be preserved during the upgrade by storing them in a centralized location as follows:
Create a second library directory, called
custom_lib.Move your custom NARs to this new lib directory.
Add a new line to the
nifi.propertiesfile to specify this new lib directory:
nifi.nar.library.directory=./lib
nifi.nar.library.directory.custom=/opt/configuration_resources/custom_lib
Preserving your modified NARs
If you have modified any of the default NAR files, an upgrade will overwrite these changes. To preserve the customization:
Identify and save the changes you have made to the default NAR files in your existing instance.
Perform your upgrade to the new NiFi release.
Redo the changes you have made to the NAR files in your upgraded NiFi instance.
Installation
1. Download and extract the new NiFi tar.gz file.
2. For each node in your cluster, install the new NiFi in a directory parallel to your existing NiFi installation. Assuming you have installed into an opt/ directory, for example:
Host Machine - Node 1
|--> opt/
|--> existing-nifi
|--> new-nifi
Host Machine - Node 2
|--> opt/
|--> existing-nifi
|--> new-nifi
Host Machine - Node 3
|--> opt/
|--> existing-nifi
|--> new-nifi
Note: Make sure that all file and directory ownerships for your new NiFi directories match what you set on the existing directories.
3. On your existing NiFi installation:
- Stop all processors responsible for ingesting new data.
- Allow NiFi to run until no more FlowFiles are queued anywhere in the dataflow(s).
- Shutdown your existing NiFi instance(s).
4. Use the configuration files from your existing NiFi installation (<installation-directory>/conf) to update the corresponding properties in your new NiFi deployment.
| Configuration file | Necessary changes |
|---|---|
| authorizers.xml | Copy the <authorizer>...</authorizer> configured in the existing NiFi to new the NiFi file. |
If you are using the default “file-provider” authorizer, ensure that you copy the Configuration best practices recommend creating a separate location outside of the NiFi base directory for storing such configuration files. If you are storing these files in a separate directory, you do not need to move them. Instead, ensure that the new NiFi is pointing to the same files. For example: | |
| bootstrap-notification-services.xml | Update the values based on values from the existing NiFi file. |
| bootstrap.conf | Use the existing NiFi bootstrap.conf file to update properties in the new NiFi. |
| logback.xml | If you added any custom logging modification to the existing NiFi, make those same changes to the new NiFi. |
| login-identity-providers.xml | If you used a provider value in the existing NiFi, copy the |
| nifi.properties | Use the existing file to populate the same properties in the new NiFi This file contains the majority of NiFi configuration settings, so ensure that you have copied the values correctly. |
If you followed NiFi best practices, the following properties should be pointing to external directories outside of the base NiFi installation path. If the below properties point to directories inside the NiFi base installation path, you must copy the target directories to the new NiFi. Stop your existing NiFi installation before you do this. | |
If you have retained the default value, ( Alternately, you can copy to an external location and update the property value to point there. | |
Same applies as above if you want to retain archived copies of the | |
Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external database repository location. | |
Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external flowfile repository location. Warning: You may experience data loss if flowfile repositories are not accessible to the new NiFi. | |
Best practices recommends that you use an external location for each repository. Point the new NiFi at the same external content repository location. Your existing NiFi may have multiple content repos defined. Make sure the exact same property names are used and point to the appropriate matching content repo locations. For example: nifi.content.repository.directory.content1= nifi.content.repository.directory.content2= Warning: You may experience data loss if content repositories are not accessible to new NiFi. Warning: You may experience data loss may if property names are wrong or the property points to wrong content repository. | |
Best practices recommends that you use an external location for each repository. Point new NiFi at same external provenance repository location. Your existing NiFi may have multiple provenance repos defined. Make sure exact same property names are used and point to appropriate matching provenance repo locations. For example: nifi.provenance.repository.directory.provenance1= nifi.provenance.repository.directory.provenance2= Note: You may not be able to query old events if provenance repos are not moved correctly or properties are not updated correctly. | |
| state-management.xml | For the “local-provider” value, verify the location of “state/local” directory. If you have retained the default location ( Configuration best practices recommend that you move the state to an external directory like |
For a NiFi cluster, the “cluster-provider” ZooKeeper “Connect String" should be using the same external ZooKeeper as the existing NiFi installation. | |
| For a NiFi cluster, make sure the “cluster-provider” ZooKeeper "Root Node" matches exactly the value you used in the existing NiFi. | |
| If you are also setting up a new external ZooKeeper, see the ZooKeeper Migrator section of the NiFi System Administrator’s Guide for instructions on how to move ZooKeeper information from one cluster to another and migrating ZooKeeper node ownership. |
Note:
- Do not copy configuration files from your old NiFi version to the new NiFi version. The newer configuration files may introduce new properties that would be lost by doing so.
- Double check all configured properties for typos.
- If you are encrypting sensitive component properties in your dataflow via the sensitive properties key in
nifi.propertiesfile, make sure the same key is used when copying over yourflow.xml.gz. If you need to change the key, use the flow migration procedure and the NiFi Encrypt-Config Tool to produce a newflow.xml.gzfile from the existing one.
5. Start each of your new NiFi instances.
6. Verify that:
- All your data flows have returned to a running state. Some processors may have new properties that need to be configured, in which case they will be stopped and marked “invalid”.
- All your expected controller services and reporting tasks are running again. Address any controller services or reporting tasks that are marked "invalid".
7. After confirming your new NiFi instances are stable and working as expected, the old instances can be removed.
Note: If original NiFi was setup to run as a service, update any symlinks or service scripts to point to the new NiFi version executables.