...
- Introduce the concept of a Parameter Provider, which would allow allows Parameter names and values to be fetched from sources external to NiFi
- Provide a way to specify both Sensitive and Non-Sensitive Parameter Providers in a Parameter Context
...
Basic Concept
Parameter Providers could would be introduced at the Controller Settings level as a way to fetch parameters from an external source. These could be selectable in the Add/Update Parameter Context view under the SETTINGS tab. Two new drop-down fields would be added on this tab:
...
In the PARAMETER PROVIDERS list tab, a "Fetch Parameters" button button could be available on each Parameter Provider, which would cause it to pull all available parameters and values from the external source, based on the current configuration. These should would be cached in the Parameter Context Provider as a set of proposed changes, but not actually persisted yet. Next, a Fetch Apply Fetched Parameters view should pop up, which initiates the application of the new values to the flow, similar to when a Parameter Context is updated today:
...
It would also be possible to have both an APPLY and an APPLY AND FETCH button in the Add Parameter Provider view. The former would apply the configuration changes and close the dialog, and the latter would apply the changes, close the dialog, and then launch the fetch sequence described above.
Cluster Integrity for Parameters
The Refresh operation must detect when parameters might be different between nodes in the cluster. Consider the case where an Environment Variable is updated on the system of only one running node. If the node restarts, due to the parameter cache, all parameters would still be synchronized among the cluster. However, if the Fetch Parameters button is used, the parameters for that node would be out of sync with the rest of the cluster unless the Fetch operation detected such differences. A discrepancy should cause the Fetch to fail with a useful warning to the user, who could then ensure all Environment variables were in sync before attempting another refresh/apply sequence described above.
Opt-in Auto-refresh
To remove the need for manually refreshing Parameter Providers in order to apply external changes (as in the case of password rotation), an auto-refresh feature could be configured on the Add/Update Parameter Provider view. This should be disabled by default, and a clear warning should be displayed when enabled, indicating that an updated value would automatically stop and start processors and controller services in the flow.
...