...
Anchor | ||||
---|---|---|---|---|
|
Authority Provider model has been replaced by a Multi-tenant Authorization model. Access privileges are now defined by policies that can be applied system-wide or to individual components. Details can be found in in the Multi-tenant Authorization in Hortonworks DataFlow Administration section of the System Administrator's Guide.
The system properties nifi.authority.provider.configuration.file and nifi.security.user.authority.provider have been replaced byby nifi.authorizer.configuration.file and nifi.security.user.authorizer, respectively. Details on configuration can be found in in the Authorizer Configuration in Hortonworks DataFlow Administration section of the System Administrator's Guide.
You can convert NiFi 0.7.0 authorized users and roles to the new authorization model. An existing
authorized-users.xml
file can be referenced in theauthorizers.xml
"Legacy Authorized Users File" property to automatically generate users and authorizations. Details on configuration can be found in in the Authorizers.xml Setup in Hortonworks DataFlow Administration section of the System Administrator's Guide.
Anchor | ||||
---|---|---|---|---|
|
HTTP(S) protocol is now supported in Site-to-Site as an underlying transport protocol.
HTTP(S) protocol is enabled by default (nifi.remote.input.http.enabled=true). Configuration details can be found in in the Site-to-Site Properties in Hortonworks DataFlow Administration section of the System Administrator's Guide.
Of note:
With both socket and HTTP protocols supported, nifi.remote.input.socket.host has been renamed to nifi.remote.input.host.
nifi.remote.input.secure is now set to false by default.
...
Anchor | ||||
---|---|---|---|---|
|
Master/slave clustering model has been replaced by a Zero-Master Clustering paradigm. Each node in a NiFi cluster performs the same tasks on the data, but each operates on a different set of data. A DataFlow manager can now interact with the NiFi cluster through the UI of any node.
ZooKeeper elects a single node as the Cluster Coordinator and also handles failover. All cluster nodes report heartbeat and status information to the Cluster Coordinator, which is responsible for disconnecting and connecting nodes. Additionally, every cluster has one Primary Node, also elected by ZooKeeper.
Configuration details can be found in the Clustering Configuration, the Cluster Common Properties, the Cluster Node Properties, and the ZooKeeper Properties sections of Hortonworks DataFlow Administration.of the System Administrator's Guide.
NoteOf note for your upgrade:
NiFi Cluster Manager (NCM) configuration and properties are no longer relevant and have been removed.
The following properties should be set on each node:
nifi.web.http.port=<node port>
nifi.cluster.is.node=true
nifi.cluster.node.address=<fully qualified hostname of the node>
nifi.cluster.node.protocol.port=<node protocol port>
nifi.state.management.embedded.zookeeper.start=true
nifi.state.management.provider.cluster=zk-provider
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
nifi.zookeeper.connect.string=<A comma-separated list of host:port pairs to connect to ZooKeeper. For example, my-zk-server1:2181,my-zk-server2:2181,my-zk-server3:2183>
Coordinated dataflow selection across cluster nodes. During startup, a cluster coordinator is selected at random, and manages the distribution of the dataflow across all nodes. You should set the following to properties, to ensure that the cluster coordinator and other nodes have time to select the correct dataflow:
nifi.cluster.flow.election.max.wait.time=5 mins
nifi.cluster.flow.election.max.candidates=<number of NiFi nodes in the cluster>
Embedded ZooKeeper setup
The zookeeper.properties file needs to be populated with a list of each node's embedded ZooKeeper server. The servers are specified in the form of server.1, server.2, to server.n. Each of these servers is configured as <hostname>:<quorum port>[:<leader election port>]. For example, server.1=nifi-node1-hostname:2888:3888.
The
zookeeper.properties
file has a property named dataDir which is set to ./state/zookeeper by default. For each node, create a file named myid and place it in this directory. The contents of this file should be the index of the server as specified by the server.<number>. Configuration details can be found in in the Embedded ZooKeeper Server in Hortonworks DataFlow Administration section of the System Administrator's Guide.
State Management – In the state-management.xml file, set the “Connect String” property to the same list of ZooKeeper host:port pairs used for the nifi.zookeeper.connect.string property value.
Secure Clustered Environment – The identities for each node must be specified in the authorizers.xml file. The authorization policies required for the nodes to communicate will then be created during startup. Details on configuration can be found in Authorizers.xml Setup in Hortonworks DataFlow Administration section of the System Administrator's Guide.
Anchor | ||||
---|---|---|---|---|
|
The username and password mechanism to provide ZooKeeper authentication is no longer supported. As a result, the “Username” and “Password” properties in the
state-management.xml
file have been removed.The “Access Control” property in the
state-management.xml
file is now set to “Open” by default. It should be changed to “CreatorOnly” when ZooKeeper is secured via Kerberos.
...