...
Here are sequence diagrams describing complete HTTP request/response sequence of successful scenario for both input-ports and output-ports. Other semi-normal and error cases are described in Interaction Scenarios.
input-ports/
The A_component in the diagrams is a component which uses SiteToSiteClient class, such as:
- NiFiReceiver for Apache Spark
- NiFiBolt and NiFiSpoutReceiver for Apache Storm
- StandardRemoteGroupPort, this is the Remote Process Group processor in a NiFi data flow
input-ports/
The input-ports endpoint is used for sending data from source NiFi to target NiFi. When a transaction is created, a HTTP POST request is created towards a transactionId-1. Then while there is more data packet to be sent, the same HTTP POST request will be used, to send data in streaming manner. The POST request also returns the final transaction location, which is transactionId-2. When the client finishes sending all data packets to output stream, it flushes the stream. Finally, the client send a DELETE request to complete the transaction towards transactionId-2.
PlantUML |
---|
actor A_component
actor HttpClient
actor HttpClientTransaction
actor SiteToSiteRestApiUtil
actor SiteToSiteResource
' comment: initialize
A_component -> HttpClient: createTransaction
HttpClient -> SiteToSiteRestApiUtil: initiateTransaction
SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/input-ports/{portId}/transactions
SiteToSiteRestApiUtil <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion
HttpClient <-- SiteToSiteRestApiUtil
HttpClient -> HttpClientTransaction: new
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED
HttpClientTransaction |
PlantUML |
actor A_component actor HttpClient actor HttpClientTransaction actor SiteToSiteRestApiUtil actor SiteToSiteResource ' comment: initialize A_component -> HttpClient: createTransaction HttpClient -> SiteToSiteRestApiUtil: initiateTransactionopenConnectionForSend SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/input-ports/{portId}/transactions SiteToSiteRestApiUtil/{transactionId-1} HttpClient <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion HttpClient <-- SiteToSiteRestApiUtil HttpClient -> HttpClientTransaction: new HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED HttpClientTransaction -> SiteToSiteRestApiUtil: openConnectionForSend SiteToSiteRestApiUtilHttpClientTransaction A_component <-- HttpClient: Transaction 'comment: receive alt while there is data packet to send A_component -> HttpClientTransaction: send HttpClientTransaction -> SiteToSiteResource: writes POSTdata /site-to-site/input-ports/{portId}/transactions/{transactionId-1} HttpClient <-- HttpClientTransaction to outputstream HttpClientTransaction -> HttpClientTransaction: state = DATA_EXCHANGED A_component <-- HttpClient: Transaction HttpClientTransaction end 'comment: receiveconfirm alt while there is data packet to send A_component -> HttpClientTransaction: send A_component -> HttpClientTransaction: confirm HttpClientTransaction -> SiteToSiteRestApiUtil: finishTransferFlowFiles SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionId-2, serverChecksum HttpClientTransaction -> SiteToSiteResourceHttpClientTransaction: writes datavalidate toserver outputstreamChecksum HttpClientTransaction -> HttpClientTransaction: state = DATATRANSACTION_EXCHANGEDCONFIRMED A_component <-- HttpClientTransaction end 'comment: confirmcomplete A_component -> HttpClientTransaction: confirmcomplete HttpClientTransaction -> SiteToSiteRestApiUtil: commitTransferFlowFiles SiteToSiteRestApiUtil -> SiteToSiteResource: finishTransferFlowFiles DELETE /site-to-site/input-ports/{portId}/transactions/{transactionId-2} SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionId-2, serverChecksum200 OK HttpClientTransaction <-- SiteToSiteRestApiUtil HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_CONFIRMEDCOMPLETED A_component <-- HttpClientTransaction 'comment: complete A_component -> HttpClientTransaction: complete HttpClientTransaction -> SiteToSiteRestApiUtil: commitTransferFlowFiles SiteToSiteRestApiUtil -> SiteToSiteResource: DELETE /site-to-site/input-ports/{portId}/transactions/{transactionId-2} SiteToSiteRestApiUtil <-- SiteToSiteResource: 200 OK HttpClientTransaction <-- SiteToSiteRestApiUtil HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_COMPLETED A_component <-- HttpClientTransaction |
output-ports/
output-ports/
The output-ports endpoint is used for receiving data from target NiFi to source NiFi. When a transaction is created, a HTTP GET request is created towards a transactionId-1. Then while there is more data packet to be received, the same HTTP GET request will be used, to receive data in streaming manner. The GET request also returns the final transaction location, which is transactionId-2. When the client finishes consuming all data packets from input stream, and confirm() method is called, the client send a DELETE request with Checksum to complete the transaction.
The complete() method doesn't do anything other than update state to TRANSACTION_COMPLETED.
PlantUML |
---|
actor A_component
actor HttpClient
actor HttpClientTransaction
actor SiteToSiteRestApiUtil
actor SiteToSiteResource
' comment: initialize
A_component -> HttpClient: createTransaction
HttpClient -> SiteToSiteRestApiUtil: initiateTransaction
SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/output-ports/{portId}/transactions
SiteToSiteRestApiUtil <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion
HttpClient <-- SiteToSiteRestApiUtil
HttpClient -> HttpClientTransaction: new
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED
HttpClientTransaction -> SiteToSiteRestApiUtil: openConnectionForReceive
SiteToSiteRestApiUtil -> SiteToSiteResource: GET /site-to-site/output-ports/{portId}/transactions/{transactionId-1}
SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionUrl-2
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClient <-- HttpClientTransaction
A_component <-- HttpClient: Transaction
'comment: receive
alt while there is data packet to receive
A_component -> HttpClientTransaction: receive
HttpClientTransaction <-- SiteToSiteResource: read from inputstream
HttpClientTransaction -> HttpClientTransaction: state = DATA_EXCHANGED
A_component <-- HttpClientTransaction: data packet
end
'comment: confirm
A_component -> HttpClientTransaction: confirm
HttpClientTransaction -> SiteToSiteRestApiUtil: commitReceivingFlowFiles(checksum)
SiteToSiteRestApiUtil -> SiteToSiteResource: DELETE /site-to-site/output-ports/{portId}/transactions/{transactionId-2}
SiteToSiteResource -> SiteToSiteResource: validate client Checksum
SiteToSiteRestApiUtil <-- SiteToSiteResource: 200 OK
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_CONFIRMED
A_component <-- HttpClientTransaction
'comment: complete
A_component -> HttpClientTransaction: complete
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_COMPLETED
A_component <-- HttpClientTransaction |
...