Versions Compared

Old Version 15

changes.mady.by.user Koji Kawamura

Saved on

compared with

New Version 16

changes.mady.by.user Koji Kawamura

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Here are sequence diagrams describing complete HTTP request/response sequence of successful scenario for both input-ports and output-ports. Other semi-normal and error cases are described in Interaction Scenarios.

input-ports/

The A_component in the diagrams is a component which uses SiteToSiteClient class, such as:

  • NiFiReceiver for Apache Spark
  • NiFiBolt and NiFiSpoutReceiver for Apache Storm
  • StandardRemoteGroupPort, this is the Remote Process Group processor in a NiFi data flow

 

input-ports/

The input-ports endpoint is used for sending data from source NiFi to target NiFi. When a transaction is created, a HTTP POST request is created towards a transactionId-1. Then while there is more data packet to be sent, the same HTTP POST request will be used, to send data in streaming manner. The POST request also returns the final transaction location, which is transactionId-2. When the client finishes sending all data packets to output stream, it flushes the stream. Finally, the client send a DELETE request to complete the transaction towards transactionId-2.

 

PlantUML
actor A_component
actor HttpClient
actor HttpClientTransaction
actor SiteToSiteRestApiUtil
actor SiteToSiteResource

' comment: initialize
A_component -> HttpClient: createTransaction
HttpClient -> SiteToSiteRestApiUtil: initiateTransaction
SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/input-ports/{portId}/transactions
SiteToSiteRestApiUtil <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion
HttpClient <-- SiteToSiteRestApiUtil
HttpClient -> HttpClientTransaction: new
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED
HttpClientTransaction
PlantUML
actor A_component
actor HttpClient
actor HttpClientTransaction
actor SiteToSiteRestApiUtil
actor SiteToSiteResource

' comment: initialize
A_component -> HttpClient: createTransaction
HttpClient -> SiteToSiteRestApiUtil: initiateTransactionopenConnectionForSend
SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/input-ports/{portId}/transactions
SiteToSiteRestApiUtil/{transactionId-1}
HttpClient <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion
HttpClient <-- SiteToSiteRestApiUtil
HttpClient -> HttpClientTransaction: new
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED
HttpClientTransaction -> SiteToSiteRestApiUtil: openConnectionForSend
SiteToSiteRestApiUtilHttpClientTransaction
A_component <-- HttpClient: Transaction

'comment: receive
alt while there is data packet to send
    A_component -> HttpClientTransaction: send
	HttpClientTransaction -> SiteToSiteResource: writes POSTdata /site-to-site/input-ports/{portId}/transactions/{transactionId-1}
HttpClient <-- HttpClientTransaction
to outputstream
	HttpClientTransaction -> HttpClientTransaction: state = DATA_EXCHANGED
	A_component <-- HttpClient: Transaction
HttpClientTransaction
end
 
'comment: receiveconfirm
alt while there is data packet to send
    A_component -> HttpClientTransaction: send
	A_component -> HttpClientTransaction: confirm
HttpClientTransaction -> SiteToSiteRestApiUtil: finishTransferFlowFiles
SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionId-2, serverChecksum
HttpClientTransaction -> SiteToSiteResourceHttpClientTransaction: writes datavalidate toserver outputstreamChecksum
	HttpClientTransaction -> HttpClientTransaction: state = DATATRANSACTION_EXCHANGEDCONFIRMED
	A_component <-- HttpClientTransaction
end
 
'comment: confirmcomplete
A_component -> HttpClientTransaction: confirmcomplete
HttpClientTransaction -> SiteToSiteRestApiUtil: commitTransferFlowFiles
SiteToSiteRestApiUtil -> SiteToSiteResource: finishTransferFlowFiles DELETE /site-to-site/input-ports/{portId}/transactions/{transactionId-2}
SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionId-2, serverChecksum200 OK
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_CONFIRMEDCOMPLETED
A_component <-- HttpClientTransaction

'comment: complete
A_component -> HttpClientTransaction: complete
HttpClientTransaction -> SiteToSiteRestApiUtil: commitTransferFlowFiles
SiteToSiteRestApiUtil -> SiteToSiteResource: DELETE /site-to-site/input-ports/{portId}/transactions/{transactionId-2}
SiteToSiteRestApiUtil <-- SiteToSiteResource: 200 OK
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_COMPLETED
A_component <-- HttpClientTransaction
 

output-ports/

 
output-ports/
The output-ports endpoint is used for receiving data from target NiFi to source NiFi. When a transaction is created, a HTTP GET request is created towards a transactionId-1. Then while there is more data packet to be received, the same HTTP GET request will be used, to receive data in streaming manner. The GET request also returns the final transaction location, which is transactionId-2. When the client finishes consuming all data packets from input stream, and confirm() method is called, the client send a DELETE request with Checksum to complete the transaction.
The complete() method doesn't do anything other than update state to TRANSACTION_COMPLETED.
 PlantUML
actor A_component
actor HttpClient
actor HttpClientTransaction
actor SiteToSiteRestApiUtil
actor SiteToSiteResource

' comment: initialize
A_component -> HttpClient: createTransaction
HttpClient -> SiteToSiteRestApiUtil: initiateTransaction
SiteToSiteRestApiUtil -> SiteToSiteResource: POST /site-to-site/output-ports/{portId}/transactions
SiteToSiteRestApiUtil <-- SiteToSiteResource: transactionUrl-1, transactionProtocolVersion
HttpClient <-- SiteToSiteRestApiUtil
HttpClient -> HttpClientTransaction: new
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_STARTED
HttpClientTransaction -> SiteToSiteRestApiUtil: openConnectionForReceive
SiteToSiteRestApiUtil -> SiteToSiteResource: GET /site-to-site/output-ports/{portId}/transactions/{transactionId-1}
SiteToSiteRestApiUtil <-- SiteToSiteResource: 201 Created: transactionUrl-2
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClient <-- HttpClientTransaction
A_component <-- HttpClient: Transaction

'comment: receive
alt while there is data packet to receive
    A_component -> HttpClientTransaction: receive
	HttpClientTransaction <-- SiteToSiteResource: read from inputstream
	HttpClientTransaction -> HttpClientTransaction: state = DATA_EXCHANGED
	A_component <-- HttpClientTransaction: data packet
end
 
'comment: confirm
A_component -> HttpClientTransaction: confirm
HttpClientTransaction -> SiteToSiteRestApiUtil: commitReceivingFlowFiles(checksum)
SiteToSiteRestApiUtil -> SiteToSiteResource: DELETE /site-to-site/output-ports/{portId}/transactions/{transactionId-2}
SiteToSiteResource -> SiteToSiteResource: validate client Checksum
SiteToSiteRestApiUtil <-- SiteToSiteResource: 200 OK
HttpClientTransaction <-- SiteToSiteRestApiUtil
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_CONFIRMED
A_component <-- HttpClientTransaction
 
'comment: complete
A_component -> HttpClientTransaction: complete
HttpClientTransaction -> HttpClientTransaction: state = TRANSACTION_COMPLETED
A_component <-- HttpClientTransaction
 
...