You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

(warning) WARNING: Work In Progress (warning)

The descriptions of the interactions below take this form.

[Protocol] message( input ) : output

The [Protocol] portion describes the protocol, authentication mechanism and identities exchanged.

Suffixes are used in many cases to denote type
tgt = Kerberos Ticket Granting Ticket
kp = Kerberos Principal: nn-kp = The Kerberos principal for the NameNode nn.
kt = Kerberos Ticket: u-jt-kt = A Kerberos Ticket for User u to access the JobTracker jt.

Secure MapReduce Clientu Clientu KDC KDC NameNode NameNode DataNode DataNode JobTracker JobTracker TaskTracker TaskTracker Task Login 1[KRB]kinit(): tgt Setup 2[KRB]ticketRequest( tgt, jt-kp ): u-jt-kt 3[RPC+SASL/KRB:u-jt-kt]getNewJobId(): job-id 4[KRB]ticketRequest( tgt, nn-kp ): u-nn-kt 5[RPC+SASL/KRB:u-nn-kt]getDelegToken(): deleg-token loop[job-cfg-file, job-jar-file, ?] 6[RPC+SASL/KRB:u-nn-kt]createFile( file-loc, deleg-token ): block-loc, block-token Store jobconfig files loop[blocks] 7[BTP]writeBlock( block-token, block-data ): void Submit 8[RPC+SASL/KRB:u-jt-kt]submitJob( job-id, job-cfg-dir ): status Execute 9[RPC?]heartbeat(): work alt[If deleg-token expired] 10[RPC?]renewDelegToken( deleg-token ): void How does JT know whenDT renewal is required? 11new() Task loop[job-cgf-files, job-jar-file, ?] 12[RPC?]readFile( file-loc, deleg-token ): block-loc, block-token Extract job filesto local dir loop[blocks] 13[BTP]readBlock( block-token ): block-data 14exec( job-jar, job-dir ) How does this change between map and reduce?Where does shuffle fit in? loop[result-files] 15[RPC?]writeFile( file-loc, deleg-token ): block-loc, block-token Store job resultinto HDFS loop[blocks] 16[BTP]writeBlock( block-token, block-data ): void 17[RPC?]status( job-token, status ): void
  • No labels