2018-02-08 Meeting Notes

Agenda:

Minutes:

1. Status Updates
   1. Yahoo/Salesforce Merge
      1. All yahoo changes are either merged or PRs are out for review. The remaining big change is "prioritize recovery reads", which is blocked on BC tests and BC tests are blocked on flakiness running on jenkins CI.
      2. Salesforce: the planned merges are all out merged or out of review.
   2. BC testing
      1. BC tests are in. Ivan is working on enabling BC tests on Jenkins CI. There is flakiness on 4.6.x shutdown. It seems that the shutdown hook is not triggered in 4.6.x sometimes.
   3. 4.7.0 release
      1. Sijie: it is planned to be released end of Feb
      2. JV: what big changes are included in 4.7.0
      3. Sijie: need to go through github issues on what already merged. for what are not (fully) merged:
         1. BP-29: have metadata api module in 4.7.0 so it allows supporting multiple metadata implementations in 4.8.0 (e.g. BP-28: etcd).
         2. BP-30: include this as a preview/contrib module in 4.7.0, start moving this effort forward in the releases onward.
         3. BP-27: the new bookkeeper CLI
2. JV's durability proposal
   1. JV is going to create a master BP for tracking all the efforts on addressing the durability concerns.
3. Metadata Propsoals
   1. Sijie is giving the thoughts behind BP-29, BP-28 and BP-30. BP-29 is to provide a clear metadata api module. BP-28 and BP-30 are the two parallel efforts on this.
   2. Ivan is asking about Etcd vs ZooKeeper
      1. Sijie: zookeeper ties session together with connections. a connection is not usable after session expires. it impacts non-session related operations. Etcd decouples this: lease ("session" in etcd) and kv usage and also decouple lease from physical connections.
      2. Sam: The snapshotting is different in etcd. zookeeper dumps the whole memory, while etcd has on-disk store, which it doesn't need to dump the whole state to disk. This allows scaling up the amount of metadata can be stored on metadata store.
      3. JV: Etcd can place watchers on range of keys.
   3. Sijie: all the BPs are sent out for review. Please spend some time on reviewing this and let's move the discussion forward.
4. BPs:
   1. BP-14: asking for helps
   2. BP-27:
      1. Charan is asking for what are the rules for classifying commands. 
      2. Clarification on the usage of annotations : "Parameter", "Parameters"
         1. Sijie: these annotations are not lombok annotations, they are jcommander annotations. The new cli framework for parsing commandlines to support command groups.
5. Jepsen
   1. JV has been working with Ivan on running jepsen.
   2. Ivan is going to write a doc about that.