This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • 2019-02-21 Meeting Notes
Skip to end of metadata
Go to start of metadata



  • (Enrico) BlobIt - An object storage over bookkeeper providing swift compatible API

    • Discussed the implementation

      • HerdDB to store the mapping between buckets to ledgers

      • Using LedgerAdv API

      • AI: Enrico will send an email to mailing list about the details. Will talk about more on production

      • Not on cloud, but on prem

    • BookKeeper API is limited, lacking the mapping between naming and ledger id. Ledger Id is 64 bits (can be extended to 128 bits)

    • Scale: 3-5 machines, 30 millions blobs per day, keep them for 2 weeks

  • (Enrico) Started using dlog for some WAL use cases

  • (Enrico & Sijie) : Talked about Zoom, Google Docs, and ASF policy.

  • (Sijie): Documentation of BookKeeper

  • (Sijie): BP-38

    • AI: Sijie ask Arvin to join next meeting discussion

    • JV:

      • Proposal:

        • Close-to-open semantic

        • Add a close RPC, similar as what we do for force API

        • If this type of ledger is not close, we just throw it away

      • Salesforce is working at metadata scrutiny. It will skip this type of ledgers if they are not closed.

        • Local Checker (contributed from Sam and merged to master)

        • Distributed Checker: (Charan is working on)

          • Ensemble placement policy

          • ZooKeeper metadata and index files are agreeing or not.

  • (Sijie): Bookie Id Advertisement

    • This is related to cloud deployment

    • Getting rid of using ip address + port as the id, instead using UUID as the bookie id.

    • Discussed proposal:

      • Bookie : init - generate UUID and record the UUID as part of cookie

      • A mapping between UUID and bookie’s network location

      • Bookie can advertise additional information

        • Supported protocol

        • Supported versions (support forward compatibility)

      • AI: Sijie put down a BP for this

  • Etcd:

    • Updates (from Sijie):

      • Everything except autorecovery is done

      • The remaining piece is the metadata interfaces used by Audit

      • AI: Sijie ping the author of Etcd driver to join the next weekly meeting

  • (Sijie):

    • Weekly Bug Triage

      • AI: Sijie writes down the weekly bug triage process that he is doing with Ivan

  • Issue 1949:

    • Agreed on that using rocksdb as entry map is not a good approach.

    • It requires more thoughts and discussions on the PR. We need to address the fundemental problem of the issue first before choosing a completely different approach.

  • No labels