(Enrico) BlobIt - An object storage over bookkeeper providing swift compatible API
Discussed the implementation
HerdDB to store the mapping between buckets to ledgers
Using LedgerAdv API
AI: Enrico will send an email to mailing list about the details. Will talk about more on production
Not on cloud, but on prem
BookKeeper API is limited, lacking the mapping between naming and ledger id. Ledger Id is 64 bits (can be extended to 128 bits)
Scale: 3-5 machines, 30 millions blobs per day, keep them for 2 weeks
(Enrico) Started using dlog for some WAL use cases
(Enrico & Sijie) : Talked about Zoom, Google Docs, and ASF policy.
(Sijie): Documentation of BookKeeper
AI: Sijie ask Arvin to join next meeting discussion
Add a close RPC, similar as what we do for force API
The close RPC might be related to BP-32: https://github.com/apache/bookkeeper/issues/1343
Sijie & JV agreed on using close RPC rather than using force RPC.
If this type of ledger is not close, we just throw it away
Salesforce is working at metadata scrutiny. It will skip this type of ledgers if they are not closed.
Local Checker (contributed from Sam and merged to master)
Distributed Checker: (Charan is working on)
Ensemble placement policy
ZooKeeper metadata and index files are agreeing or not.
(Sijie): Bookie Id Advertisement
This is related to cloud deployment
Getting rid of using ip address + port as the id, instead using UUID as the bookie id.
Bookie : init - generate UUID and record the UUID as part of cookie
A mapping between UUID and bookie’s network location
Bookie can advertise additional information
Supported versions (support forward compatibility)
AI: Sijie put down a BP for this
Updates (from Sijie):
Everything except autorecovery is done
The remaining piece is the metadata interfaces used by Audit
AI: Sijie ping the author of Etcd driver to join the next weekly meeting
Weekly Bug Triage
AI: Sijie writes down the weekly bug triage process that he is doing with Ivan
Issue 1949: https://github.com/apache/bookkeeper/pull/1949
Agreed on that using rocksdb as entry map is not a good approach.
It requires more thoughts and discussions on the PR. We need to address the fundemental problem of the issue first before choosing a completely different approach.