Conference theme: Apache EventMesh(incubating) bi-weekly meeting

Meeting time: 2022/07/21 21:00-22:00

Tencent Conference: https://meeting.tencent.com/dm/wes6Erb9ioVV

Conference number: 346-6926-0133

Host: Mike Xue

Topics：

1. Feature progress synchronization

a.WebHook integration (about 70% overall)——Weiming

    • The structure of the runtime server will be adjusted. The changes will be completed this week, and the self-test will be completed next week. A PR will be submitted for review.
    • The logic of trace needs to be confirmed.

b. DashBoard Development - Xiaoyang

    • The management of the client is completed, in the PR review, waiting for the merge;
    • Take client management as an example to guide the students of Open Source Summer;
    • Prepare a dashboard demo;
    • Pay attention to EventCatelog, follow-up DashBoard needs to consider implementing similar management functions;

c. Knative Connector——jojo

    • Producer part submits PR, Code Review;
    • The Consumer part is under development and is expected to submit a PR next week;
    • Thinking about usage scenarios and new features, soliciting ideas from everyone;
    • Check whether the Knative official accepts the adaptation of EventMesh (after the function development of EventMesh is completed, you can consult it on Knative's github)
    • Pay attention to some changes in the HTTP interface of the new version of EventMesh;

d. Kafka Connector——Haohai

    • PR submission of the Consumer part, in Code Review;
    • Producer is partly developed and is expected to continue investing this weekend;

e.Etcd Registry——Jianbo

    • There is a problem with the build on Mac OS and linux. After the meeting, we will open a specific failure log to troubleshoot and solve it;
    • PR has been submitted, Code Review;
    • The gRPC that etcd-core depends on conflicts with the version that EventMesh depends on;

f. Consul Registry

    • Submitted PR, CI/CD failed some tests;

2.workflow arrangement scheme——yuweizhu

  • Program document
  • Arrange a separate meeting to discuss specific implementation details;

3. Storage API specification and design——Lao Hu

  • Discuss and revise the details of the plan, and synchronize them after the revision is completed;

4. GSOC project

  • Mid-term review: pay attention to the review time - Xiaoyang, jojo

5. Summer of Open Source Project

a.Dledger project——Liu Wei

i. An exploratory implementation of a version referring to standalone, the Dledger interface is slightly simpler and there is no pub/sub mechanism. Currently considering the final effect based on Dledger. Plans and issues will be discussed separately in a special meeting.

ii.DLedgerClient only provides append and get methods, so if you install DLedger Server externally and use client to access, the basic storage scheme is as follows:

1. Data storage: use a DLedger group to store all topics, and get an incremental index every time a message of a topic is written

2. Metadata storage: each Topic and the corresponding index need to be recorded, probably a data structure of type Map<String, Queue<Long>>

3. Problem: Map<String, Queue<Long>> is stored in memory, and it will be lost once it crashes, so we need to consider how to persist it

iii. Refer to RocketMQ's DLedgerCommitLog to build DLedger Server. In this way, the DLedgerStore can be operated directly. (simple research) 1. However, the interface provided by DLedgerStore is also relatively simple, with relatively more functions such as getLedgerEndIndex and getCommittedPos. You still need to implement the publish-subscribe mechanism yourself.

iv. Pulsar's BookKeeper is also an additional storage, and its storage structure has three layers: Entry, Ledger, and Bookie. Entry is a record, Ledger is a collection of Entry, and Bookie is a collection of Ledger. The storage structure of DLedger has only two layers: commitlog and DLedger Server. A commitlog is a record, and DLedger Servers belonging to the same group jointly maintain a set of commitlogs.

2. Notes on community contribution

a. ISSUE and PR should be divided into independent small functions as far as possible, so as to facilitate more effective code review of PR.

3. Interactive Q&A

a. The implementation details of the implementation of the three network protocols (TCP, HTTP, gRPC) - mention the ISSUE to describe the specific problems.

b. The problem of Nacos plug-in implementation;

  • No labels