...
1. Problem statement and context
Figure 1. The barrier between the two technology stacks
...
In the process of application modernization, it is common for users to use multiple microservice technology stacks (servicecomb-service-center and istio) at the same time. So, they have a high demand to break the communication barrier between two heterogeneous tech stacks which is caused by the different Registration and Discovery mechanisms.
...
2. Our solution
Figure 2. Our solution
We propose a tool that can synchronizes microservices from servicecomb-service-center to Istio system which makes istio-based services are able to discover servicecomb-service-center-based services, refer to Figure 2.
...
2.1
...
Architecture
...
Figure 3. Servicecomb-service-center structureatchitecture
As we can see from this Figure 3, servicecomb-service-center-istio synchronizes service register information from servicecomb-service-center side to istio side. It mainly does three steps:
- It watches servicecomb-service-center updates (microservice instance status changes event: add, delete, update) via a WebSocket connection.
- It converts servicecomb-service-center data model (microservice and microservice instance) to Istio data model (ServiceEntry and WorkloadEntry)
- It pushed converted data model to K8S api server through REST api.
2.2 CSE2Mesh
After doing all these steps, Istio automatically captures these converted ServiceEntry and WorkloadEntry and dispatches them to each Envoy sidecar, so that each sidecar can discover those servicecomb-service-center services.
CSE2Mesh containes 3 components:
2.2.1 CSE Controller
CSE controller responses to watch CSE Microservice Instance update.
CSE controller leveraging go-chassis SDK Watch API to watch CSE Update.
The controller will create a websocket connection with a CSE application, and create a watcher for every Microservice, all the update events will be push back from CSE through the websocket connection.
Note: Currently, CSE SDK only provides Microservice instance level watch, so the controller need to fetch all service from CSE every 5 seconds
2.2.2 Translator
Translator is used to convert CSE language (Microservice and Instance) to istio language (Serviceentry and Workloadentry)
2.2.3 Istio controller
Istio controller response to receive update events from CSE and push to K8S api server.
There are serval solution of push configs to K8S, please visit this WIKI page to learn more.
2.2.3.1 Debouncing
CSE Controller is Event driven, for every CSE microservice instance update event will trigger the istio push operation, so when update events are very frequent, there will be frequent istio pushes which gives prassure to Istio side.
In order to solve this we implemented the debouncing feature. Which is a kind of buffer, the buffer responsible for collecting events and then pushing them in a unified manner.
debounce rules:
- When CSE Watcher does not receive a new Event within 1 second after receiving an updated Event from CSE, perform a push operation
- When CSE Watcher receives a new Event within 1 second after receiving the updated Event from CSE, continue to wait for the next 1 second and execute the same logic until a push is made after 10 seconds
- When CSE Watcher receives a new Event within 1 second after receiving the updated Event from CSE, continue to wait for the next 1 second and execute the same logic, if the number of Events exceeds 1000 in this push cycle, execute a pus
2.3 High avalibility
To improve availability, we use the Leader Election function provided by K8S to achieve high availability of CSE2Meh. We also add command line parameters so that users can choose whether to enable high availability or not.
The user can deploy multiple CSE2Mesh in the cluster, but only one will be choosen as the leader and active working, the others are on hold status, whenever the leader is unavailable, a new leader will be elected ad keep working.
2.4 Performance
For performance information please go to Performance evaluation
3. Limitations
When there are large numbers of microservice in CSE, after sync them to Istio, will cause scalability issue, the envoy sidecar's memory consumption will be very high. The performance evaluation result could be found here
Note: To solve this problem you can adopt Lazy xDS
CSE2Mesh passively receives update events from CSE services and performs Istio pushing, while if a user manually removes a serviceentry from K8S/Istio that is synced over, CSE2Mesh does not sense it, so the serviceentry will not be synced again.
...