Introduction

Currently we use redundant virtual router enabled networks to prevent network outage even if one of the virtual router fails. This is achieved by using redundant virtual routers along with VRRP protocol. This has increased the network robustness. However  updating a network with new network offering involves some downtime due to Vr restarts, this causes loss of existing connections and interrupts network services. In case of redundant VR based networks we can sequence the network update operations to update the backup first and then update the master while master's operations are being handled by backup.

Purpose

The purpose of this feature is to minimise network downtime during network updates. The current scope of this feature is to address the downtime only in case of redundant VR based networks.

References

JIRA Tickets

CLOUDSTACK-8751

Functional requirements

• Sequence the router restarts in RVR based networks to minimise the network downtime during network update.
• The network update in non RVR networks will function as earlier.
• No additional API will be added to achieve this, the existing update network API will  suffice.
• No new network operations will be allowed when network update is in progress.
• All the existing network connections and services will not be affected.

Work Flow

There are no changes to the workflow of updating a network.  This will only change the behaviour of network restart when redundant VR is being used.

Design.

Element level changes.

We want to implement this feature as in a generic way i.e. any element which supports redundancy and offers routing services like virtual router should be able to do this without changing the orchestration code every time. So in

order to do this  we are introducing a new interface UpdateResourcesInSequence. Any network element which wants to take advantage of redundancy and update its resource in sequence should implement this interface. Presently 

we are adding this ability to VirtualRouter so we have modified virtulalrotuerElement to implement this.

Interface UpdateResourcesInSequence.

While updating the state of the router is tracked using the update_state(newly added) column of the domain_router column. The state of the router during update can be updateInProgress or updateNeeded or updateComplete.

VirtualRouterElement Changes.

We have modified the VritualRouerElement to implement UpdateResourcesInSequence and added a new method getRouters(Network). This method is called when ever the virtual router details corresponding to a particular network are

needed. The getRouters method always returns the router which is currently being updated or the one Which needs to be updated if there is no router whose update is in progress. The update state of the router is tracked based on the 

update_state column in the domain_router table. The getRouters method functions as earlier if network update is not in progress, This can be check by the updateInProgress flag in the network details table. This behaviour of the getRouters

ensures that only the router which is being updated is accessible while the update is in progress. This leaves the other router (which is not being updated at that time) to service the existing connections. 

Service  and Orchestration level changes.

The service layer exposes the update network service to the api layer. The network update is broken down into three steps, shutdown all the routers, reconfigure the network and then reimplement the routers. The service layer takes 

help of NetworkOrchestrationService(NOS)  to perform these steeps in sequence to update the network. The orchestration service deals with individual elements that proved the network services. We added methods 

 ConfigureUpdateInSequence and isUpdateComplete to theNetworkOrchestrationService interface. Now in the current implementation to enable updating elements in sequence, network service layer calls the ConfigureUpdateInSequence

method if the network supports redundant routers. The ConfigureUpdateInSequence method of NetworkOrchestrationService calls the configureResourceUpdateInSequence of the network element if it implementsUpdateResourceInSequence 

interface.Once the element is configured the network service performs the network element shutdown, network offering update and network element implement repeatedly until the isUpdateComplete method of the network element returns false.

Interface NetworkOrchestrationService

Db schema Changes.

Added a new column update_state to track the state of the router when network update is in progress.

Usage  Changes:

Not applicable.