Page History

...

This framework provides CloudStack users the ability to backup their guest VMs for recovery purposes, in case they suffer a hardware or software issue with their instances or the underlying infrastructure. This framework allows CloudStack to be integrated with Backup and Recovery providers, as pluggable solutions. Each provider should implement the backup and recovery methods provided by the framework in their own way. Previous draft: Backup & Recovery

Use Case

Users wish to be able to back up their guest instances for recovery purposes should they suffer a hardware or software issue.

The current mechanism which users leverage is the volume snapshot feature as this has the feature set closest to that of a backup regime. i.e. images are stored on alternate location (although this may actually be the same physical array) it can be scheduled and users can set how many backups can be kept.

The volume snapshot mechanism causes a VM snapshot to be taken, then the required volume transferred to secondary storage from a hypervisor host in the cluster within which the VM resides. In the case of VMware, the image is compressed into an OVA by the SSVM.

Problems

This transfer can be slow and at times unreliable. Also the user cannot snapshot other volumes attached to the VM during this time, nor perform a number of other BAU operations. Large transfers and concurrency issues can saturate available bandwidth on network or storage.

The requirements for the a feature to fix this problem are:

Must Have

• Users able to backup whole VM or individual volumes
• Users able to schedule their backups and keep n versions
• Users able to restore VM in-place (overwrite existing VM)
•  Ability for Cloud Operators to leverage hardware capabilities of their respective storage solutions.
•  Seamless operation of SAN assisted backup vs non-assisted backups

Highly Desirable

• Users able to restore to alternate location
• Users able to restore to original location when original VM has been deleted
• Support for commercial backup software; Rubrik, Veeam, CommVault
• Support for 3rd Party Backup Solutions (e.g. Amanda)
• Support for in-guest (client based) backup solutions.

Nice to Have

• Support for Grandfather, Father, Child backup.
• User option for storage location
• Multi-tier archiving of volumes

Feature Specification

High level component diagram

It is possible to register a Backup and Recovery provider per zone. The entry point for each API request is the BackupManager. The BackupManager invokes the corresponding BackupProvider registered in a zone, and it should communicate with the Backup and Recovery provider API Server. A high level representation of this is shown in the following diagram:



...

APIs

The Backup and Recovery framework defines new functionalities which should be implemented by each Backup and Recovery provider plugin as a pluggable solution.

...

• Register a Backup and Recovery provider in a zone: This is done via dynamic Zone scoped configurations: 'backup.framework.enabled' and 'backup.framework.provider.plugin'
  • 'backup.framework.enabled' is false by default
  • 'backup.framework.provider.plugin' is 'dummy' by default. For a zone we can set the Backup provider name is lowercase. Available values for the moment are: 'dummy' and 'veeam'
• List available Backup and Recovery providers in CloudStack: listBackupProviders API method. 
• List Backup and Recovery Provider Policies: listBackupPolicies API method, setting the parameter 'external=true' and zoneid=ZONE_ID It returns every backup policy existing in the ZONE_ID backup provider. The goal of this method is listing backup policies to allow administrators to import them into CloudStack, so users can assign them to their guest VMs.
• Import a Backup Policy: importBackupPolicy API method. Allows importing an external backup policy from the zone Backup and Recovery provider into CloudStack.
• Delete a Backup Policy from CloudStack: deleteBackupPolicy API method. Removes the previously imported backup policy

For users:

Backend Configuration related:

• listBackupProviders: lists available backup provider plugins
• listBackupProviderOfferings: lists external backup policy/offering from a provider
• importBackupProviderOfferings: allows importing of an external backup policy/offering to CloudStack as a backup offering
• listBackupOfferings: lists CloudStack's backup offerings
• deleteBackupOffering: deletes a backup offering

Offering assignment based:

• assignVMToBackupOffering
• removeVMFromBackupOffering

Backup operational APIs:

• createVMBackup
• listVMBackups
• deleteVMBackup
• restoreVMBackup
• restoreVolumeFromBackup

User-defined backup schedule:

• createBackupSchedule
• updateBackupSchedule
• deleteBackupSchedule
• listBackupSchedules
• List imported Backup Policies: listBackupPolicies API method. Only backup policies imported by the administrator are listed.
• Assign a VM to a Backup Policy: assignBackupPolicy API method. This method assigns a VM to a backup policy 
• List VM Backups: listBackups API method. List existing backups for a VM
• Recover a full backed up VM: restoreBackup API method. Restores a VM entirely, from a previously created backup
• Recover a VM volume and attach it to a VM: restoreBackupVolume API method. Restores a volume and attachs its to a VM

Schema

This feature introduces changes in the schema, adding the following entities:

• Backup table: Represented in the table ' backups'
• Backup PolicyOffering: Represented in the table ' backup_policy'offering
• Backup Policy - VM mapping: Represented in the table 'backup_policy_vm_map'usage helper table: usage_backup 

Backup and Recovery Providers

...

A dummy Backup and Recovery provider is already created , it simply implements each of the methods above. for testing purposes.

Screenshots

Zone level settings for registering a Backup and Recovery provider

...