You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Introduction

This framework provides CloudStack users the ability to backup their guest VMs for recovery purposes, in case they suffer a hardware or software issue with their instances or the underlying infrastructure. This framework allows CloudStack to be integrated with Backup and Recovery providers, as pluggable solutions. Each provider should implement the backup and recovery methods provided by the framework in their own way. Previous draft: Backup & Recovery

Use Case

Users wish to be able to back up their guest instances for recovery purposes should they suffer a hardware or software issue.

The current mechanism which users leverage is the volume snapshot feature as this has the feature set closest to that of a backup regime. i.e. images are stored on alternate location (although this may actually be the same physical array) it can be scheduled and users can set how many backups can be kept.

The volume snapshot mechanism causes a VM snapshot to be taken, then the required volume transferred to secondary storage from a hypervisor host in the cluster within which the VM resides. In the case of VMware, the image is compressed into an OVA by the SSVM.

Problems

This transfer can be slow and at times unreliable. Also the user cannot snapshot other volumes attached to the VM during this time, nor perform a number of other BAU operations. Large transfers and concurrency issues can saturate available bandwidth on network or storage.

The requirements for the a feature to fix this problem are:

Must Have

  • Users able to backup whole VM or individual volumes
  • Users able to schedule their backups and keep n versions
  • Users able to restore VM in-place (overwrite existing VM)
  •  Ability for Cloud Operators to leverage hardware capabilities of their respective storage solutions.
  •  Seamless operation of SAN assisted backup vs non-assisted backups

Highly Desirable

  • Users able to restore to alternate location
  • Users able to restore to original location when original VM has been deleted
  • Support for commercial backup software; Rubrik, Veeam, CommVault
  • Support for 3rd Party Backup Solutions (e.g. Amanda)
  • Support for in-guest (client based) backup solutions.

Nice to Have

  • Support for Grandfather, Father, Child backup.
  • User option for storage location
  • Multi-tier archiving of volumes

Feature Specification

High level component diagram

It is possible to register a Backup and Recovery provider per zone. The entry point for each API request is the BackupManager. The BackupManager invokes the corresponding BackupProvider registered in a zone, and it should communicate with the Backup and Recovery provider API Server. A high level representation of this is shown in the following diagram:



APIs

The Backup and Recovery framework defines new functionalities that should be implemented by each Backup and Recovery provider plugin as a pluggable solution.

For administrators:

  • Register a Backup and Recovery provider in a zone: This is done via dynamic Zone scoped configurations: 'backup.framework.enabled' and 'backup.framework.provider.plugin'
    • 'backup.framework.enabled' is false by default
    • 'backup.framework.provider.plugin' is 'dummy' by default. For a zone we can set the Backup provider name is lowercase. Available values for the moment are: 'dummy' and 'veeam'
    • 'backup.framework.sync.interval': 300 by default. B&R framework background sync task internal in seconds that performs GC, syncs usage/stats and schedules backups etc.
  • Veeam B&R provider settings: (all have zone scope, i.e. admin can override on per-zone basis)
    • backup.plugin.veeam.url: Veeam B&R server URL (default:  http://localhost:9399/api/)
    • backup.plugin.veeam.username: Veeam B&R username (default: administrator)
    • backup.plugin.veeam.password: Veeam B&R password (default: P@ssword123)
    • backup.plugin.veeam.validate.ssl: Whether to validate server certificate if Veeam B&R server is SSL/TLS enabled (default: false)
    • backup.plugin.veeam.request.timeout: Veeam B&R API request timeout in seconds (default: 600)

Backend Configuration related:

  • listBackupProviders: lists available backup provider plugins
  • listBackupProviderOfferings: lists external backup policy/offering from a provider
  • importBackupProviderOfferings: allows importing of an external backup policy/offering to CloudStack as a backup offering
  • listBackupOfferings: lists CloudStack's backup offerings (searching via keyword, and pagination supported)
  • deleteBackupOffering: deletes a backup offering by its ID

Offering assignment based:

  • assignVMToBackupOffering: adds a VM to a backup offering (returns backup object)
  • removeVMFromBackupOffering: removes a VM from a backup offering (returns success object), a forced=true parameter can be passed as for Veeam B&R provider this is needed which also removes VM from the Veeam Job deletes any backups

Backup operational APIs:

  • createBackup: creates an adhoc backup for a VM
  • deleteVMBackup: deletes a VM backup (not support for per restore point for Veeam, it should use the removeVMFromBackupOffering API)
  • listBackups
  • restoreBackup
  • restoreVolumeFromBackup

User-defined backup schedule:

  • createBackupSchedule: creates a backup schedule for a VM
  • updateBackupSchedule: updates backup schedule
  • listBackupSchedule: returns backup schedule of a VM if defined

Notes and assumptions:

  • The backup schedule defines a CloudStack driven backup schedule where CloudStack will schedule/start adhoc backups based on provided interval/schedule for a VM.
  • The backup schedule runs in addition to operator-defined settings for an external backup job/offering which may enforce its own schedule and retention limit (such as the number of backups to keep). 
  • The backup schedule is idempotent and can be only created, listed and deleted, there is no update. The reason for this is that similar updateSnapshotPolicy API for example does not allow to change the policy settings, but only the customid/fordisplay fields.

Schema

This feature introduces changes in the schema, adding the following entities:

  • Backup table: backups
  • Backup Offering: backup_offering
  • Backup usage helper table: usage_backup 

Backup and Recovery Providers

Each Backup and Recovery provider plugin in CloudStack should implement the BackupProvider interface.

A dummy Backup and Recovery provider is already created for testing purposes.

Screenshots

Zone level settings for registering a Backup and Recovery provider

  • No labels