Feature Design for: Purge Expunged Resources functionality

Project Introduction

Over the time there are chances of piling up of millions of database records for the removed or expunged resources. The presence of a lot of useless records in the database can also affect the performance of the cloud so it is needed to purge such entries systematically.



Functional Description

  • Systematic means for purging expunged resources or extra database entries.

  • In this iteration, only VirtualMachines are supported. Therefore, only VM records and the records of their associated resources will be purged.
  • Purging will be done through background task, API, and service offering config.
    • Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings.
    • API - New admin-only API `purgeExpungedResources`. It will allow passing the following parameters - `resourcetype`, `batchsize`, `startdate`, `enddate`. Currently, API is not supported in the UI.
    • Config for service offering - Service offerings can be created with `purgeresources` parameter which would allow purging resources immediately on expunge.

1.  New APIs

  • purgeExpungedResources : Purge expunged resources

Parameters:


Parameter nameTypeRequiredDescription
resourcetypeStringNo

The type of the resource which needs to be purged. Supported types: VirtualMachine

batchsizeNumberNoThe size of batch used during purging
startdateStringNo

The start date range of the expunged resources used for purging (use format "yyyy-MM-dd" or "yyyy-MM-dd HH:mm:ss")

enddateStringNo

The end date range of the expunged resources used for purging (use format "yyyy-MM-dd" or "yyyy-MM-dd HH:mm:ss")


2. API Changes

  • createServiceOffering

New parameter named purgeresources to be added in both API and API response.

3. New Global configurations


Config nameDefault valueDescription

expunged.resources.purge.enabled

falseWhether to run a background task to purge the expunged resources

expunged.resources.purge.resources


(empty)A comma-separated list of resource types that will be considered by the background task to purge the expunged resources. Currently only VirtualMachine is supported. An empty "value will result in considering all resource types for purging

expunged.resources.purge.interval


86400Interval (in seconds) for the background task to purge the expunged resources

expunged.resources.purge.delay


300Initial delay (in seconds) to start the background task to purge the expunged resources task.

expunged.resources.purge.batch.size


50Batch size to be used during expunged resources purging.

expunged.resources.purge.start.time


(empty)Start time to be used by the background task to purge the expunged resources. Use format yyyy-MM-dd or yyyy-MM-dd HH:mm:ss.

expunged.resources.purge.keep.past.days


30The number of days in the past from the execution time of the background task to purge the expunged resources for which the expunged resources must not be purged. To enable purging expunged resource till the execution of the background task, set the value to zero.

expunged.resource.purge.job.delay

180Delay (in seconds) to execute the purging of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used.


Database Changes

None