Introduction

CloudStack currently does not allow an easy way to add new guest OS types, for example, a standard way to add say, CentOS 6.5.

Part of the reason is since the OS to hypervisor-specific platform mappings are currently hard-coded into the code-base (e.g : https://github.com/apache/cloudstack/blob/master/plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/KVMGuestOsMapper.java)

To support such new OS addition, the current way is to manipulate the DB using upgrade scripts and make the necessary code changes.

This proposal aims to partially mitigate this issue by allowing the CloudStack admin the ability to add new OS in the list, and update the mapping to hypervisor-specific platform names, via APIs / UI. For now, the admin will be responsible for providing the mapping to hypervisor-specific platform names based on his knowledge, which may be enhanced in future.

Scope

  1. Should be able to add guest os
  2. Should be able to delete guest os
  3. Should be able to modify guest os
  4. Should be able to add guest os -> hypervisor emulator platform mapping
  5. Should be able to delete guest os -> hypervisor emulator platform mapping
  6. Should be able to modify guest os -> hypervisor emulator platform mapping

Implementation

At the core, the mappings will be stored in a table in DB.The table guest_os_hypervisor may be reused for this purpose (it is not being used elsewhere)

To accommodate Xen specifics (mappings are version specific in some cases), the hypervisor version will be stored in the table too.

Additionally, a table will be needed to store the available platform emulators for a hypervisor.

The admin is responsible for passing the right mapping.

The VM TO will contain an additional field to contain the platform emulator name (i.e. the mapping value). This change is implemented in the TO since the resource layer cannot talk to the DB, and TO is the only way to communicate the mapping.

In addition, the snapshot commands use this mapping too. The commands sent to the agent will contain the corresponding mapping needed.

In case a mapping is not found, we let the hypervisor take the decision to error out or not.

Since the OS and mappings are soft-deleted (i.e., use the "removed" column as standardized by the DAO layer), it will conform to the user-view of mappings while the command is being executed.

API

The APIs needed are :

  1. Create new guest os type
    1. oscategory is required
    2. display name is required. Display names are unique
    3. name is optional (defaults to NULL)
  2. Modify guest os type
    1. UUID of guest os to modify is required
    2. display name is required. Display name should be unique
  3. Remove guest os type. It is a soft delete.
    1. UUID is required
  4. List all mappings
    1. hypervisor as parameter. Not mandatory. This will list all mappings for given hypervisor
    2. hypervisor version as parameter. Not mandatory. This is used in conjunction with hypervisor
  5. Add new mapping. Does not allow duplicate mappings
    1. hypervisor, hypervisor version, guest OS id and platform emulator name as parameters
  6. Remove existing mapping. This is a soft delete
    1. UUID of mapping is required
  7. Modify existing mapping
    1. UUID is required
    2. hypervisor specific name is required.

The APIs will not allow for duplicate guest OS -> platform emulator mappings for a specific hypervisor.

All the APIs, other than list, shall be async APIs.

The APIs above are sufficient to implement a UI on top of this.

Sample responses

listOsMapping

{"count"=>1, "guestosmapping"=>[{"id"=>"3ef8e918-aa14-11e3-9293-8674b42bba24", "hypervisor"=>"XenServer", "hypervisorversion"=>"default", "ostypeid"=>"35b4370e-aa14-11e3-9293-8674b42bba24", "osdisplayname"=>"CentOS 4.5 (32-bit)", "osnameforhypervisor"=>"CentOS 4.5 (32-bit)"}}

addGuestOs

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.AddGuestOsCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"ostype"=>{"id"=>"8c6eaca7-6bd5-4600-8524-62d66e4a1913", "oscategoryid"=>"abdbfeee-a947-11e3-9293-8674b42bba24", "description"=>"amoghtestasync"}}, "created"=>"2014-03-11T15:09:57-0700", "jobid"=>"fcad9481-9de2-4443-b456-b26a94acd9f2"}

updateGuestOs

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.UpdateGuestOsCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"ostype"=>{"id"=>"8c6eaca7-6bd5-4600-8524-62d66e4a1913", "oscategoryid"=>"abdbfeee-a947-11e3-9293-8674b42bba24", "description"=>"Windows XP SP2 (32-bitss)"}}, "created"=>"2014-03-11T15:11:09-0700", "jobid"=>"327c83a6-2af5-45bf-bbe2-b609762f42a8"}

removeGuestOs

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.RemoveGuestOsCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"success"=>true}, "created"=>"2014-03-11T15:11:36-0700", "jobid"=>"00baa71f-0af5-40f5-884a-b60a6af1224a"}

addGuestOsMapping

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.AddGuestOsMappingCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"guestosmapping"=>{"id"=>"73bd536f-a5ee-4a9a-b14d-5d9bc9040be8", "hypervisor"=>"KVM", "hypervisorversion"=>"default", "ostypeid"=>"abe1a236-a947-11e3-9293-8674b42bba24", "osdisplayname"=>"CentOS 6.4 (64-bit)", "osnameforhypervisor"=>"amoghtestxen"}}, "created"=>"2014-03-11T15:13:06-0700", "jobid"=>"b7e444de-f998-42a5-885c-861a200ea229"}

updateGuestOsMapping

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.UpdateGuestOsMappingCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"guestosmapping"=>{"id"=>"73bd536f-a5ee-4a9a-b14d-5d9bc9040be8", "hypervisor"=>"KVM", "hypervisorversion"=>"default", "ostypeid"=>"abe1a236-a947-11e3-9293-8674b42bba24", "osdisplayname"=>"CentOS 6.4 (64-bit)", "osnameforhypervisor"=>"amoghsname"}}, "created"=>"2014-03-11T15:14:51-0700", "jobid"=>"3abbcb0a-b9b4-46cf-aafd-bedc2b2f0399"}

removeGuestOsMapping

{"accountid"=>"acc29016-a947-11e3-9293-8674b42bba24", "userid"=>"acc2dfe4-a947-11e3-9293-8674b42bba24", "cmd"=>"org.apache.cloudstack.api.command.admin.guest.RemoveGuestOsMappingCmd", "jobstatus"=>1, "jobprocstatus"=>0, "jobresultcode"=>0, "jobresulttype"=>"object", "jobresult"=>{"success"=>true}, "created"=>"2014-03-11T15:16:06-0700", "jobid"=>"d38b54b6-fec4-4eef-b1df-e0112c87389f"}

 

DB Schema

Schema shall be changed as :

  • GUEST_OS_HYPERVISOR table that stores the mappings:

 

ALTER TABLE `cloud`.`guest_os_hypervisor` ADD COLUMN `hypervisor_version` varchar(32) NOT NULL DEFAULT 'default' COMMENT 'Hypervisor version for this mapping';
ALTER TABLE `cloud`.`guest_os_hypervisor` ADD COLUMN `uuid` varchar(40) COMMENT 'UUID of the mapping';
ALTER TABLE `cloud`.`guest_os_hypervisor` ADD CONSTRAINT `uc_guest_os_hypervisor__uuid` UNIQUE (`uuid`);
ALTER TABLE `cloud`.`guest_os_hypervisor` ADD COLUMN `created` datetime COMMENT 'Time when mapping was created';
ALTER TABLE `cloud`.`guest_os_hypervisor` ADD COLUMN `removed` datetime COMMENT 'Time when mapping was removed if deleted, else NULL';
UPDATE `cloud`.`guest_os_hypervisor` SET `uuid` = UUID();
UPDATE `cloud`.`guest_os_hypervisor` SET `created` = now();

 

  • GUEST_OS table changes :

ALTER TABLE `cloud`.`guest_os` ADD COLUMN `created` datetime COMMENT 'Time when Guest OS was created in system';
ALTER TABLE `cloud`.`guest_os` ADD COLUMN `removed` datetime COMMENT 'Time when Guest OS was removed if deleted, else NULL';
UPDATE `cloud`.`guest_os` SET `created` = now();

 

The ResourceBase / OsMapper files will load the mappings from DB, via the respective Gurus.

Comments / Suggestions

The dev mail thread is here :

markmail.org/message/uw3jfnrv2ssp4h6c

  • No labels

7 Comments

  1. Alex Huang

    Today, it's hard-coded so it's actually much worse than stated.

     

    • When a hypervisor is added, do we autodetect the mapping or require someone to add it specifically?
    • Is there a UI?
    • There's already an existing table but I believe it's not being used and is not complete.
    • Should spell out when the mapping is retrieved.  A sequence diagram that shows Guru and Resource would be nice.
    • Is there a different API for each hypervisor?

  2. Amogh Vasekar

    1. We ask someone to add it manually for the current phase
    2. No
    3. True, the table guest_os_hypervisor is being reused
    4. There are two cases here
      1. For deploy VM, the mapping is retrieved when the TO for VM is being created
      2. For snapshots (XenServer particularly), the mapping is retrieved when the SnapshotCmd is being sent. This is needed in XenServer since the implementation for snapshots first creates a blank template using the platform emulator, before loading the snapshot
    5. No, currently the API takes hypervisor as parameter. Discussed this with Alex, and the change will need each hypervisor having its own "plugin" architecture

  3. Amogh Vasekar

    1. If the existing entries can be deleted by admins or not. And what would happen if the admin deletes the default entries and then upgrades to a newer version.

    – This is a good point. Will change the table to add additional field "isSystemDefined" to distinguish. Admin specified mappings will override the defaults

    2. If the mapping is in code as supposed to DB, then how would these APIs / UI help?

    -- The hypervisor Gurus are being changed to address this

    3. I am assuming this would work for all Hypervisors but this is not mentioned in the FS.

    – Correct

    4. Can you update the FS if the UI is going to make it or not?

    -- There are no plans for UI for initial implementation

  4. manasa

    1.  What if the user is using the guest OS and admin is trying to delete/update the same OS ?
    2. When unsupported/invalid guest OS is mapped to hyper-visor, will the API throw proper error message ?

    1. Amogh Vasekar

      1. It is a soft delete, hence any existing referred IDs will be referred throughout the lifetime of the product.
      2. No, validation is not in scope of the feature.

  5. Manan Shah

    Can you add some clarity on the following 3 questions:

    1. When a new version of the product is released with new entries that are duplicate of the ones that admins have entered, would the UI show duplicate entries and is the admin required to manually delete those entries?
    2. Can the product released entries be deleted by admins?
    3. What happens when admin tries to delete an entry based on which there are running VMs?

    1. Amogh Vasekar

      1. Yes, the admin will be required to manually remove his defined guest entry.
      2. System defined entries cannot be modified / deleted.
      3. It is a soft delete, hence any existing referred IDs will be referred throughout the lifetime of the product. It may not be used to create new VMs etc.