Versions Compared

Old Version 3

changes.mady.by.user Dingane Hlaluku

Saved on

compared with

New Version Current

changes.mady.by.user Dingane Hlaluku

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

CloudStack users wish to have the ability functionality to add “extra configuration” metadata to a user VM during deployment in a manner that is similar across the 3 major supported hypervisors. This data could be anything ranging from instructions to install hypervisor specific required tools and drivers in each VM that is deployed from a template which does not have these tools installed. To simplify the implementation, the user will pass this additional configuration as URL encoded string that will be decoded and parsed for each hypervisor type. These additional metadata must be included in the VM instance definition.

Any additional configuration setting/command needs to be first included into the list of allowed settings by the Root Admin through a configurable global configuration.  No command/setting is allowed by default.


  • KVM

The primary configuration file for KVM host guest instances is a domain XML. The additional configuration for KVM should be provided in XML format in a URL encoded string which will be parsed and append to the KVM instance XML configuration file builder.

A configurable global config 'allow.additional.vm.configuration.list.kvm' is available for Root admin to specify a list options that are allowed with this feature. with KVM No option is allowed by default. 

Example: the following metadata can be passed to specify guest VM NUMA topology using the <numa> element in the domain XML

...

enable hugepages on KVM hosts, with 'memoryBacking' included in the list of allowed setting/configuration

<memoryBacking>
       <hugepages/>
</memoryBacking>

User needs to pass the following URL encoded string:

...

         %3CmemoryBacking%3E%0D%0A++%3Chugepages%2F%3E%0D%0A%3C%2FmemoryBacking%3E


Another Example: the following examples allows user to add EHCI controller to VM, also with the 'controller' element/tag included in the list of allowed settings/configuration

<controller type='usb' index='1' model='ehci'></controller>

...

%3Ccontroller%20type%3D%27usb%27%20index%3D%271%27%20model%3D%27ehci%27%3E%3C%2Fcontroller%3E


  • XenServer

Similar for this hypervisor, the primary configuration file of the guest VM is built by CloudStack, therefor, the additional metadata will also be parsed and append to the builder.

Example: the following metadata can passed to pass a USB controller to the guest VM:

...

xe vm-param-set manual page

Support for XenServer has only been added for the XAPI's 'vm-param-set' configuration using its arguments/keys as strings.  Below is a full list of all supported operations with this feature;

Simple key=value pair configurations, e.g PV-args=hvc0, is-a-template=true. Any of these settings will override an existing setting with the new value from the user.

A configurable global config 'allow.additional.vm.configuration.list.xenserver' is available for Root admin to specify a list of 'vm-param-set' options that are allowed with this feature. No configuration is allowed by default.

    • is-a-template, 

    • memory-static-max, 

    • memory-dynamic-max, 

    • memory-dynamic-min, 

    • memory-static-min, 

    • VCPUs-max, 

    • VCPUs-at-startup, 

    • HVM-boot-policy, 

    • order,

    • shutdown-delay, Long

    • start-delay, Long

    • ha-restart-priority, 

    • PV-bootloader-args, 

    • PV-bootloader, 

    • PV-legacy-args, 

    • PV-args, 

    • PV-ramdisk, 

    • PV-kernel, 

    • HVM-shadow-multiplier,

To set a (key,value) pair in a map parameter, use the syntax 'map-param:key=value'. Any of this settings do not override already defined params but append to the map. For example 'platform:timeoffset=value' will not override already defined platform settings but will append to the settings this new additional configuration that is being set.

    • VCPUs-params:, 

    • platform:, 

    • HVM-boot-params:, 

    • other-config:, 

    • xenstore-data:


Example: Following commands are used to convert a VM from HVM to PV;, also admin needs to include HVM-boot-policy, PV-bootloader and PV-args for the configuration below to be allowed.

HVM-boot-policy=
PV-bootloader=pygrub
PV-args=hvc0

User needs to pass the following URL encoded string:

            HVM-boot-policy%3D%0APV-bootloader%3Dpygrub%0APV-args%3Dhvc0


For validation, user can log/ssh on to the VM hypervisor host and execute the following commands on the terminal to verify that additional configuration is available in the state.db for that instance :

  • xe vm-param-get param

...

  • -name="HVM-boot-policy" uuid=<instance uuid>
  • xe vm-param-get param-name="PV-bootloader" uuid=<instance uuid>
  • xe vm-param-get param-name="PV-args" uuid=<instance uuid>

And verify that they have been correctly set.


The algorithm for XenServer does the following checks;

  1. Is the command option passed a valid key/value pair 'key1=value1\nkey2=value2' and so on. Throws an Exception if user input is not valid key/value pair.
  2. The next algorithm checks if passed command contains any keys from the allowed list of keys keys and throws a CloudRountimeException if false. This check is case insensitive. 
  • VMware

Similar for this hypervisor, the primary configuration file of the guest VM is built by CloudStack, therefor, the additional metadata will also be parsed and append to the VMX file builder.

A configurable global config 'allow.additional.vm.configuration.list.vmware' is available for Root admin to specify a list options that are allowed with this feature.

Example: the following is an example to set Hyper-V to run on ESXi, with 'hypervisor.cpuid.v0' included on the list of allowed settings/configuration keys.

hypervisor.cpuid.v0= "FALSE" – to run Hyper-V on ESXi

User needs to pass the following URL encoded string:

...

        hypervisor.cpuid.v0%3DFALSE

Similar to the other hypervisors, user need to ssh into the host and examine the vmx file to see extra configuration is present. Execute the following commands from the terminal.

  • find / -name *.vmx
  • cat <vmx file path> | grep \w ‘extra config’

The configuration passed is parsed as key/value pair and every key is validated against against the allowed list of commands set by Root admin. An Exception is thrown if an invalid key is found.


Admin users will be allowed to use this feature with very few restrictions, whilst normal users will be constrained to several restrictions that are controlled by the admin.

...

The ‘deployVirtualMachine’ , and (updateVirtualMachines) API commands are refactored to include a new parameter ‘extraconfig’. This parameter will accept the URL encoded string and CloudStack will store in the 'user_vm_details' under the key 'extraconfig'.Global configuration settings are added with Account scope  in the Service Layer for the admin user to allow and control passing of additional data;

  • A Boolean setting to allow sending of additional data from the API. This config is not dynamic and requires a restart of the CloudStack management server in order for changes to take effect. This setting is added with Account scope.
    • 'enable.additional.vm.configuration'. False by default


  • 3 new settings A new setting with a list of options that are not allowed to passed as additional data. The admin user will populate this setting with a list of tags/commands that will be used for validating against user supplied additional configuration. All this settings are dynamic and should not requires a restart of the CloudStack management server in order for changes to take effect. Please note that this are only available as global settings for the Root admin only.
    • 'allow.additional.vm.configuration.list.kvm'
    • 'allow.additional.vm.configuration.list.xenserver'
    • 'allow.additional.vm.configuration.list.vmware'


UserVmManageImpl class is refactored to include the new Boolean configuration setting to allow/disallow passing of additional data during VM deployment/updating,  and stores the data in the 'user_vm_details' table.

...

VirtualMachineTO class is refactored to carry the additional configuration data from the Service Layer to Hypervisor Layer.

Hypervisor/Resource Layer

KVM : LibvirtComputingResource class is refactored to include additional configuration metadata into the VM domain XML before starting it.

  • The extraconfig is retrieved from the VirtualMachineTO in an XML formatted string and is appended onto the VM's domain XML


XenServer: CitrixResourceBase class is refactored as well to include additional metadata in the primary configuration file builder of the VM before starting it.

  • The extraconfig is retrieved from the VirtualMachineTO as a list of key/value pairs that are supported by the 'vm-param-set' command option and are set using the predefined setters from the 'xenapi' plugin against the target VM.


VMware: VmwareResource is also refactored to include additional configuration into the VMX configuration file builder before starting the VM.

  • All the key/value configuration settings stored under the 'user_vm_details' table are appended to the VM's vmx file by the VMware API and ignoring duplicates.

UI integration

N/A