Currently, CloudStack does not allow passing of additional configuration metadata to guest VMs during deployment when required. This feature aims to provide operators the functionality to pass arbitrary additional configuration metadata that will be included every time the guest VM is started.
In CloudStack, when a new guest VM is deployed by the user and answers questions about the template/iso, service offerings and networking e.t.c, these answers are used to produce the VM's primary configuration file. This file format is dynamically determined according to the supported hypervisor in the cluster which the VM is being deployed. CloudStack supports 3 major hypervisors i.e. KVM, XenServer and VMware, and this feature is implemented to support all of them.
CloudStack users wish to have the ability to add “extra configuration” metadata to a user VM during deployment in a manner that is similar across the 3 major supported hypervisors. This data could be anything ranging from instructions to install hypervisor specific required tools and drivers in each VM that is deployed from a template which does not have these tools installed. To simplify the implementation, the user will pass this additional configuration as URL encoded string that will be decoded and parsed for each hypervisor type. These additional metadata must be included in the VM instance definition.
The primary configuration file for KVM host guest instances is a domain XML. The additional configuration for KVM should be provided in XML format in a URL encoded string which will be parsed and append to the KVM instance XML configuration file builder.
Example: the following metadata can be passed to enable hugepages on KVM hosts specify guest VM NUMA topology using the <numa> element in the domain XML
<memoryBacking>
<hugepages/>
</memoryBacking>
User needs to pass the following URL encoded string:
%3CmemoryBacking%3E%0D%0A++%3Chugepages%2F%3E%0D%0A%3C%2FmemoryBacking%3E
Another Example: the following examples allows user to add EHCI controller to VM
<controller type='usb' index='1' model='ehci'></controller>
User input URL encoded string:
%3Ccontroller%20type%3D%27usb%27%20index%3D%271%27%20model%3D%27ehci%27%3E%3C%2Fcontroller%3E
A configurable global config 'additional.vm.configuration.black.list.kvm' is available for Root admin to specify a list options that are disallowed with this feature.
The keys in the blacklisted commands are used to validate the tags from the user passed XML as extraconfig data. Any tag/element that is found with the name in the blacklist of commands will result to an Exception.
e.g. <name>my vm </name> or <memory unit="KiB">262144</memory> or <currentMemory unit="KiB">262144</currentMemory> will throw an exception by default because this type of XML tags have been blacklisted.
Similar for this hypervisor, the primary configuration file of the guest VM is built by CloudStack, therefor, the additional metadata will also be parsed and append to the builder.
Support for XenServer has only been added for the XAPI's 'vm-param-set' configuration using its arguments/keys as strings. Below is a full list of all supported operations with this feature;
Simple key=value pair configurations, e.g PV-args=hvc0, is-a-template=true. Any of this settings will override existing with the new value from the use, behaviour can be prevented by blacklisting the key in the global config with the list of disallowed keys.
is-a-template,
memory-static-max,
memory-dynamic-max,
memory-dynamic-min,
memory-static-min,
VCPUs-max,
VCPUs-at-startup,
HVM-boot-policy,
order,
shutdown-delay, Long
start-delay, Long
ha-restart-priority,
PV-bootloader-args,
PV-bootloader,
PV-legacy-args,
PV-args,
PV-ramdisk,
PV-kernel,
HVM-shadow-multiplier,
To set a (key,value) pair in a map parameter, use the syntax 'map-param:key=value'. Any of this settings do not override already defined params but append to the map. For example 'platform:timeoffset=value' will not override already defined platform settings but will append to the settings this new additional configuration that is being set.
VCPUs-params:,
platform:,
HVM-boot-params:,
other-config:,
xenstore-data:
Example: Following commands are used to convert a VM from HVM to PV;
HVM-boot-policy=
PV-bootloader=pygrub
PV-args=hvc0
User needs to pass the following URL encoded string:
HVM-boot-policy%3D%0APV-bootloader%3Dpygrub%0APV-args%3Dhvc0
For validation, user can log/ssh on to the VM hypervisor host and execute the following commands on the terminal to verify that additional configuration is available in the state.db for that instance :
And verify that they have been correctly set.
Default values are; ' is-a-template, memory-static-max, memory-dynamic-max, memory-dynamic-min, memory-static-min'
The algorithm for XenServer does the following checks;
Similar for this hypervisor, the primary configuration file of the guest VM is built by CloudStack, therefor, the additional metadata will also be parsed and append to the VMX file builder.
Example: the following is an example to set Hyper-V to run on ESXi
hypervisor.cpuid.v0=FALSE
User needs to pass the following URL encoded string:
hypervisor.cpuid.v0%3DFALSE
Similar to the other hypervisors, user need to ssh into the host and examine the vmx file to see extra configuration is present. Execute the following commands from the terminal.
The configuration passed is parsed as key/value pair and every key is validated against against the blacklisted commands set by Root admin. An Exception is thrown if an invalid key is found.
Admin users will be allowed to use this feature with very few restrictions, whilst normal users will be constrained to several restrictions that are controlled by the admin.
KVM, XenServer and VMware
The ‘deployVirtualMachine’ , and (updateVirtualMachines) API commands are refactored to include a new parameter ‘extraconfig’. This parameter will accept the URL encoded string and CloudStack will store in the 'user_vm_details' under the key 'extraconfig'.
UserVmManageImpl class is refactored to include the new Boolean configuration setting to allow/disallow passing of additional data during VM deployment/updating, and stores the data in the 'user_vm_details' table.
VirtualMachineManagerImpl class is refactored to include methods for retrieving 'extraconfig' from the database and validating against the global setting that contains the list of unwanted arguments before orchestrating the starting of the guest VM.
VirtualMachineTO class is refactored to carry the additional configuration data from the Service Layer to Hypervisor Layer.
KVM : LibvirtComputingResource class is refactored to include additional configuration metadata into the VM domain XML before starting it.
XenServer: CitrixResourceBase class is refactored as well to include additional metadata in the primary configuration file builder of the VM before starting it.
VMware: VmwareResource is also refactored to include additional configuration into the VMX configuration file builder before starting the VM.
N/A