Feature reference

CLOUDSTACK-9813 - Getting issue details... STATUS

Config drive is an ISO file that is mounted as a cd-rom on a user VM and contains the user VM related userdata, metadata (incl. ssh-keys) and password files.

This work is loosely based on previous work on config drive for Cloudstack as described in “DHCP/DNS offload and Config drive support for adv shared network“. And extended from shared networks to all networks.

Supporting ConfigDrive

Extra data is added to the VM profile to enable the creation of the config drive:

VMdata - a list of String arrays representing [“directory”, “filename”, “content”] on the ConfigDrive device.

<mountdir>/cloudstack

/metadata:

availability-zone.txt
instance-id.txt
service-offering.txt
cloud-identifier.txt
local-hostname.txt
vm-id.txt
public-keys.txt

/password

vm_password.txt
vm_password_md5checksum (for windows VM’s)

<mountdir>/openstack/version/: (added later on based on Miami conference feedback)
- user_data (=hardlink to <mountdir>/cloudstack/user_data/user_data.txt)
- vendor_data.json
- meta_data.json
- Network_data.json

label, which is configurable in global settings:

name : vm.configdrive.label

default: config-2 (same as in OpenStack)

ConfigDrive network element

A new provider “ConfigDrive, implementing the userdata service, will be made available to create, update and delete the ConfigDrive ISO on the secondary store.

When triggered to create userdata

the vmdata is created and added to the virtual machine profile. As is the ConfigDrive label fetched from the global settings.

The secondary store agent is triggered to create the ISO file.

At successful creation the ISO file is added as a new disk to the virtual machine profile

An update of userdata/password/ssh-keys has a few extra steps as some hypervisors don’t handle the update of the ISO file, while attached to the virtual machine, very well.

Detach ISO

Update

Re-attach

For update of userdata it is important the user VM does not keep the ConfigDrive cdrom mounted all the time as update will fail if the ConfigDrive cdrom is in use by the user VM.

The ConfigDrive ISO file is removed from the secondary store up on release of the default nic and, if still necessary, at VM expunge time.

ISO creation

The ConfigDrive network element in the management server triggers the Secondary Storage VM to build the ISO.

The metadata, userdata en password information associated with the VM are used to create the above directory structure in a the base directory /tmp/<VM-NAME>/configdrive/. Then the genisoimage utility is used to create the ISO file. The temporary data is removed as soon as the ISO is created. The ISO is stored on the secondary storage at

ConfigDrive/<vmName>/configdrive.iso

Creating the ISO file implies that the SystemVM needs the genisoimage package installed and the cloudstack software updated.

ConfigDrive availability

At VM start the config drive iso is attached on 2nd cd/dvd drive of the user instance, such that any other iso image (e.g. boot image or vmware tools) get mounted on 1st cd/dvd drive. This means existing functionality of supporting 1 cd/dvd drive is still available.

At Password reset or update userdata, Secondary Storage VM will rebuild the ConfigDrive iso. That is the existing ISO is mounted on a temporary directory, password, userdata or ssh-keys are updated and a new ISO is build from the updated directory structure.

In case of password reset the new password will be picked-up at VM start

To access updated userdata the user needs to remount the config drive cdrom

As a VM is stopped the ConfigDrive network element will trigger the Secondary Storage VM to remove the ISO from the secondary storage

Since the ISO is available on the second storage there is no need for an extra implementation in case of migration

Enable config drive

To use the config drive the network offering must have the “ConfigDrive” provider selected for userdata service.

If the networkoffering uses ConfigDrive for userdata and the template is password enabled, the password string for the VM is placed in password.txt file and it is included in the ISO.

To enable the ConfigDrive functionality on VPC tiers a few extra changes are required:

Update the UI to allow the selection of the ConfigDrive provider when creating a VPC network offering for Vpc tiers.

Add ConfigDrive as a supported provider for VPC.

Accessing data in the ConfigDrive

Accessing ConfigDrive depends on the guest VM OS. The guest VM template needs to be adapted to retrieve the password from the config drive i.s.o. the Virtual router. It is also possible to enable both options.

Windows VM
For windows VM the ISO is mounted automatically. So user can browse through the files in the ConfigDrive cd.

Linux VM

Config drive iso is created by-label under /dev/disk/by-label/config.
It can be found using
blkid -t LABEL='config‘ /dev/xvdb: LABEL="config" TYPE="iso9660"
or
blkid -t LABEL=”config” /dev/disk/by-id/*

Hypervisor support

This feature will be implemented and tested for KVM and ESXI.

OpenStack support

Based on the feedback that was received from the community at the CloudStack conference in Miami the config-drive implementation was extended to support the OpenStack config-drive implementation. This allows for easy integration with cloud-init and similar systems.

OpenStack (cloud-init) compatibility (version 2)

The following criteria are required for config drive in OpenStack:

Must be formatted with vfat (deprecated) or iso9660 filesystem
have a filesystem label of config-2
The files that will typically be present in the config drive are:

<mountdir>/openstack/version/: (only support version = latest, same approach as coreOS)

user_data
vendor_data.json
meta_data.json
network_data.json

This map structure will be created beside the original CloudStack map structure. The config drive will contain the following folders and files:

<mountdir>

/cloudstack

/metadata:

availability-zone.txt
instance-id.txt
service-offering.txt
cloud-identifier.txt
local-hostname.txt
vm-id.txt
public-keys

/password

vm_password.txt
vm_password_md5checksum (for windows VM’s)

/user_data

user_data.txt (note about _ characters below in text)

/openstack

/version

User_data (=hardlink to <mountdir>/cloudstack/user_data/user_data.txt)
vendor_data.json
meta_data.json
network_data.json

Content of user_data:

We make a hard link to the user_data file in the cloudstack directory (soft link does not work in iso).

Content of meta_data.json (json encoded):

meta
files
network_config
public_keys: [{‘’data’’: “ssh-key value”, type: “ssh”, name: “name of key”}]
keys: [{‘’data’’: “ssh-key value”, type: “ssh”, name: “name of key”}]
hostname
name
launch_index
availability_zone
random_seed
project_id
devices
uuid

CloudStack config-drive supports the following:

availability-zone
hostname
name
uuid
keys
public-keys

I.e. we leave out the content we don’t support (list based on: https://github.com/openstack/nova/blob/39c6e442c628a881b4fce4d8b74e6b7d174f72f2/nova/api/metadata/base.py )

Content of vendor_data.json (json encoded):

Vendor-Data is a way for a vendor to provide site specific information. This can be information on local mirrors, a local proxy, or a one-time registration code. It is found in the vendor_data.json file residing on ConfigDrive or the Neutron Metadata Service via 169.254.169.254/openstack/latest/vendor_data.json. By default, Cloud-Init executes the Vendor-Data on every boot. Currently, we don’t plan to support this.

Is allowed to be empty {}

Content of network_data.json (json encoded):

Network-Data is network information found on ConfigDrive. It resides in the network_data.json file. Nova retrieves this info from Neutron. It includes fixed ip addresses, MAC addresses, port-id's, network-id's, subnet-id's, DNS name-servers, etc. Currently, we don’t plan to support this.
Is allowed to be empty {}

Changes needed to CloudStack

The current config-drive PR needs to be changed. We plan to create an extra OpenStack directory beside the original proposed CloudStack directory. We also need to change the label of the disk from config to config-2. The way the iso image is created needs to be adapted to make it align with the way OpenStack creates the iso img (provide extra params https://github.com/openstack/nova/blob/master/nova/virt/configdrive.py). A side effect is that underscores will be written to the iso image as underscores and not replaced with dashes (current behaviour). Because of this, we have to update the coreOS PR and the CloudStack documentation.

The OpenStack file structure contains a version folder. This version is equal to the release date of a specific OpenStack version

FOLSOM = 2012-08-10
GRIZZLY = 2013-04-04
HAVANA = 2013-10-07
LIBERTY = 2015-10-15
NEWTON_ONE = 2016-06-30
NEWTON-TWO = 2016-10-06
OCATA = 2017-02-22

So for example an OpenStack config-drive will have the following file structure:

/openstack

/2012-08-10

...

/2013-04-04

...

/2013-10-07

...

/2015-10-15

...

/2016-06-30

...

/2016-10-06

...

/2017-02-22

…

/latest (which is equal to 2017-02-22)

…

Our proposal is to only create the latest folder on the CloudStack config-drive. The reason behind this is that the software only reads the latest folder from the OpenStack config-drive. The drawback is that when OpenStack config drive breaks backward compatibility we will have to update our CloudStack implementation.

Cloud init support:

In order for Cloud init to work, OpenStack config-drive provider must be called before the CloudStack (metadata-service) provider. This can be defined in the cloud.cfg file (for example see arch-linux wiki: https://wiki.archlinux.org/index.php/Cloud-init#Configuration (Configuring data sources))

CoreOS support

CoreOS keeps the CloudStack Config-Drive implementation (Nuage PRs).
Reason: CoreOS uses the ec2 format for Config-drive in combination with OpenStack. We propose to support only the OpenStack (beside the CloudStack) format and not ec2.

PR's:

Space shortcuts

Child pages

Feature reference

CLOUDSTACK-9813 - Getting issue details... STATUS

Supporting ConfigDrive

ConfigDrive network element

ISO creation

ConfigDrive availability

Enable config drive

Accessing data in the ConfigDrive

Hypervisor support

OpenStack support

OpenStack (cloud-init) compatibility (version 2)

Content of user_data:

Content of meta_data.json (json encoded):

Content of vendor_data.json (json encoded):

Content of network_data.json (json encoded):

Changes needed to CloudStack

Cloud init support:

CoreOS support

Space shortcuts

Child pages

Using ConfigDrive for Metadata, Userdata and Password

Feature reference

CLOUDSTACK-9813 - Getting issue details... STATUS

Supporting ConfigDrive

ConfigDrive network element

ISO creation

ConfigDrive availability

Enable config drive

Accessing data in the ConfigDrive

Hypervisor support

OpenStack support

OpenStack (cloud-init) compatibility (version 2)

Content of user_data:

Content of meta_data.json (json encoded):

Content of vendor_data.json (json encoded):

Content of network_data.json (json encoded):

Changes needed to CloudStack

Cloud init support:

CoreOS support