You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 76 Next »

Glossary

GPU - Graphics Processing Unit

pGPU - Physical Graphics Processing Unit.

vGPU - Virtual Graphics Processing Unit

card - GRID cards, which supports vGPU functionality.

GPU group - a group has one of more cards of same type.

Introduction

GPU (graphics processing unit) is used for accelerating the image output in a frame buffer intended for output to a display. GPU-accelerated computing offers unprecedented application performance by offloading compute-intensive portions of the application to the GPU, while the remainder of the code still runs on the CPU. From a user's perspective, applications simply run significantly faster [1]. GPU can also be used as General Purpose Computing as GPGPU(General-purpose computing on graphics processing units) which has significant importance in the field of machine learning, image processing, linear algebra, statistics and Cloud Computing.

Now a days, physical servers are coming up with graphic cards that have multiple GPUs, VMs running on cloud can leverage the high computation power of GPU to meet high graphics processing requirements like Auto-CAD, Photoshop etc. Also, there are cards in market which supports sharing of a GPU cards among multiple VMs by creating vGPUs for each VM, e.g. NVIDIA has introduced vGPU capable cards GRID K1 and K2 which allow multiple vGPUs on a single physical GPU.

With vGPU technology, the graphics commands of each virtual machine are passed directly to the underlying dedicated GPU, without translation by the hypervisor. This allows the GPU hardware to be time-sliced and shared across multiple VMs [2].

Xenserver has added the support for NVIDIA GRID K1 and GRID K2 cards [3]. It allows the VMs on XenServer hosts to use the GPU cards in following ways:

  • GPU-passthrough: It allows the hypervisor to assign the entire PGPU to a VM, this is useful for a power users.
  • VGPU: It allows the VM to share a PGPU device with other VMs, this is useful for tier 2 users.

Purpose

This is the functional specification of feature "GPU and vGPU support for CloudStack Guest VMs" which has jira ids CS-4760 and CS 4762.

Bug Reference

References

  1. http://www.nvidia.com/object/what-is-gpu-computing.html#sthash.L9RfBfxB.dpuf
  2. http://www.nvidia.com/object/virtual-gpus.html
  3. http://us.download.nvidia.com/Windows/Quadro_Certified/GRID/312.56/312.56-nvidia-quadro-control-panel-quick-start-guide.pdf
  4. http://on-demand.gputechconf.com/gtc/2013/webinar/gtc-express-vgpu.pdf

Feature Specification

Currently, CloudStack doesn't understand and leverage the GPU cards installed in a host. This feature will help users/admin to assign a physical GPU(GPU-passthrough) or a portion of a physical GPU card(vGPU) to a guest VM at the time of VM deployment or at any later stage by changing the service offering. It will help run graphical applications on VMs running in CloudStack.

  • CloudStack does following:
    • Add hosts with GPU/vGPU capability provisioned by admin.
    • Query hypervisor to obtain information about the cards, supported vGPU types (in case of grid cards), capacity of the cards.
    • Create compute offering with GPU/vGPU capability.
    • Deploy VM with GPU/vGPU support.
    • Destroy VM with GPU/vGPU support and releases the resources accordingly.
    • Allow user to add GPU/vGPU support to a VM (without GPU/vGPU support) by changing service offering and vice-versa.
    • Migrate VMs (cold migration) with GPU/vGPU support.
    • Manages GPU cards capacity.
  • Gracefully failure and recovery scenario.
  • CloudStack generates alerts whenever a vm deployment with GPU/vGPU service offering is requested but there are not suitable GPU/vGPU capacity available.
  • CloudStack generates suitable usage events when a vm is deployed, destroyed or migrated using GPU/vGPU.
  • Global configuration:
    • None
  • In Xenserver, there are two algorithm to create vGPUs in the given set of PGPUs:
      • DFS
      • BFS
    • Default allocation algorithm is DFS.
Negative Usage scenario
  1. VM deployment with GPU support if the hosts with enough GPU capacity is not available.
  2. Creation of service offering with GPU key/values which are not supported.
  3. Live migration of GPU enabled VM (not supported).

General Workflow

  1. Add a host which contains GPU card:
    1. Admin adds a host with existing addHost API command and cloudstack checks if the host is GPU enabled or not.
    2. Cloudstack queries the hypervisor and detect if the given host is GPU enabled.
    3. If the host has GPU card in it, then CloudStack queries the hypervisor and as a response it gets the GPU group information, which get stores in op_host_gpu_groups table.
    4. If the cards has only GPU-passthrough support, then CloudStack stores the vGPU type as passthrough in vgpu_types table.
    5. If the cards also has vGPU support, then CloudStack checks for the enabled vGPU types in the hypervisor and stores them in vgpu_types table.
    6. CloudStack also manages these cards by maintaining the capacity of each cards.
  2. Create a compute offering with GPU/vGPU support:
    1. Create a compute offering with following two options:
      1. GPU-passthrough: To assign entire PGPU.
      2. vGPU: To share a PGPU device. Admin also needs to pass the vGPU type. Steps are explained in Web Services APIs section.
  3. Deploy a VM with GPU/vGPU support:
    1. Admin/user can deploy a VM with GPU/vGPU support by selecting appropriate service offering.
    2. Once the deployment of VM with GPU/vGPU support is triggered, the allocator will decide which host to choose for VM deployment based on following criteria:
      1. Host has GPU cards in it.
      2. In case of vGPU, it check if cards has the required vGPU type support.
      3. Cards has enough capacity available.
    3. Once the host is final, Cloudstack will go ahead with VM deployment.
    4. In case if allocator doesn't finds any appropriate host, then it will through an InsufficientServerCapacity exception.
    5. Once the VM deployment is successful, CloudStack will make a resource call to hypervisor to get the remaining GPU capacity in the card and update the DB accordingly.
  4. Destroy a VM with GPU/vGPU support:
    1. Admin/user can destroy a VM with GPU/vGPU support by navigating through the existing destroyVM flow.
    2. Hypervisor will detach the GPU card from the VM and make it available to use.
    3. Once the destroy VM is successful, CloudStack will make a resource call to hypervisor to get the remaining GPU capacity in the card and update the DB accordingly.
  5. Migrate(cold migration) a VM:
    1. Search for hosts available for VM migration and satisfies GPU requirement.
    2. If the host is available, stop the VM in the current host and perform the VM migration task, else
    3. If the VM migration is successful, update the remaining GPU capacity for both the hosts accordingly.
DB Schema changes:

New tables

  • op_host_gpu_groups (id, group_name, host_id, update_time) :  This table contains the mapping between host and available pci_devices in that host.

    Field

    Type

    NULL

    Key

    Default

    Comments

    id

    bigint(20) unsigned

    NO

    PRI

    NULL

    auto_increment

    group_id

    varchar(40)

    NO

     

    NULL

    GPU group name

    host_id

    bigint(20) unsigned

    NO

     

    NULL

     

  • vgpu_types (id, vgpu_type, gpu_group_id, remaining_vm_capacity) :  This table contains the record of available vgpu profiles.

    Field

    Type

    NULL

    Key

    Default

    Comments

    id

    bigint(20) unsigned

    NO

    PRI

    NULL

    auto_increment

    gpu_group_id

    bigint(20) unsigned

    NO

     

    NULL

    "id" in the op_host_gpu_groups table

    vgpu_type

    varchar(40)

    NO

     

    NULL

    Name label of the vgpu type.

    remaining_vm_capacity

    bigint(20) unsigned

     

     

    NULL

    remaining no. of vgpu the given gpu group with given vgpuType can create.

Web Services APIs:
  • createServiceOffering : This API allows creation of service offering. Now, to create a service offering with gpu/vgpu support, admin needs to pass additional below mentioned information with the help of serviceofferingdetails parameter in key-value pair format.

Supported new keys

Corresponding values

pciDevice

  • gpuPassthrough
  • VGPU

vGPUType

  • GRID_K100
  • GRID_K140Q
  • GRID_K200
  • GRID_K240Q
  • GRID_K260Q

         e.g.: To create a service offering with GPU Passthrough support:
                http://localhost:8080/client/api?command=createServiceOffering&serviceofferingdetails[0].key=pciDevice&serviceofferingdetails[0].value=gpuPassthrough&response=json&.........

                To create a service offering with vGPU support:
                http://localhost:8080/client/api?command=createServiceOffering&serviceofferingdetails[1].key=pciDevice&serviceofferingdetails[1].value=VGPU&serviceofferingdetails[2].key=vgpuType&serviceofferingdetails[2].value=GRID%20K260Q&response=json........

  • listHosts : If the host is GPU enabled, then this API will also list the following GPU related information:
    • What are the GPU cards managed by this host.
    • Used Capacity
    • Available capacity
  • listVirtualMachines : If the VM has GPU support, then this API will also include the following information in VM statistics:
    • VGPU type
    • GPU utilzed

Interoperability and compatibility requirements

  • Supports XenServer 6.2 or later with required hotfixes [4].
  • HVM template is required to create a VM with GPU/VGPU support.

  • NVIDIA GRID K1 AND K2 cards supports homogeneous virtual GPUs i.e. at any given time, the vGPUs resident on a single physical GPU must be all of the same type. However, this restriction doesn't extend across physical GPUs on the same card. Each physical GPU on a K1 or K2 may host different types of virtual GPU at the same time. For example, a GRID K2 card has two physical GPUs, and supports three types of virtual GPU; GRID K200, GRID K240Q, AND GRID K260Q [3].

    GRID

    Profile Name

    No. of Physical GPUs

    Video Ram per vGPU

    Max Resolution

    vGPUs per GPU

    No. of vGPUs per Card

    KI

    GRID K100

    4

    256 MB

    1920 x 1200

    8

    32

    K1

    GRID K140Q

    4

    1 GB

    2560 x 1600

    4

    16

    K2

    GRID K200

    2

    256 MB

    1920 x 1200

    8

    16

    K2

    GRID K240Q

    2

    1 GB

    2560 x 1600

    4

    8

    K2

    GRID K260Q

    2

    2 GB

    2560 x 1600

    2

    4

Logging and debugging

  • All the VM activities involving GPU would be logged at different log levels (INFO, ERROR, DEBUG) in management server log file.

Operations not supported

  • Live VM migration.
    Current Limitations
  • Reservation of GPU resources: CloudStack does not hold the GPU resources in cases like stopped VM or to recover a destroyed VM in the same host.
  • XenServer supports only one VGPU creation per VM, so admin/user cannot create more than one VGPU for a VM.

Upgrade Scenario

  • During CloudStack Operation, cloudstack checks for GPU cards in all the hosts that are being managed by CloudStack.
  • If cloudstack finds any GPU enabled host, it persists the entry in op_host_gpu_groups and vgpu_types table.

UI Flow

  • UI will get updated to allow creation of service offering with GPU and vGPU support.
  • In Dashboard, "System Capacity" will have one more pie chart to display Zone-Wide GPU information.
  • Infrastructure -> Hosts -> Details -> Statistics : New parameters will get added to display 1) Total GPU, 2) GPU Allocated 3) GPU available
  • No labels