Versions Compared

Old Version 10

changes.mady.by.user Suresh Ramamurthy

Saved on

compared with

New Version Current

changes.mady.by.user Suresh Ramamurthy

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Bug Reference

TBD

Branch

TBD

Introduction

IPv6 is supported in CloudStack only for a Shared network. Now, the IPv6 support will be extended to VPC networks and other services.

Purpose

Functional specification for IPv6 support in VPC Networks and CloudStack services like DHCP, DNS, Network ACL, HAProxy, Password Reset.

References

http://docs.cloudstack.apache.org/en/latest/networking/ipv6.html

Scope

  • IPv6 will be supported initially for VPC networks
  • IPv6 will be supported initially for following CloudStack services 
    • DHCP
    • DNS
    • Network ACL (IPtables)
    • Load Balancing (HAProxy)
    • Password Reset

Note: Design will be generic enough so that it will be extensible for Isolated network and other services supported in VPC

Feature Specifications

IPv6 Network Allocation for VPC and its Tiers

When an advanced zone is provisioned, the user is supposed to provide the Super IPv6 CIDR if he wants to use IPv6 in his VPC networks. This IPv6 CIDR will be carved into network sub levels and create the routed tiers. The first sub level network will be for the VPC Public interface and then for each VPC further networks will be carved for the network tiers and the VPC Router. The first network in the zone will be used for all the VPC Routers in the VPC, the rest of the network will used for the VPC tiers. The diagram below shows how the super IPv6 CIDR will be carved to multiple networks.

 

 

Example: if the user enters 2001:67c:2834:0:0:0:0:0/50 as the Super IPv6 CIDR, then 

  • 2001:67c:2834:0:0:0:0:0/50 will be split into 1024 networks using /60 which would be 2001:67c:2834::/60 - 2001:67c:2834:3ff0::/60. This means a total of 1024 VPCs can be created in one single Zone.
  • Then each 2001:67c:2834::/60 will be further split into 16 network using /64 which would be 2001:67c:2834::/64 - 2001:67c:2834:3fff::/64. This means the first /64 network is for the VPC Router and the next 15 for the VPC tiers.

Initial implementation will split the Super IPv6 CIDR to /60 networks and each /60 network will be split into to /64 networks. If the Super Ipv6 CIDR is not sufficiently big enough to split into network sub levels then appropriate errors will be given to the user.   

Mock screen shots to provision Super IPv6 CIDR for VPC is shown as below:

  1. Zone : When a Zone is created, the user shall be able to configure the Super IPv6 CIDR and Private Autonomous System Number(ASN). Initial implementation will use this CIDR only for VPCs in the Zone. At a later point one extra field could be added for Isolated networks (in which case each isolated network will get a /64 from the CIDR configured for Isolated network). ASN will be used for the BGP session as described below in section BGP on VPC Router.

2.  VPC : When a VPC is created, the user shall configure an IPv6 CIDR. It should be within the super IPv6 CIDR configured for the Zone. In this case it would be a /60 prefix. The initial implementation would be to split the super IPv6 CIDR into /60 for the VPCs. If the user does not configure any value, CloudStack automatically picks an unused /60 CIDR and assigns it to the VPC. So, it is not a mandatory attribute. An appropriate error will be given to user, if he tries to configure IPv6 CIDR with out configuring the super IPv6 for the Zone.

3. VPC Tier : When a VPC tier is created, the user shall configure an IPv6 CIDR and the IPv6 gateway. The tier IPv6 CIDR should be within the super IPv6 CIDR configured for its VPC. In this case, it would be a /64 prefix. If the user does not configure any value, CloudStack automatically picks an unused /64 CIDR and assigns it to the VPC. So, it is not a mandatory attribute. An appropriate error will be given to user, if he tries to configure IPv6 CIDR with out configuring the super IPv6 for the Zone.

 

IPv6 in VPC Router

In 4.6, a VPC Router does not support IPv6. The following needs to be enabled in a VPC Router

  1. IPv6 needs to be enabled so that VPC Router Public and Guest Network NICs can be assigned with an IPv6 address. The Public NIC will get an IPv6 address from the address range configured for the public network(using createVlanIpRange API)
  2. Route Advertisement (RA) needs to enabled in the dnsmasq running in the VPC Router. The enable-ra line has to be added in dnsmasq.conf file. This will set M and O flag in the RA advertisement, telling Instances to ONLY use that router as their default gateway and not to use SLAAC to auto-configure their IP-Address. CloudStack allocates and controls the IP address allocation to the guest VMs, so we need to disable SLAAC in order to maintain this behavior. 
  3. Also, the VPC Router needs to be programmed to set the IPv6 network prefix that it needs to advertise
    Example:

                     dhcp-range=2001:067c:2834:0010::1,static

We need to create couple of new commands that can enable IPv6, RA and set dhcp-range and only if IPv6 is enabled in VPC, then we can to execute these commands accordingly.


IPv6, DHCPv6 client in Guest VMs

Guest VM needs to support IPv6. So, the user is supposed to use VM templates with IPv6 and DHCPv6 enabled in them.

 

DNSMASQ for DHCPv6/DNS

Initial support for IPv6 will not support Stateless Auto Configuration of the IP address. So, we will support DHCPv6 (IPv6 address, gateway address, DNS domain, NTP server).

dnsmasq will be tweaked to support RA and DHCPv6. Example configuration to achieve this is described in example configuration in section IPv6 in VPC Router. The work flow to push IPv6 IP of the VM to the VPC Router's dhcphost.txt file will remain the same. But, we need to use DUID of the VM rather than the MAC address of the VM. This is because, DHCPv6 client will use DUID to request the IP from the DHCP server (http://tools.ietf.org/html/rfc3315).    

DUID-LL will be used so that we can predict the DUID of the client VM. Also, we will mandate that the client VM needs to support DUID-LL as mentioned the functional spec for IPv6 support in Shared Networks (https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+support) 

Current Type 3 DUID Cloudstack implementation:

github: apache/cloudstack/blob/master/server/src/com/cloud/network/router/CommandSetupHelper.java#L241 (github.com)
dhcpCommand.setDuid(NetUtils.getDuidLL(nic.getMacAddress()));

Only IPv6 addresses that are handed out will be stored in the Database.


Network ACL

As the IPv6 address are in public space, the VPC Router will take care of routing the traffic between and across the tier as a normal router. But, by default, ACLs will be added to block the traffic from and to (egress/ingress) network tiers or VPC Router interfaces (except for ICMPv6 which will be allowed by default). This default ACL is added to maintain current VPC network traffic functionality. 

The VPC Network ACL will be similar to the current implementation of IPv4 based Network ACLs. The VPC Router will use IP6Tables to implement the ACLs. 

Note: CentOs has a bug(https://bugzilla.redhat.com/show_bug.cgi?id=591630) where DHCPv6 response are blocked in the firewall. So, we might end up adding a firewall rule to allow the DHCPv6 responses like one mentioned below

-A INPUT -m state --state NEW -m udp -p udp --dport 546 --sport 547 -s
fe80::/10 -d fe80::/10 -j ACCEPT


Loadbalancing(HA Proxy)

HA proxy which is used for load balancing will be enhanced to support IPv6, which is a default feature.


Password Reset

The CloudStack Management server uses Control IP of the VPC Router VM to save the guest VMs password. This will remain the same because Control, Management and Storage traffic will still be IPv4. So, there is no change needed here.

For password management, the guest VM uses the VPC Router gateway IP to make a wget request. So, we can either

            i) Modify the password script to use the IPv6 address of the VPC Router or
           ii) Use cloud-init which should take of using the IPv6 address of VPC Router.

In any case, whenever the wget is based on IPv4 or IPv6 (depending on the domain lookup) this is transparent to the VPC Router as long as the Apache process is listening on both addresses.

Note: According to latest document, password management is not supported by cloud-init. But, when it is available it should make sure that it supports IPv6


BGP on VPC Router

BGP will be enabled in the VPC Router for the IPv6 announcement.

There are several OpenSource router projects, for example Quagga and Bird, for now Quagga will be used.

The BGP routing process is installed on every VPC Router and will advertise the VPC network to the upstream gateway, thus all VPC routes will be advertised across the VPC Routers. This will eliminate neighbor IPv6 VPC network traffic to go through the Cloud Uplink Router but directly between the VPC Routers in the same Supernet /50.

All VPC Routers have the BGP session to the first available IPV6 address in the network (Cloud uplink routers) and accepts all networks in the Supernet /50 and are a member of a Private AS (this needs to be configurable in the CS UI to avoid duplicates).




IPv6 to IPv4

HAProxy is able to do the IPv6 to IPv4 proxying (another solution is Socat or iptables module tproxy). With simple configuration lines it is possible to proxy connections between the two protocols.

A basic Layer 4 TCP config:

Code Block
global
    user haproxy
    group haproxy
    daemon
    log /dev/log daemon

defaults
    timeout client 5000
    timeout connect 5000
    timeout server 10000

listen webserver1
    bind 2001:db8:abc:123::cafe:80
    mode tcp
    server webserver1 192.0.2.1:80

 

A basic Layer 7 HTTP config:

Code Block
listen webserver1
    bind 2001:db8:abc:123::cafe:80
    mode http
    option forwardfor
    server webserver1 192.0.2.1:80
 
listen webserver1-ssl-passthrough
    bind 2001:db8:abc:123::cafe:443
    mode tcp
    server webserver1 192.0.2.1:443

 

Another possible implementation is using NAT64 with for example Tayga (most common stateless NAT64 daemon, http://www.litech.org/tayga/). Downside is the extra process needed on the VPC Router.

Configuration is relative simple and straightforward in taiga.conf.

Code Block
tun-device nat64
ipv4-addr 192.168.255.1
prefix fd9f:590a:b158:ffff:ffff:ffff::/96
dynamic-pool 192.168.255.0/24
data-dir /var/spool/tayga


DNS64

Configuring DNS64 can be done with for example Bind by telling that it has to return a special AAAA-record if a client from a specific range requests the IP address of a hostname that has no AAAA record. This AAAA-record is constructed by Bind, using a prefix. When a client tries to connect to an IP starting with the prefix, it will be forwarded (through routing) to the NAT64 setup.

 

 

Code Block
dns64 2001:67c:2834:ff99::/96 {
clients { any; };
mapped { any; };
suffix ::;
recursive-only yes;
};

 

Both NAT64 and DNS64 needs more investigation.

 

Future Enhancement
  • Support for Network ACL to allow ingress/egress traffic for an external network. For this a network administration facility could be introduced in ACS (low pro)