Static Routing IPv4 (Routed mode)

Project Introduction

Currently CloudStack VR of isolated network and VPC works in NAT mode. The guest VMs are publicly accessible by

• Static NAT

• Load balacing

• Port Forwarding

The guest VMs are guarded by

• Firewall rules (isolated networks)
• NetworkACL (VPC)

This project adds a new network mode (ROUTED) to CloudStack Isolated networks and VPC .

Community PR (target to 4.20): https://github.com/apache/cloudstack/pull/9346

Functional Description

In ROUTED mode, CloudStack VR will act as the gateway of the guest networks. The IP of guest VMs will be publicly reachable from the outside of the network.

the following services are supported

• Dns
• Dhcp
• Userdata
• Firewall (isolated network)
• NetworkACL (vpc)

High-level Design

• 1. Add a new property "network_mode" to network/vpc offering

only the network with network_mode=ROUTED can run in routed mode

SourceNat/StaticNat/Lb/PortForwarding/Vpn service are not supported in Routed mode

rename nsx_mode to network_mode.

A service "Gateway" is added to the network/vpc offering if network_mode=ROUTED

• 2. Add database tables

1. dc_ip4_guest_subnets:  the subnets for IPv4 static/dynamic routing
2. ip4_guest_subnet_network_map: the (created or allocated) smaller subnets for networks with ROUTED mode.
   
   
   

• 3. Add APIs to manage IPv4 subnets of zone

support APIs for zone IPv4 subnet: add/list/update/remove/dedicate/releaseDedicated

• 4. Allocate Ipv4 subnet for guest networks

new API:

1. CreateIpv4SubnetForGuestNetwork 
2. ListIpv4SubnetsForGuestNetworkCmd
3. DeleteIpv4SubnetForGuestNetworkCmd

each subnet 

1. network cidr
2. gateway
3. the first IP (reserved) is network address
4. the last IP (reserved) is broadcast address

new global setting: 

1. max network size
2. min network size

CreateIpv4SubnetForGuestNetwork 

1. if networkCidr is specified, check if it is available
   1. only available for root admins ?
2. if cidrsize is passed, auto-allocate a subnet by network size (/ notation)
   1. Variable Length Subnet Mask (VLSM)
   2. a global setting to enable/disable auto-selection for end users ?

• 6. create network from offerings with ROUTED mode

A Public IP needs to be allocated as the Gateway (similar source nat)

• 7. create VPC and vpc tier with ROUTED mode

A Public IP needs to be allocated as the Gateway (similar source nat)

the network offering for vpc tier must be ROUTED mode as well.

• 8. VR creation and settings

allocate a public IP for VR: if network has service "Gateway" or "SourceNat"

isolated network: eth0 (guest), eth1 (linklocal), eth2 (public)

vpc: eth0 (linklocal), eth1 (public), eth2/eth3... (guest), 

• 9. firewall rules and network ACL in VRs

add nftables rules in the VR

remove all iptables rules

integration test

1. setupClass

create vpc offering

create network offering for isolated network

create network offering for vpc tier

2. test_01_dc_ipv4_subnets

add ipv4 subnet for zone

list, update and dedicated/release  ipv4 networks for zone

3. test_02_routing_isolated_network

create isolated network with routed mode

add firewall rules

validate the firewall rules in the VR (via linklocal IP)

remove firewall rules

4. test_03_routing_vpc_tier

create vpc and vpc tier with routed mode

create network acl list

replace vpc tier with network acl

add network acl to VPC tier

validate the network acl in the VPC VR

remove network acls

5. test_04_test_connectivity

add static routes in the VRs (manually)

test the connectivity between the VMs and VRs (VR to VM in another network)

Considerations

1. DualStack (Ipv4 and Ipv6)

network can be created

supports routing firewall rules and ipv6 firewalls 

th network ACL should support both ipv4/ipv6 is cidr is null

2. Redundant VRs

Is it necessary ?

3. Auto-generate subnet based on cidrsize

next stage ?

4. dedicated vlan/public IPs for routing network (optional)

the guest IPs are actually accessible from the outside

the public IP of VRs do not need to be Public

they can use private IPs

5. Dynamic routing

next stage ?

6. add static routes in VRs (optional)

new APIs: support customized static routes ?

Non-Functional Requirements

to be added.

User Interface

to be added.

Milestones

Glossary

NAT - Network address translation

Database Changes

• CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.network_offerings','network_mode', 'varchar(32) COMMENT "mode in which the network would route traffic"');

• CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.vpc_offerings','network_mode', 'varchar(32) COMMENT "mode in which the network would route traffic"');

References

• NSX integration #7919
• https://docs.cloudstack.apache.org/en/latest/plugins/ipv6.html