To set-up the QA system from scratch needs a *lot* of time and familiarity with CI tools. Tools like Cobbler have been chosen for this purpose so that tasks like PXE server set-up, and Puppet interaction, is made easier. The steps involved to set-up a system for scratch would be :

  1. Create a QACloud VM on XenServer host
  2. Configure private network switch with VLANs etc.
  3. Install the following on QACloud
    1. Cobbler
    2. Puppet
    3. DNSMASQ
    4. ipmitool
    5. Squid
    6. NFS server
  4. Use this sample for Cobbler settings
  5. Import Hypervisor images (XenServer, KVM, ESXi) into Cobbler
    1. (TODO - Insert instructions to import here. They are tricky)
  6. Import Management Server OS into Cobbler (CentOs etc.)
  7. Create kickstart files for each OS
    1. Sample for centos
    2. Sample for xenserver
    3. Sample for KVM
    4. Sample for ESXi
  8. Cobbler integrates well with Puppet. However, the recipies etc work well with Puppet 2.7. Thus, we add a custom Puppet 2.7 repo to the Cobbler profiles.
    1. cobbler profile edit --name=centos63-ms --repo=<Path to local Puppet repo>
    2. In /var/www/html/puppet.conf , add this file. It is used to bootstrap Puppet on the Cobbler provisioned hosts.
  9. Configure MAC address for baremetal hosts in Cobbler for PXE provisioning
  10. Configure DNSMASQ for the above MAC addresses
    1. Use this sample config
    2. Sample DNS lookup config in /etc/dnsmasq.d/hosts
  1. Set-up Puppet master to enable provisioning services like mysql etc. on management server VM in an automated way.
    1. Use this sample config
    2. Use puppet recipies from here
  2. Set-up NFS server for secondary storage
  3. Seed systemVM templates on NFS
  4. Set-up Squid HTTP Proxy server. This is required since QACloud  is the only one expected to have public internet link. All other hosts live on a completely isolated network which is dedicated for testing. However, tasks like yum package download may need internet access, and thus the proxy server.
  5. Add Cobbler snippet to set up http_proxy server on hosts
    1. Use this as reference
  6. Add QACloud to Jenkins master by providing IP address, login and password.
  7. Set correct networking :
    1. Sample ip route
      1. 10.223.255.192/26 dev eth1  proto kernel  scope link  src 10.223.255.196
        172.16.88.0/24 via 172.16.88.1 dev eth0
        172.16.88.0/24 dev eth0  proto kernel  scope link  src 172.16.88.5
        172.16.0.0/16 via 172.16.88.1 dev eth0
        169.254.0.0/16 dev eth0  scope link  metric 1002
        169.254.0.0/16 dev eth1  scope link  metric 1003
        10.0.0.0/8 via 10.223.255.193 dev eth1
        default via 10.223.255.193 dev eth1
    2. Sample eth 0
      1. DEVICE=eth0
        HWADDR=62:D5:19:9C:CD:7E
        ONBOOT=yes
        TYPE=Ethernet
        NM_CONTROLLED=yes
        USERCTL=no
        BOOTPROTO=none
        IPADDR=172.16.88.5
        NETMASK=255.255.255.0
        GATEWAY=172.16.88.1
        DNS1=8.8.8.8
    3. Sample eth1
      1. DEVICE=eth1
        HWADDR=FA:14:7D:9D:96:CB
        TYPE=Ethernet
        UUID=574ed75c-ff2d-4a38-a2c9-11dbe0479eca
        ONBOOT=yes
        NM_CONTROLLED=no
        BOOTPROTO=none
        IPADDR=10.223.255.196
        NETMASK=255.255.255.192
        GATEWAY=10.223.255.193
        DNS1=10.223.240.232
    4. Sample /etc/hosts (required by puppet)
      1. 172.16.88.5    cobblerbvt cobblerbvt.automation.sc.com
        127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
        ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
    5. Set localhost and localdomain correctly so that hostname --fqdn returns correctly from command line (Else Puppet will fail)
  8. (OPTIONALLY) add Windows VM to host vCenter, and manage a ESXi host running NetScalar, SRX etc.
  • No labels