User:Jhedden/notes/cloudvirt-nic

Cloudvirt dual NIC configuration

eno1

The eno1 network interface is configured with an address on the cloud-hosts1-b-<site> VLAN and subnet. This interface is used for everything except virtual machine traffic.

The configuration for this interface is defined in /etc/network/interfaces.

# The primary network interface
allow-hotplug eno1
iface eno1 inet static
        address 10.192.20.13/24
        gateway 10.192.20.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 10.3.0.1
        dns-search codfw.wmnet
   pre-up /sbin/ip token set ::10:192:20:13 dev eno1
   up ip addr add 2620:0:860:118:10:192:20:13/64 dev eno1

eno2

The eno2 network interface is setup to use the cloud-instances2-b-<site> VLAN, but is not associated with any subnet. This interface is only used by Neutron to bridge network communication between virtual machines and external networks.

The configuration for this interface is defined in /etc/network/interfaces.

auto eno2.2105
iface eno2.2105 inet manual
   up ip link set $IFACE up
   down ip link set $IFACE down

The systemd service neutron-linuxbridge-agent will create a linux bridge attached to the eno2 network interface. You can view the linux bridge with the brctl command.

cloudvirt2002-dev:~$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
brq05a5494a-18          8000.d09466936da1       no      eno2.2105

When a virtual machine is running you'll see the tap interfaces for each virtual machine also attached to the same bridge.

cloudvirt2002-dev:~$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
brq05a5494a-18          8000.d09466936da1       no      eno2.2105
                                                        tap0c8ccf84-bc
                                                        tap7729745f-c0

Cloudvirt single NIC configuration

In a single NIC configuration the physical port on the switch for eno1 is reconfigured from an access port to a trunk port. This allows the eno1 network interface to communicate through the physical switch on different VLANs.

The cloud-hosts1-b-<site> VLAN should be set as the native VLAN and cloud-instances2-b-<site> as a member of the trunk group. Phab task information on switch config

Steps to change from a dual NIC to a single NIC configuration:

1. Evacuate all virtual machines from the cloudvirt
2. Update network interface mappings in hiera example patch
3. Request the switch port to be reconfigured example
4. Verify network connectivity to the cloudvirt (the primary interface on eno1 does not require any changes)
5. Update /etc/network/interfaces swapping eno2.2105 with eno1.2105
6. Run the puppet agent to update Neutron with the latest hiera data
7. Reboot the host

OPTIONAL: If you choose not to reboot the host you can run the following commands instead:

# Stop the neutron linux bridge agent
$ sudo systemctl stop neutron-linuxbridge-agent

# Detach the eno2 interface from the bridge
$ sudo brctl delif br-internal eno2.2105

# Shutdown and delete the eno2.2105 device
$ sudo ip link set dev eno2.2105 down
$ sudo ip link delete eno2.2105

# Restart networking
$ sudo systemctl restart networking

# Start the neutron linux bridge agent
$ sudo systemctl start neutron-linuxbridge-agent

Configuration after reconfiguring the cloudvirt

cloudvirt2001-dev:~$ ip link show eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether d0:94:66:93:97:77 brd ff:ff:ff:ff:ff:ff

cloudvirt2001-dev:~$ ip link show eno1.2105
57: eno1.2105@eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brq05a5494a-18 state UP mode DEFAULT group default qlen 1000
    link/ether d0:94:66:93:97:77 brd ff:ff:ff:ff:ff:ff

cloudvirt2001-dev:~$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
brq05a5494a-18          8000.d09466939777       no              eno1.2105