Overlay Networks & VXLAN: Virtualizing Multi-Tenant Layer 2 Topologies Over Layer 3 Fabrics

In enterprise and cloud-scale datacenters, physical switches and routers form a fixed, high-bandwidth underlying physical network known as the **Underlay** (typically structured as an IP-routed Clos / Spine-and-Leaf topology running BGP or OSPF).

However, cloud workloads require extreme agility: thousands of isolated tenant networks, arbitrary IP address overlap (e.g., two distinct enterprise customers using identical `10.0.0.0/16` CIDR blocks), live virtual machine or container migrations without IP changes, and micro-segmentation. Traditional Layer 2 IEEE 802.1Q VLAN tags support only 4,096 distinct network IDs and force spanning tree protocol (STP) topologies that disable redundant links.

To overcome these physical limitations, cloud providers utilize **Overlay Networks** powered by encapsulation protocols such as **VXLAN (Virtual Extensible LAN)** and **Geneve**. An overlay creates virtualized logical Layer 2/3 networks decoupled from the underlying physical switches.

VXLAN Packet Anatomy: L2-over-UDP/IP Encapsulation

Standardized under RFC 7348, VXLAN encapsulates a complete inner Ethernet frame generated by a virtual machine or container inside an outer UDP/IP packet routed across standard physical networks:

Encapsulated Frame Breakdown

• Inner Payload: The raw tenant Ethernet frame, containing the tenant VM's source/destination MAC addresses, private tenant IP header, and data payload.
• VXLAN Header (8 bytes): Contains operational flags and the 24-bit **VXLAN Network Identifier (VNI)**. A 24-bit space expands the multi-tenancy ceiling from 4,096 VLANs to over **16.7 million (2^24)** isolated tenant overlay domains.
• Outer UDP Header (8 bytes): Destination port is set to the standard IANA VXLAN port (4789). The outer source UDP port is dynamically set to a hash of the inner frame's headers (providing automatic Equal-Cost Multi-Path / ECMP load-balancing across physical spine switches).
• Outer IP & Ethernet Headers: Physical IP and MAC addresses of the source and destination tunnel endpoints.

VXLAN Tunnel Endpoints (VTEP)

Encapsulation and decapsulation occur at special boundary nodes called **VXLAN Tunnel Endpoints (VTEPs)**:

1. Ingress VTEP: When a VM transmits an Ethernet frame, the local hypervisor or Top-of-Rack (ToR) switch acting as the Ingress VTEP intercepts the frame.
2. Mapping Lookup: The VTEP maps the tenant's VLAN/interface to a VNI, looks up the destination tenant MAC address in its forwarding database, and identifies the physical IP of the remote Egress VTEP hosting the target VM.
3. Encapsulation: The VTEP wraps the original frame with VXLAN, UDP, and outer IP headers, then transmits the resulting packet onto the physical underlay network.
4. Egress VTEP Decapsulation: The receiving host's VTEP strips away the outer IP/UDP/VXLAN headers, validates the VNI, and delivers the clean original inner frame to the destination VM or container namespace.

Control Plane Evolution: Data-Plane Learning vs. EVPN-BGP

How does an ingress VTEP know which remote physical IP owns a specific inner tenant MAC address?

1. Multicast Flood-and-Learn (Naive)

Similar to legacy switches, unknown unicast, broadcast (ARP), and multicast (BUM) frames are flooded to an underlay IP multicast group. VTEPs inspect incoming frames and learn MAC-to-VTEP associations dynamically. This approach causes broadcast storms and does not scale well in massive multi-tenant datacenters.

2. EVPN-BGP (Ethernet VPN with Multi-Protocol BGP)

Modern cloud infrastructure completely replaces data-plane flooding with a centralized, declarative routing protocol (RFC 7432). When a container or VM boots, its hypervisor registers its MAC and IP addresses with an EVPN-BGP route reflector. Routing information is propagated across all VTEPs out-of-band via control plane messages, eliminating ARP broadcast flooding across physical links.

Operational Realities: The MTU Penalty and Offloading

Operating an overlay network introduces concrete hardware and latency considerations:

• MTU Overhead (50-byte penalty): Outer Ethernet (14) + Outer IP (20) + Outer UDP (8) + VXLAN (8) = 50 additional bytes. Standard 1500-byte guest packets cause IP fragmentation unless physical switches support **Jumbo Frames** (MTU 9000 or at least 1550).
• NIC Hardware Offloading: CPU software encapsulation creates high overhead. Modern SmartNICs and physical network cards implement VXLAN Task Offload (performing hardware-accelerated checksum verification and TSO/LRO segmentation directly on encapsulated packets).

C++ Conceptual Simulation Blueprint (VTEP Encapsulator & Routing Table)

#include <iostream>
#include <vector>
#include <string>
#include <unordered_map>
#include <cstdint>

struct TenantEthernetFrame {
    std::string srcMac;
    std::string destMac;
    std::string payload;
};

struct VxlanOuterPacket {
    std::string outerSrcIp;
    std::string outerDestIp;
    uint16_t outerDestPort; // Default 4789
    uint32_t vni;           // 24-bit Virtual Network Identifier
    TenantEthernetFrame innerFrame;
};

class VTEPEndpoint {
private:
    std::string localPhysicalIp;
    uint32_t assignedVni;
    
    // Forwarding Information Base (FIB): Target Tenant MAC -> Remote VTEP Physical IP
    std::unordered_map<std::string, std::string> macToVtepMap;

public:
    VTEPEndpoint(std::string physicalIp, uint32_t vni) 
        : localPhysicalIp(physicalIp), assignedVni(vni) {}

    void registerPeerMac(const std::string& mac, const std::string& vtepPhysicalIp) {
        macToVtepMap[mac] = vtepPhysicalIp;
    }

    // Ingress Encapsulation Phase
    bool encapsulate(const TenantEthernetFrame& frame, VxlanOuterPacket& packetOut) {
        auto it = macToVtepMap.find(frame.destMac);
        if (it == macToVtepMap.end()) {
            // In production, would drop or route to EVPN / multicast BUM handler
            return false;
        }

        packetOut.outerSrcIp = localPhysicalIp;
        packetOut.outerDestIp = it->second;
        packetOut.outerDestPort = 4789;
        packetOut.vni = assignedVni;
        packetOut.innerFrame = frame;
        return true;
    }

    // Egress Decapsulation Phase
    bool decapsulate(const VxlanOuterPacket& packetIn, TenantEthernetFrame& frameOut) {
        if (packetIn.vni != assignedVni) {
            return false; // VNI Isolation Mismatch - Drop frame
        }
        frameOut = packetIn.innerFrame;
        return true;
    }
};

Real-World Cloud & Container Networking Implementations

1. AWS Virtual Private Cloud (VPC): Implements Geneve and proprietary encapsulation handled by custom hardware ASIC chips on AWS Nitro cards to achieve wire-speed network virtualization.
2. Google Cloud Andromeda: High-performance software-defined network virtualization substrate routing packet processing through kernel bypass mechanisms and programmable NICs.
3. Kubernetes Container Network Interfaces (CNIs): Plugins like Calico (VXLAN mode), Flannel, and Cilium use VXLAN to establish seamless pod-to-pod flat overlay networks spanning across heterogenous cloud and on-prem VMs.
4. Open Virtual Network (OVN) & Open vSwitch (OVS): Open-source enterprise SDN stack using VXLAN and Geneve to build virtual distributed switches for OpenStack private clouds.