VXLAN and EVPN Explained: Overlay Networks and How to Secure Them

VXLAN is a tunnelling format that wraps an Ethernet frame inside UDP, using destination port 4789 and a 24-bit network identifier, so a data center can run up to 16,777,216 isolated Layer 2 segments over a routed IP fabric instead of 4,094 VLANs. EVPN is the BGP control plane that tells every tunnel endpoint where each MAC and IP address lives. Neither one authenticates or encrypts anything, so you secure them with underlay filtering, link or tunnel encryption, and strict BGP route policy.

This post is the data center sibling of our BGP explainer and our OSPF explainer. Those cover the routing protocols that usually build the underlay; here we stay on the overlay. We walk through the VXLAN header byte by byte, flood and learn versus the EVPN control plane and its route types 2, 3 and 5, symmetric and asymmetric integrated routing and bridging (IRB) with anycast gateways, the attack classes that follow from an unauthenticated UDP encapsulation, the controls that close them, a two part lab with iproute2 and FRRouting, and how Geneve and SRv6 compare. If you are designing the physical fabric for GPU clusters, read scale up versus scale out AI networks first.

Key takeaways

• VXLAN is defined in RFC 7348, published in August 2014 as an Informational RFC. Its 8-byte header carries one meaningful flag (the I flag) and a 24-bit VXLAN Network Identifier (VNI). IANA registered UDP port 4789 for it.
• The 12-bit VLAN ID gives 4,094 usable VLANs. RFC 7348 calls that limit "inadequate" for multi-tenant data centers; 24 bits gives 16,777,216 VNIs.
• A VTEP (VXLAN Tunnel End Point) encapsulates and decapsulates. It can live in a hypervisor, on a Linux host, or in a top of rack switch.
• RFC 7348 learns remote addresses from the data plane and floods unknown traffic over IP multicast. EVPN (RFC 7432, February 2015, and RFC 8365, March 2018, for VXLAN) replaces that with BGP routes: type 2 for MAC and IP addresses, type 3 for flood lists, and type 5 (RFC 9136, October 2021) for IP prefixes.
• RFC 9135 (October 2021) defines symmetric and asymmetric IRB. Symmetric IRB routes through an L3 VNI and keeps ARP state local; asymmetric IRB is simpler but every leaf must carry every subnet and every remote ARP entry.
• RFC 7348 states plainly that it "does not incorporate specific measures" against attacks. Anyone who can reach a VTEP on UDP 4789 can try to inject frames into any VNI that VTEP serves. EVPN's security rests on BGP's security.
• The fixes are layered: an infrastructure ACL that only lets VTEP addresses talk to VTEP addresses on UDP 4789, MACsec on fabric links or IPsec between VTEPs where traffic leaves your control, data plane learning switched off, ARP suppression, duplicate MAC detection, and BGP sessions with authentication, prefix limits and route maps that import only the route types and route targets you expect.
• Geneve (RFC 8926, November 2020, UDP 6081) adds extensible options but has the same "no inherent security" property. SRv6 (RFC 8986, February 2021) drops UDP entirely and relies on filtering at the edge of the segment routing domain.

A short timeline of data center overlays

Note the first two rows: the port registry shows VXLAN's port registered in April 2013 and Geneve's in March 2014, well before either RFC was finished. Early Linux VXLAN code predates both, which is why the Linux kernel still defaults to a different port, as we will see.

What problem overlays solve, and why the VLAN limit pushed data centers to VXLAN

A traditional data center network gave each tenant or application a VLAN, stretched that VLAN across switches with trunks, and relied on the Spanning Tree Protocol to stop loops. RFC 7348 lists three problems with that design once virtualization took off.

The first is the number. An IEEE 802.1Q VLAN tag carries a 12-bit VLAN ID. Twelve bits give 4,096 values, and two of them (0 and 4095) are reserved, which leaves 4,094. RFC 7348 notes that this was fine for many enterprises but that "the 4094 VLAN limit is often inadequate" for a provider hosting many tenants, each wanting several isolated networks.

The second is Spanning Tree. STP blocks redundant links to stop loops, so half the cabling you paid for sits idle. A routed fabric can use every link at once with equal cost multipath (ECMP) routing, but a routed fabric does not carry a stretched Layer 2 segment by itself.

The third is table size. Once virtual machines share physical servers, every switch in a stretched VLAN learns the MAC addresses of every virtual machine rather than of a few physical servers. RFC 7348 warns that when a switch's MAC table overflows, it can stop learning and start flooding.

An overlay fixes all three by separating two networks that used to be one:

• The underlay is a plain routed IP fabric, typically a leaf and spine design running BGP or OSPF between switches. It only needs to move IP packets between tunnel endpoints, and it can use every link with ECMP.
• The overlay is the set of virtual Layer 2 and Layer 3 networks that tenants see. Their frames ride inside IP packets, so the underlay switches never learn tenant MAC addresses.

Worked example 1: identifier space

A provider with 3,000 tenants who each want five isolated segments needs 15,000 segments. That is 3.7 times the VLAN space and under 0.1 percent of the VNI space. The Linux kernel documentation states the motivation directly: VXLAN exists "to solve the problem of limited VLAN IDs (4096) in IEEE 802.1q".

What the VXLAN header looks like, and what a VTEP is

A VTEP is the device that turns a tenant Ethernet frame into a VXLAN packet and back. RFC 7348 originally pictured the VTEP inside the hypervisor, so the virtual machine never sees the VNI, but it also allows VTEPs on physical switches. Today VTEPs commonly live in top of rack switches, in Linux hosts running a routing daemon, and in virtual switches.

Reading from the outside in, a VXLAN packet on the wire looks like this:

Outer Ethernet header   14 bytes  (underlay next hop MACs; 18 with an 802.1Q tag)
Outer IPv4 header       20 bytes  (source = local VTEP IP, destination = remote VTEP or multicast group)
Outer UDP header         8 bytes  (source port = hash of inner headers, destination port = 4789)
VXLAN header             8 bytes
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |R|R|R|R|I|R|R|R|            Reserved (24 bits)                 |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |     VXLAN Network Identifier (VNI, 24 bits)   | Reserved (8)  |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Inner Ethernet frame    tenant destination MAC, source MAC, optional VLAN tag, EtherType, payload

The fields that matter, as RFC 7348 defines them:

• Flags. The I flag "MUST be set to 1 for a valid" VNI. The other seven bits are reserved, set to zero on send and ignored on receipt.
• VNI. A 24-bit value that names the overlay segment. RFC 7348 says machines in different VXLAN overlay networks "cannot communicate with each other". That isolation is a forwarding rule, not a cryptographic one, which is the root of the security story below.
• Reserved fields. 24 bits and 8 bits, zero on send, ignored on receipt. Extensions such as the group policy option in the Linux gbp flag reuse some of these bits.
• UDP destination port. IANA assigned 4789 and the RFC says it "SHOULD be used by default", while keeping it configurable because early implementations used other values.
• UDP source port. RFC 7348 recommends a hash of the inner headers, in the range 49152 to 65535, so that underlay ECMP spreads different tenant flows across different links. That range gives 16,384 possible values.
• UDP checksum. It "SHOULD be transmitted as zero" over IPv4, and a receiver must accept a zero checksum. The checksum is an integrity check against corruption, not against tampering.

The Linux default port trap

The Linux kernel documentation says its VXLAN implementation "pre-dates the IANA's selection of a standard destination port number" and keeps the old value as its default. The kernel source is explicit: "The IANA assigned port is 4789, but the Linux default is 8472". The IANA registry lists 8472 for an unrelated protocol, Overlay Transport Virtualization. If you create a VXLAN device with ip link add ... type vxlan and forget dstport 4789, a Linux VTEP will not talk to a standards based VTEP, and your firewall rules for 4789 will not match its traffic. Always set the port explicitly and write your filters for the port you actually use.

Worked example 2: MTU overhead

Encapsulation adds headers, and RFC 7348 says VTEPs "MUST NOT fragment VXLAN packets" and that a receiving VTEP "MAY silently discard" fragments. The underlay MTU therefore has to fit the largest tenant frame plus the overhead.

With an IPv4 underlay and a tenant IP MTU of 1,500 bytes:

So every underlay link needs an IP MTU of at least 1,550. Add 4 bytes if the inner frame carries a VLAN tag (1,554), and with an IPv6 underlay the outer IP header grows from 20 to 40 bytes (1,570). Seen from the other side, if your underlay is stuck at 1,500, the overlay can carry at most 1,450 byte tenant IP packets over IPv4, which is why a Linux VXLAN device created with dev pointing at a 1,500 byte interface comes up with an MTU of 1,450. Without dev, the kernel cannot derive the MTU from the underlay, so set it yourself. Most fabrics avoid the arithmetic by running jumbo frames of 9,000 bytes or more on the underlay.

Flood and learn versus a BGP EVPN control plane

A VTEP needs to answer one question for every frame: which remote VTEP owns this destination MAC address? There are two families of answer.

Flood and learn (RFC 7348)

RFC 7348 describes data plane learning. Each VNI maps to an IP multicast group, configured through management rather than by the protocol. When a VTEP sees a frame for an unknown MAC, or a broadcast such as an ARP request, it sends it to the VNI's multicast group, and every VTEP that joined the group receives a copy. When the reply comes back, the receiving VTEP learns that the inner source MAC lives behind the outer source IP, exactly as a switch learns which port a MAC address sits on. The underlay must run multicast routing, typically PIM sparse mode, with IGMP to join groups.

It works, but it has three costs. The underlay needs multicast, which many teams would rather not operate. Every ARP and unknown unicast frame floods. And the binding between a MAC and a VTEP is learned from whatever packet arrives first, with nothing to check that the sender was entitled to send it. Hold that last point for the security section.

A variant replaces multicast with a static list of remote VTEPs, often called head end or ingress replication: the sending VTEP makes one unicast copy per peer. On Linux that is a set of all-zero MAC entries in the VXLAN forwarding table, one per remote VTEP.

EVPN as the control plane (RFC 7432 and RFC 8365)

EVPN moves learning into BGP. It was first defined in RFC 7432 for MPLS networks; RFC 8365 adapts it to VXLAN and NVGRE. RFC 8365 lists the main benefits, including that "control-plane learning is used for MAC (and IP) addresses instead of data-plane learning", which removes the need to flood unknown unicast and ARP, and that broadcast and multicast traffic can use either a multicast tree or ingress replication.

EVPN routes travel in multiprotocol BGP with AFI 25 (L2VPN) and SAFI 70 (EVPN). Each route has a one octet route type. RFC 7432 defined four and RFC 9136 added a fifth:

For VXLAN, RFC 8365 reuses the MPLS label fields in these routes to carry the VNI, using "the entire 24-bit field". It also requires the BGP Encapsulation Extended Community on EVPN routes so a receiver knows to use VXLAN rather than MPLS.

Route type 2 in detail

When a host comes up behind leaf 1 in VNI 110, leaf 1 learns the host's MAC locally (from the access port, never from the fabric) and advertises a type 2 route. Other leaves install the MAC in their bridge forwarding table pointing at leaf 1's VTEP address. If the host's IP is also known, typically from ARP, the same route carries the IP, and remote leaves can answer ARP requests for it locally. That local answering is called ARP suppression or neighbour suppression, and it removes most broadcast traffic from the fabric.

Type 2 routes also carry mobility state. When a host moves from leaf 1 to leaf 2, leaf 2 advertises the MAC with a higher sequence number in a MAC Mobility extended community and the old route loses. RFC 7432 adds a safety valve: if a VTEP sees N moves of the same MAC within M seconds, with defaults of N = 5 and M = 180, it concludes there is a duplicate MAC, "MUST alert the operator" and stops advertising that MAC until someone fixes it. That rule matters for security too, because a spoofed MAC that keeps fighting the real one trips it.

Route type 3 in detail

Type 3 routes build the flood list per VNI. Each VTEP advertises one IMET route per VNI it serves, with a PMSI Tunnel attribute saying how to deliver flooded traffic to it (ingress replication to its VTEP address, or a multicast group). A VTEP floods a broadcast in VNI 110 by sending one copy to each VTEP that advertised a type 3 route for VNI 110. RFC 9469 describes this independence from underlay multicast as one of EVPN's main advantages: "there is no need to enable Protocol Independent Multicast (PIM) in the underlay network".

Route type 5 in detail

Type 2 routes describe hosts. Type 5 routes describe prefixes: a default route from a border leaf, a subnet behind a firewall, an aggregate learned from outside the fabric. RFC 9136 says the type 5 route "decouples the IP prefix advertisements from the MAC/IP Advertisement routes". Type 5 is how an EVPN fabric connects tenant IP-VRFs to the rest of the world, and it is also the route type whose misuse can quietly attract a tenant's traffic, so it deserves the strictest import policy.

Route targets decide where routes land

The FRRouting documentation gives the most important rule in one line: IP-VRFs and EVPN instances "are identified by Route Targets, never by labels (VNIs or MPLS labels)". The VNI in a route is data plane information that tells the sender what to put on the wire. The route target is the control plane membership test. A mistake in route targets leaks routes between tenants just as surely as a mistake in VNIs.

Flood and learn versus EVPN at a glance

Symmetric and asymmetric IRB, and anycast gateways

Bridging inside one VNI is only half of a tenant network. Hosts in 10.0.10.0/24 need to reach hosts in 10.0.20.0/24, and RFC 9135 defines integrated routing and bridging for EVPN to do that inside the fabric rather than hairpinning through a central router.

Asymmetric IRB

In asymmetric IRB the ingress leaf does all the work. RFC 9135 says the ingress performs "a MAC lookup, followed by an IP lookup, followed by a MAC lookup again", and the egress performs "just a single MAC lookup". The ingress leaf routes the packet from subnet A to subnet B itself, then bridges it into subnet B's VNI with the destination host's real MAC address. To do that, every leaf must have subnet B configured, and RFC 9135 says each leaf "MUST maintain ARP entries for remote hosts" and bridge tables for all subnets in the tenant's VRF, even subnets with no local hosts.

Asymmetric IRB is easy to understand, needs no extra VNI, and decrements TTL once. Its cost is state.

Symmetric IRB

In symmetric IRB both ends route. The ingress leaf routes the packet into the tenant's IP-VRF and sends it across the fabric in a dedicated L3 VNI for that VRF, with an inner Ethernet header made from the two leaves' own router MAC addresses. The egress leaf routes it again from the VRF into the destination subnet. RFC 9135 says each leaf "only maintains ARP entries for locally connected hosts" and bridge tables "for only locally configured subnets". Remote hosts still appear in each leaf's VRF routing table as host routes learned from type 2 routes, and TTL is decremented twice, once per leaf.

The router MAC is carried in an EVPN Router's MAC extended community. In FRRouting's single VXLAN device setup, the documentation calls this the "routermac" and advertises it "with each Type-2 route advertising an IP address, and each Type-5 route".

Worked example 3: how much state each model needs

Take one tenant with 50 subnets spread over 40 leaves, 2,000 hosts per subnet (100,000 hosts), and assume each leaf has hosts in 5 of the 50 subnets, about 2,500 local hosts.

The numbers are illustrative, but the shape is what RFC 9135 describes. Symmetric IRB moves remote host state out of ARP and MAC tables and into the routing table, which hardware tends to handle at much larger scale, and which you can summarise with type 5 prefixes.

Anycast gateways

With routing on every leaf, every leaf must answer as the default gateway for its local subnets. The anycast gateway model gives every leaf the same gateway IP and the same gateway MAC for a subnet, so a host keeps the same ARP entry for its gateway after it moves between leaves. RFC 9135 describes two options: all leaves share an anycast gateway IP and MAC, or each uses its own MAC and advertises it with a Default Gateway extended community, and it recommends the first. When the anycast MAC is derived automatically, RFC 9135 uses the VRRP style addresses 00-00-5E-00-01-(VRID) for IPv4 and 00-00-5E-00-02-(VRID) for IPv6.

FRRouting's documentation implements the anycast gateway on Linux with a macvlan device carrying the same MAC on every VTEP, plus a local forwarding entry on each VTEP so that packets for the anycast MAC never cross the overlay. It notes the side benefit: "No risk of duplicate address detection (DAD) issues".

Why VXLAN has no built-in authentication or encryption

VXLAN was designed as a fast, hardware friendly encapsulation for networks the operator owns. Authentication would add per packet cryptography to every frame in the data center. So the RFC deliberately leaves security to other layers, and says so: "This document does not incorporate specific measures against such attacks, relying instead on other traditional mechanisms layered on top of IP."

RFC 7348's own security section describes what changes when you move Layer 2 onto IP. On a classic LAN, an attacker had to be physically or logically inside the LAN. With VXLAN, "a MAC-over-IP mechanism for delivering Layer 2 traffic significantly extends this attack surface": an attacker can subscribe to multicast groups that carry a segment's broadcasts, or source "MAC-over-UDP frames into the transport network to inject spurious traffic, possibly to hijack MAC addresses". Geneve's RFC 8926 says the same of its own design: as a UDP encapsulation it "does not have any inherent security mechanisms", and an attacker with underlay access can "snoop on, alter, or inject packets".

EVPN inherits BGP's security model. RFC 7432 warns that a compromised control plane "could result in a PE sending customer data belonging to some EVPN to another EVPN", black holing traffic or sending it to an eavesdropper. RFC 9135 states that the scheme does not provide "confidentiality protection, source authentication, integrity protection, and replay protection" and that these "must be provided by mechanisms that are outside the scope".

Attack classes, described conceptually

These are mechanisms and consequences only. None of them needs a new vulnerability; they follow from the design.

Two details make these more than theoretical. First, every VTEP is a UDP listener that will decapsulate traffic for every VNI it serves, so the reachability of that listener is the whole question; on Linux, check which addresses it is bound to with ss -uln and look for your VXLAN port. Second, the VNI travels in clear text, so anyone who can see one tenant packet on the underlay learns that tenant's VNI.

How underlay ACLs, MACsec or IPsec, and EVPN route filtering close the gaps

RFC 7348 itself sketches the defences: limit who can deploy and manage endpoints, use 802.1X admission control, use "5-tuple-based ACL" functionality because VXLAN is plain UDP, use IPsec to "authenticate and optionally encrypt VXLAN traffic", and carry VXLAN on a dedicated VLAN. RFC 8926 adds that inside a single operator's data center "the most common and highest-performing security mechanism is isolation of trusted components", with filtering at untrusted boundaries and IPsec across untrusted links. NIST SP 800-215 frames the same idea for enterprise networks as microsegmentation and zero trust: trust nothing because of where it sits on the network.

Here is how the layers fit together.

Layer 1: make the underlay unreachable from anywhere it should not be

• Dedicate an address block to VTEPs. Put every VTEP source address (usually a loopback /32 on each leaf) in one prefix, for example 10.255.0.0/24, and never reuse that block for anything else.
• Filter at the fabric edge. On every border, firewall and host facing port, drop inbound packets with a destination in the VTEP block. RFC 8754 describes the same pattern for segment routing as an "infrastructure access list". Nothing outside the fabric should ever be able to send a packet to a VTEP address.
• Filter on the VTEP. Accept UDP on the VXLAN port only when the source address is in the VTEP block. On switches this is a control plane or interface ACL; on Linux it is an nftables rule (shown in the lab).
• Keep the underlay in its own routing domain. Do not leak the VTEP prefix into tenant VRFs, and do not let tenant routes into the underlay.
• Turn on reverse path filtering at the edge so packets that claim a VTEP source address cannot arrive from a non fabric interface.

This single control, "only VTEPs talk to VTEPs on UDP 4789", closes spoofed VTEP injection, VNI hopping from outside, and MAC hijack from outside. It does not protect against a compromised leaf or a compromised host that is itself a VTEP.

Layer 2: encrypt where you cannot trust the path

RFC 7348 notes that IPsec "will, of course, need to be coupled with an authentication infrastructure". Plan key distribution before you plan tunnels. On Linux, MACsec devices come from ip link add ... type macsec and support GCM-AES-128 and GCM-AES-256 ciphers, with extended packet number variants for high speed links, according to the ip-macsec man page.

Layer 3: lock down the VTEP's data plane

• Turn off data plane learning on EVPN fabrics. FRRouting's documentation says "Dynamic MAC/VTEP learning should be disabled on VXLAN interfaces used in EVPN" and that VXLAN interfaces "should not have a remote VTEP defined". On Linux that means nolearning on the VXLAN device and learning off on its bridge port. With learning off, a spoofed packet cannot rewrite where a MAC lives; only BGP can.
• Filter VNIs. With the Linux vnifilter option on an external mode VXLAN device, "the device can only receive packets with vni's configured in the vni filtering table". Configure only the VNIs this leaf should serve.
• Turn on ARP and neighbour suppression (neigh_suppress on), which FRRouting says should be enabled only on VXLAN interfaces. It cuts flooding and the amplification it enables.
• Rely on duplicate MAC detection (RFC 7432, N = 5 moves in M = 180 seconds by default) and alert on it. A duplicate MAC alert in an EVPN fabric is either a misconfiguration or an attack, and both need a human.

Layer 4: treat EVPN as BGP and harden it like BGP

Our BGP explainer covers the general controls. For EVPN specifically:

• Authenticate every session. RFC 7432 points to TCP based authentication of BGP messages "making it harder to spoof updates". FRRouting supports neighbor PEER password (TCP MD5). Use TCP-AO (RFC 5925) where your platforms support it.
• Configure neighbours explicitly. Do not use dynamic or listen ranges on route reflectors that accept anyone in a subnet.
• Use GTSM on directly connected eBGP underlay sessions. FRRouting's neighbor PEER ttl-security hops 1 enforces RFC 5082, so a BGP packet from more than one hop away is dropped.
• Set prefix limits with maximum-prefix on underlay sessions so a misbehaving peer cannot flood the table.
• Filter EVPN routes by type, VNI and route target. FRRouting route maps support match evpn route-type with the names ead, macip, multicast, es and prefix, and match evpn vni. A leaf that should never receive type 5 routes from another leaf can reject them. A border leaf can accept type 5 only for the VRFs it serves.
• Keep route targets per tenant and never wildcard across tenants. Wildcard route targets make operations easier and isolation weaker; if you use them, scope them to one tenant.
• Watch the RIB. show bgp l2vpn evpn route type prefix and type macip should be boring. A new VTEP address, a new route target, or a MAC that suddenly appears behind a different VTEP is worth an alert.

Decision table: which control stops which attack

No single column stops everything. A compromised leaf is a trusted VTEP and a trusted BGP speaker at the same time, which is why monitoring the EVPN tables matters as much as filtering.

How to build a VXLAN and EVPN lab with Linux bridges, iproute2 and FRRouting

The lab has two parts. Part A uses only iproute2 and network namespaces on one Linux machine to show the data plane and flood and learn behaviour. Part B uses two Linux virtual machines with FRRouting to run an EVPN control plane with the hardening settings above.

These commands change network configuration. Run them only in a lab virtual machine you can throw away, not on a production host or a machine you reach over SSH through the interfaces you are changing. Before each block, check that the interface names in the commands match your machine with ip -br link, and that nothing you rely on already uses them.

Part A, step 1: create two VTEPs and an underlay link

# Check first: these names must not already exist
ip netns list
ip -br link | grep -E 'veth-a|veth-b' || echo "names free"

sudo ip netns add vtep1
sudo ip netns add vtep2
sudo ip link add veth-a type veth peer name veth-b
sudo ip link set veth-a netns vtep1
sudo ip link set veth-b netns vtep2

sudo ip -n vtep1 addr add 192.0.2.1/24 dev veth-a
sudo ip -n vtep2 addr add 192.0.2.2/24 dev veth-b
for ns in vtep1 vtep2; do
  sudo ip -n $ns link set lo up
done
sudo ip -n vtep1 link set veth-a up
sudo ip -n vtep2 link set veth-b up
sudo ip netns exec vtep1 ping -c 2 192.0.2.2

The veth pair stands in for the routed underlay. In a real fabric the two VTEP addresses would be loopbacks on different leaves, reachable through spines.

Part A, step 2: add a VXLAN device and a bridge on each side

# VTEP 1
sudo ip -n vtep1 link add vxlan100 type vxlan id 100 \
  local 192.0.2.1 remote 192.0.2.2 dev veth-a dstport 4789 nolearning
sudo ip -n vtep1 link add br100 type bridge
sudo ip -n vtep1 link set vxlan100 master br100
sudo ip -n vtep1 addr add 10.100.0.1/24 dev br100
sudo ip -n vtep1 link set vxlan100 up
sudo ip -n vtep1 link set br100 up

# VTEP 2
sudo ip -n vtep2 link add vxlan100 type vxlan id 100 \
  local 192.0.2.2 remote 192.0.2.1 dev veth-b dstport 4789 nolearning
sudo ip -n vtep2 link add br100 type bridge
sudo ip -n vtep2 link set vxlan100 master br100
sudo ip -n vtep2 addr add 10.100.0.2/24 dev br100
sudo ip -n vtep2 link set vxlan100 up
sudo ip -n vtep2 link set br100 up

The ip-link man page defines remote as the unicast destination for frames whose destination "is not known in the VXLAN device forwarding database", so remote is a one peer flood list. Note the explicit dstport 4789; without it the kernel would use 8472. The dev option names the underlay interface, which lets the kernel size the VXLAN device's MTU from it.

Part A, step 3: watch the encapsulation

# Terminal 1: capture on the underlay
sudo ip netns exec vtep2 tcpdump -ni veth-b -vv udp port 4789

# Terminal 2: send overlay traffic
sudo ip netns exec vtep1 ping -c 3 10.100.0.2

# Inspect the device and forwarding table
sudo ip -n vtep1 -d link show vxlan100
sudo ip netns exec vtep1 bridge fdb show dev vxlan100
sudo ip -n vtep1 link show br100 | grep mtu

The capture shows outer IP 192.0.2.1 to 192.0.2.2, UDP to port 4789 with a varying source port, a VXLAN header with VNI 100, and the inner ARP and ICMP frames in clear text. The forwarding table contains an all-zero MAC entry pointing at the remote VTEP, which is the default flood destination. The VXLAN device, and the bridge above it, should report an MTU of 1450 on a 1,500 byte veth, matching worked example 2.

Look at the capture again with an attacker's eyes: the VNI, the inner MAC addresses, the inner IP addresses and the payload are all readable by anyone on the underlay path. That is the property the security section is about.

Part A, step 4: see why data plane learning is risky

Recreate the VTEP 2 device with learning enabled (learning instead of nolearning), send traffic, and look at bridge fdb show dev vxlan100 again. You will see the inner source MAC of VTEP 1's bridge learned behind 192.0.2.1. The device learned that binding from the first packet that carried it, without any check that 192.0.2.1 was entitled to speak for that MAC. In a flood and learn network, the same mechanism lets any sender who can reach the VTEP teach it a wrong binding, which is why EVPN fabrics turn learning off. We do not show how to craft such packets; the lesson is in the forwarding table entry itself.

Part A, step 5: add a VTEP source filter with nftables

sudo ip netns exec vtep2 nft -f - <<'EOF'
table inet vxlan_guard {
  set vteps {
    type ipv4_addr
    elements = { 192.0.2.1 }
  }
  chain input {
    type filter hook input priority 0; policy accept;
    udp dport 4789 ip saddr != @vteps counter drop
  }
}
EOF
sudo ip netns exec vtep2 nft list table inet vxlan_guard

This drops VXLAN packets from any source that is not a known VTEP, while leaving other traffic alone. In production, use the whole VTEP prefix rather than single addresses, keep the policy in configuration management, and alert on the drop counter: any increase means something outside your VTEP list is sending VXLAN to this host.

Clean up Part A with sudo ip netns del vtep1 and sudo ip netns del vtep2, which removes the namespaces and the devices inside them.

Part B, step 1: two leaves with FRRouting

Use two Linux virtual machines, leaf1 and leaf2, each with one underlay interface (eth1, linked to the other VM) and one host facing interface (eth2). Install FRRouting from your distribution or the project's packages and enable bgpd in /etc/frr/daemons. Check the version with vtysh -c "show version".

On leaf1 (mirror on leaf2 with .2 addresses), following the single VXLAN device model from the FRRouting EVPN documentation:

# Check first: eth1 and eth2 must be the lab interfaces, not your management link
ip -br addr show eth1 eth2

sudo ip addr add 10.255.0.1/32 dev lo                 # VTEP address
sudo ip addr add 192.0.2.0/31 dev eth1                # underlay link (leaf2 uses 192.0.2.1/31)
sudo ip link set eth1 mtu 9000 up

sudo ip link add br0 type bridge vlan_filtering 1 vlan_default_pvid 0
sudo ip link add vxlan0 type vxlan dstport 4789 local 10.255.0.1 \
  nolearning external vnifilter
sudo ip link set br0 addrgenmode none
sudo ip link set vxlan0 addrgenmode none master br0
sudo bridge link set dev vxlan0 vlan_tunnel on neigh_suppress on learning off

# Map VLAN 10 on the bridge to L2 VNI 110
sudo bridge vlan add dev vxlan0 vid 10
sudo bridge vni add dev vxlan0 vni 110
sudo bridge vlan add dev vxlan0 vid 10 tunnel_info id 110

# Host facing access port in VLAN 10
sudo ip link set eth2 master br0
sudo bridge vlan add dev eth2 vid 10 pvid untagged
sudo ip link set eth2 up
sudo ip link set br0 up
sudo ip link set vxlan0 up

FRRouting's documentation notes that it "does not manage network interfaces directly"; it reads them from the kernel. The external flag hands the forwarding decisions to the control plane, vnifilter restricts the device to VNI 110, and learning off with nolearning means only EVPN can teach this VTEP where remote MACs live. Because this device has no dev, the kernel does not derive its MTU from eth1; check it with ip -d link show vxlan0 and set it explicitly if your tenants need more than the default.

Part B, step 2: the BGP and EVPN configuration

On leaf1, in /etc/frr/frr.conf (leaf2 mirrors it with its own addresses):

frr defaults datacenter
hostname leaf1
!
router bgp 65000
 bgp router-id 10.255.0.1
 neighbor 192.0.2.1 remote-as internal
 neighbor 192.0.2.1 password lab-only-change-me
 !
 address-family ipv4 unicast
  network 10.255.0.1/32
  neighbor 192.0.2.1 maximum-prefix 100
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor 192.0.2.1 activate
  neighbor 192.0.2.1 route-map EVPN-IN in
  advertise-all-vni
 exit-address-family
exit
!
route-map EVPN-IN permit 10
 match evpn route-type macip
exit
route-map EVPN-IN permit 20
 match evpn route-type multicast
exit
route-map EVPN-IN deny 100
exit

What each part does:

• advertise-all-vni enables EVPN in the underlay VRF; the FRRouting BGP documentation calls it "mandatory for EVPN-VXLAN to work".
• The network 10.255.0.1/32 line advertises the VTEP address so the other leaf can reach it.
• password turns on TCP MD5 for the session. In a real fabric, store the secret outside the file and rotate it.
• maximum-prefix 100 caps the underlay routes this peer may send.
• EVPN-IN accepts only type 2 (macip) and type 3 (multicast) routes from this peer and denies everything else, including type 5. A border leaf that needs type 5 would add a permit entry for prefix scoped to the right VNIs or route targets.

Both leaves use the same AS and iBGP, so their automatically derived route targets (AS:VNI) match. FRRouting's documentation explains that with a unique AS per leaf, which is common in eBGP underlays, auto derived route targets differ and you must configure route targets explicitly.

Restart FRRouting with sudo systemctl restart frr.

Part B, step 3: verify the control plane

sudo vtysh -c "show bgp l2vpn evpn summary"
sudo vtysh -c "show evpn vni"
sudo vtysh -c "show bgp l2vpn evpn route type multicast"
sudo vtysh -c "show bgp l2vpn evpn route type macip"
sudo bridge fdb show dev vxlan0
sudo bridge vni show

You should see one type 3 route per leaf for VNI 110 (the flood list), and once a host behind each eth2 sends traffic, type 2 routes for their MAC addresses. The kernel forwarding table on each leaf should show the remote host MAC pointing at the other leaf's VTEP address with no "learned from data plane" entries, because learning is off.

Part B, step 4: add symmetric IRB

To route between subnets, follow the FRRouting example: create a VRF bound to a kernel table, give it an L3 VNI (vrf vrf1 then vni 100 in FRRouting), map a spare VLAN to VNI 100 on the bridge, attach a VLAN interface for each L2 VNI to the VRF with the same gateway IP on every leaf, and add advertise ipv4 unicast under address-family l2vpn evpn in the VRF's BGP instance to export type 5 routes. Use the same anycast gateway MAC on every leaf, with a local forwarding entry for it, as the FRRouting documentation shows. Then extend EVPN-IN to accept prefix routes only on the leaves that need them.

Part B, step 5: apply the underlay filter

Apply the nftables guard from Part A, step 5 on both leaves with the set containing the VTEP prefix (10.255.0.0/24 in this lab, using flags interval on the set for a prefix). Then confirm with tcpdump -ni eth1 udp port 4789 that tunnel traffic still flows between the loopbacks and that the drop counter stays at zero.

How Geneve and SRv6 compare as alternatives

VXLAN is not the only overlay. Two others come up in every design review.

Geneve (RFC 8926, November 2020) was written by the NVO3 working group as a single extensible encapsulation. It runs over UDP port 6081, keeps a 24-bit VNI, and adds a variable length options area described by a 6-bit length field, so control planes can carry metadata such as policy tags in the packet. It also has a C bit to mark critical options a receiver must understand. Its security section is as blunt as VXLAN's: it "does not have any inherent security mechanisms". The options make Geneve attractive to software defined networking for virtual machines and containers, where the controller wants to carry policy or tracing metadata with each packet.

SRv6 (RFC 8986, February 2021) is different in kind. There is no UDP shim and no separate VNI. Each service is a 128-bit IPv6 address called a segment identifier (SID), structured as locator, function and argument. RFC 8986 defines behaviours such as End.DT2U and End.DT2M for Layer 2 services and End.DT4, End.DT6 and End.DT46 for Layer 3 VPNs. RFC 9252 (July 2022) carries EVPN and Layer 3 VPN services over SRv6 with BGP, so EVPN can run on SRv6 instead of VXLAN. Security again depends on the boundary: RFC 8754 says "Nodes outside the SR domain are not trusted" and requires edge filters that drop any packet entering the domain with a destination SID, warning that without it the domain is open to source routing attacks. The Segment Routing Header also has an optional HMAC TLV for authenticating segment lists.

The security lesson is the same across all three: the encapsulation is not a boundary. The boundary is the set of filters that decide who can send packets to tunnel endpoints or SIDs, and the policy that decides which control plane routes are accepted.

Common mistakes

• Leaving the Linux default port. A Linux VXLAN device without dstport 4789 uses 8472. Interoperability breaks, and firewall rules written for 4789 silently miss the traffic.
• Treating the VNI as a security boundary. It is a clear text 24-bit number. Isolation comes from who can reach the VTEP and what the VTEP accepts.
• Exposing VTEP addresses outside the fabric. If any router outside the fabric can route to a VTEP loopback, so can an attacker. Keep the VTEP block out of tenant and external routing tables and filter it at the edge.
• Leaving data plane learning on in an EVPN fabric. It reintroduces exactly the trust in first packets that EVPN was meant to remove, and it can conflict with EVPN learned entries.
• Undersized underlay MTU. VTEPs must not fragment, and receivers may silently drop fragments. Symptoms are pings that work and large transfers that hang. Size the underlay for 1,550 bytes at minimum, or use jumbo frames.
• Accepting all EVPN route types from every peer. Leaves rarely need type 5 from other leaves. Filter by route type, VNI and route target.
• Wildcard route targets across tenants. They simplify configuration and quietly merge tenants on a typo.
• Unauthenticated BGP to route reflectors. A route reflector that accepts any peer in a subnet turns one compromised host into a control plane speaker.
• Assuming MACsec protects against a compromised switch. It encrypts each link; the frame is clear inside every switch it crosses.
• Ignoring duplicate MAC alerts. In EVPN they mean a loop, a misconfiguration or a spoof, and all three need attention.
• Forgetting the hosts that are VTEPs. Container and virtual machine hosts that terminate VXLAN or Geneve need the same source filtering as switches.

FAQ

What is the difference between VXLAN and EVPN?

VXLAN is a data plane: an encapsulation that carries Ethernet frames in UDP with a 24-bit VNI (RFC 7348). EVPN is a control plane: BGP routes that tell VTEPs which MAC and IP addresses sit behind which VTEP and which VTEPs belong to each VNI (RFC 7432, adapted to VXLAN by RFC 8365). You can run VXLAN without EVPN, using multicast flood and learn or static peers, but most modern fabrics use both.

What port does VXLAN use?

IANA assigned UDP destination port 4789, and RFC 7348 says it should be the default. The Linux kernel defaults to 8472 for historical reasons, so set dstport 4789 explicitly. Geneve uses UDP 6081 and VXLAN-GPE uses 4790.

What is a VTEP in VXLAN?

A VXLAN Tunnel End Point is the device that encapsulates tenant frames into VXLAN and decapsulates them on the other side. It can be a hypervisor's virtual switch, a Linux host, or a top of rack switch. Each VTEP has an IP address in the underlay, usually a loopback.

How many VXLAN segments can you have compared with VLANs?

A 12-bit VLAN ID gives 4,094 usable VLANs. A 24-bit VNI gives 16,777,216 values, about 4,096 times more. In practice the limit is the number of VNIs a switch's hardware can program, which is far lower than the identifier space.

What are EVPN route types 2, 3 and 5?

Type 2 (MAC/IP Advertisement) announces a host's MAC address and optionally its IP. Type 3 (Inclusive Multicast Ethernet Tag) announces that a VTEP belongs to a VNI and should receive flooded traffic for it. Type 5 (IP Prefix, RFC 9136) announces IP prefixes in a tenant VRF, such as a default route from a border leaf.

What is the difference between symmetric and asymmetric IRB?

In asymmetric IRB the ingress leaf routes and bridges the packet into the destination subnet, so every leaf needs every subnet and ARP entries for every remote host. In symmetric IRB both the ingress and egress leaves route, through a per tenant L3 VNI, so each leaf keeps only local subnets and local ARP entries. RFC 9135 defines both.

Is VXLAN encrypted?

No. RFC 7348 provides no authentication or encryption, and the inner frames, MAC addresses and VNI travel in clear text. Use MACsec on underlay links, IPsec between VTEPs where traffic crosses untrusted paths, and encryption inside the tenant.

Can someone inject traffic into a VXLAN network?

Yes, if they can send UDP packets to a VTEP on the VXLAN port, the VTEP will decapsulate frames for any VNI it serves. RFC 7348's security section describes this. The defence is to make VTEP addresses unreachable from outside the fabric and to accept VXLAN only from known VTEP sources.

Should I use Geneve or SRv6 instead of VXLAN?

Geneve fits when your control plane needs to carry metadata in each packet, as software overlays for virtual machines and containers often do. SRv6 fits networks that are already IPv6 and want one technology for transport and services. VXLAN with EVPN is the long established choice for data center switch fabrics, with a fixed header that is simple to process. None of the three is secure by itself.

How do I see EVPN routes in FRRouting?

Use vtysh -c "show bgp l2vpn evpn route", optionally with type macip, type multicast or type prefix, plus show evpn vni for VNI state. On Linux, bridge fdb show dev vxlan0 shows what the kernel installed.

Where Safeguard fits

Safeguard does not configure or monitor network fabrics, it does not scan switch, NIC or router firmware, and it does not inspect VXLAN or BGP traffic. Nothing in the hardening steps above depends on it.

The honest connection is the software that runs the overlay. On Linux VTEPs and routing on the host designs, the control plane is FRRouting, the data plane is the kernel and iproute2, and around them sit container images for network functions, automation code that renders fabric configuration, and libraries that parse BGP and EVPN state for monitoring. Those are ordinary open source components. For the images and applications you build yourself, Safeguard's container scanning and software composition analysis list which package versions each image and repository carries, SBOM generation records that inventory in SPDX or CycloneDX, and the platform shows which projects contain a given component and version, so "where do we still ship this FRRouting build" has a quick answer. Vulnerability matching, reachability analysis, and policies and gates help you decide what to rebuild first and stop a pipeline that pulls in a known vulnerable version. See using SBOMs to find affected products fast.

Switch operating systems and appliance firmware still have to be fixed by their vendors; the steps below apply to the open source routing stacks, network function images and automation you build and deploy yourself. That software parses untrusted control plane and tunnel traffic, so keeping it current is part of securing the overlay, not a separate project.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

These demos show dependency, SBOM and remediation workflows, not network configuration or traffic monitoring. You can download Safeguard and follow the CLI docs to try them on your own projects.

This demo sets up the Safeguard runner on your own machine, approves it with its device code and scans from the terminal. Pointed at a network function or routing on the host image that bundles FRRouting and iproute2, the same scan lists the packages and versions inside it and their known vulnerabilities, so you know which images to rebuild after a routing daemon advisory.

Click through it at your own pace, or open it full screen.

This demo is a real CLI session: ask it to fix the lodash CVE in a repository and watch it read the finding and the advisory, ask before each command, move lodash to 4.17.21, run npm install and npm test with your approval, and push nothing. The same approve each step flow applies when you bump a dependency in the automation or monitoring code that renders and checks your fabric configuration.

Click through it at your own pace, or open it full screen.

This demo connects Claude to the Safeguard MCP server with OAuth and lets you choose which tools Claude can use. You can then ask which of your projects ship a given package version and get the answer from your own scan data rather than from memory.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• RFC 7348: Virtual eXtensible Local Area Network (VXLAN), August 2014
• RFC 7432: BGP MPLS-Based Ethernet VPN, February 2015
• RFC 8365: A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN), March 2018
• RFC 9135: Integrated Routing and Bridging in Ethernet VPN (EVPN), October 2021
• RFC 9136: IP Prefix Advertisement in Ethernet VPN (EVPN), October 2021
• RFC 7209: Requirements for Ethernet VPN (EVPN), May 2014
• RFC 7365: Framework for Data Center (DC) Network Virtualization, October 2014
• RFC 8584: Framework for EVPN Designated Forwarder Election Extensibility, April 2019
• RFC 9469: Applicability of EVPN to Network Virtualization over Layer 3 (NVO3) Networks, September 2023
• RFC 8926: Geneve: Generic Network Virtualization Encapsulation, November 2020
• RFC 8986: Segment Routing over IPv6 (SRv6) Network Programming, February 2021
• RFC 8754: IPv6 Segment Routing Header (SRH), March 2020
• RFC 9252: BGP Overlay Services Based on Segment Routing over IPv6 (SRv6), July 2022
• RFC 4301: Security Architecture for the Internet Protocol
• IANA: Service Name and Transport Protocol Port Number Registry
• Linux kernel documentation: Virtual eXtensible Local Area Networking
• Linux kernel source: drivers/net/vxlan/vxlan_core.c
• Linux kernel source: include/net/vxlan.h
• ip-link(8), Linux manual page
• bridge(8), Linux manual page
• ip-macsec(8), Linux manual page
• FRRouting documentation: EVPN
• FRRouting documentation: BGP
• FRRouting documentation: Route Maps
• NIST SP 800-215, guide to securing the enterprise network, November 2022
• IEEE SA: IEEE 802.1Q-2022, Bridges and Bridged Networks

Last reviewed: 23 October 2026. RFCs are stable, but Linux, iproute2 and FRRouting options change between releases; check the pages above for the versions you actually run.