Build a 10 Gigabit Homelab Network: SFP+, DAC, RJ45, VLANs, Testing

A 10 gigabit homelab network is worth building when a machine regularly moves more data than 1 GbE can carry: a NAS that reads faster than about 110 MB/s, large backups, hypervisor live migration, or copying local AI model files. Use SFP+ ports with short direct attach copper (DAC) cables inside the rack, multimode fibre between rooms, and 10GBASE-T over Category 6A only where you must reuse copper. Put storage and management on their own VLANs, keep the MTU consistent, and prove every link with iperf3 and ethtool.

This guide goes one level deeper than our home NAS and home firewall on a mini PC builds. Every figure comes from an IEEE 802.3 task force page, an SFF specification, a Linux manual page or kernel document, the iperf3 project, or a research network's published tuning guidance. Hardware is described by category, never by brand.

Key takeaways

• IEEE 802.3ae (10 Gb/s Ethernet over fibre) was approved in June 2002, 10GBASE-T (IEEE 802.3an) on 8 June 2006, and 2.5GBASE-T and 5GBASE-T (IEEE 802.3bz) on 22 September 2016. All now sit in IEEE 802.3-2022, board approved on 13 May 2022.
• The 802.3bz objectives target 2.5 Gb/s over 100 m of Cat 5e and 5 Gb/s over 100 m of Cat 6. If your walls hold Cat 5e, 2.5 GbE is often the cheapest upgrade.
• SFP+ modules carry an identity and diagnostics memory defined in SFF-8472 (Rev 12.5a, 16 January 2026). ethtool -m reads it. Switches that check the identity bytes are the root of "coded optics" problems.
• SFF-8472 lists SFP+ power classes of 1.0 W (Level 1), 1.5 W (Level 2), 2.0 W (Level 3) and above 2.0 W (Level 4). Passive DAC draws almost nothing; 10GBASE-T copper modules sit at the top.
• At 10 Gb/s a 1500 byte MTU means about 813,000 full size frames per second; a 9000 byte MTU means about 138,000. Every device in the broadcast domain must agree, because Ethernet cannot detect an MTU mismatch.
• Measure before you tune. Since iperf 3.16 each parallel stream runs in its own thread, so old "iperf3 can't do 10G" results may have been the tool, not the network.
• Treat switch and NIC firmware as inventory: record versions, update through signed channels, and keep management interfaces on a VLAN ordinary devices cannot reach.

When does a homelab actually need 10 GbE?

Start with arithmetic. After Ethernet, IP and TCP overhead, a single TCP stream on 1 GbE with a 1500 byte MTU tops out around 941 Mb/s of payload, roughly 117 MB/s. Real file copies land between about 105 and 115 MB/s. That ceiling shows up in four situations.

A NAS faster than the wire. Any SSD pool, or several spinning disks striped together, reads sequentially far faster than 117 MB/s. Video editing from the NAS, restoring VM images and replicating datasets all hit the wire limit first.

Backups and restores. A 1 TB copy at 112 MB/s takes 1,000,000 MB / 112 MB/s, about 8,930 seconds or 2 hours 29 minutes. At a realistic 10 GbE copy rate of 1,100 MB/s it takes about 909 seconds, just over 15 minutes, if the disks keep up. The restore is when you feel it.

Live migration. Moving a running VM copies its memory, then the pages that changed during the copy, and repeats. A 32 GB guest needs about 4.8 minutes of raw transfer at 112 MB/s and about 29 seconds at 1,100 MB/s. A faster link shrinks the window in which dirty pages pile up, so migrations converge more reliably. The Proxmox VE network documentation notes that aggregated links can improve live migration delays; one 10 GbE link does the same job more simply.

Local AI models. Model weight files are often tens of gigabytes. A 70 GB file takes about 625 seconds (10.4 minutes) at 112 MB/s and about 64 seconds at 1,100 MB/s.

You do not need 10 GbE for streaming to a TV, browsing, a single hard disk NAS, Wi-Fi clients, or an internet link under 1 Gb/s.

What about 2.5 GbE and 5 GbE?

IEEE 802.3bz was written for this middle ground. Its objectives, approved by the 802.3 working group on 12 March 2015, were to support MAC rates of 2.5 and 5 Gb/s, keep the 802.3 frame format and frame sizes unchanged, support Clause 28 auto negotiation, optional Energy Efficient Ethernet, and Power over Ethernet including 802.3bt. The cabling targets:

• 2.5 Gb/s over up to at least 100 m of four pair Class D (Cat 5e) cabling, on defined use cases.
• 5 Gb/s over up to at least 100 m of Class E (Cat 6), and up to 100 m of Class D on defined use cases.

The IEEE-SA Standards Board approved IEEE Std 802.3bz-2016 on 22 September 2016. A practical design is a 10 GbE core (NAS, hypervisors, switch uplinks) with 2.5 GbE edge ports for desktops and access points, often using the network chip already on the motherboard.

The physical layer: SFP+, DAC, fibre or 10GBASE-T?

Three options use the SFP+ cage, a pluggable slot whose electrical interface is defined by the SFF Committee (now a technical working group of the Storage Networking Industry Association) in SFF-8431, with its management memory in SFF-8472. The fourth is an RJ45 port with a 10GBASE-T PHY.

SFP+ with direct attach copper (DAC)

A DAC is a twinax cable with an SFP+ plug fixed at each end. A passive DAC has no signal electronics; the host's serializer drives the copper directly, and each plug holds only a small identification EEPROM. Active DACs add signal conditioning for longer runs.

SFF-8472 identifies these cables as compliant to "SFF-8431 Appendix E" and notes there is "no formal IEEE designation for passive and active cable interconnects". DAC is a multi source agreement product, not an IEEE PHY, which is why compatibility is decided by switch and NIC firmware rather than by a standard name.

DAC is the default for anything in the same rack. It is cheap, uses almost no power, adds negligible latency and has no optics to clean. Passive DAC is typically sold up to about 7 m.

SFP+ with multimode fibre (10GBASE-SR)

10GBASE-SR uses an 850 nm laser over multimode fibre with duplex LC connectors. SFF-8472's transceiver code table lists it as "10.3125 GBd 300 m OM3". OM3 or OM4 multimode is the right choice for runs between rooms: immune to interference, no ground potential between buildings, thin enough to pull where Cat 6A will not go. SR optics typically sit in SFF Power Level 1 (1.0 W maximum).

SFP+ with single mode fibre (10GBASE-LR)

10GBASE-LR uses a 1310 nm laser over single mode fibre; SFF-8472 lists it as "10.3125 GBd 10 km". Nobody needs 10 km at home, but single mode is worth considering for a new long run because the same fibre carries much higher speeds later with different optics. Single strand bidirectional optics also exist and must be bought as matched pairs.

10GBASE-T over twisted pair

The IEEE P802.3an task force's 10GBASE-T work was approved by the IEEE-SA Standards Board on 8 June 2006. Its September 2003 study group objectives targeted at least 100 m on Class F cabling and "at least 55 m to 100 m" on Class E (Cat 6), with media selected from ISO/IEC 11801. The commonly cited outcome is 100 m on Class EA (Cat 6A) and up to 55 m on Cat 6, with Cat 6 reach limited by alien crosstalk between bundled cables. Cat 5e is not a specified medium; short runs sometimes work, but you are outside the standard and should test.

10GBASE-T reuses RJ45 patch panels, and many ports also negotiate 1, 2.5 and 5 Gb/s. The cost is power and heat: recovering 10 Gb/s from twisted pair takes heavy signal processing, which costs several watts per port on many PHY generations. SFP+ to RJ45 copper modules typically declare SFF Power Level 3 or 4 (2.0 W or more), and some switches limit how many you can fit side by side.

Comparison table: cabling options at 10 GbE

SFF power levels are class ceilings from SFF-8472; real modules draw less. Treat "typical" entries as planning guidance and check each product's datasheet.

Which one should you choose?

How do you choose network cards and switches?

Network interface cards

Port type. SFP+ cards let you choose DAC, fibre or copper per port. RJ45 cards are simpler but run warmer. Dual port cards let a host keep storage and VM trunks on separate ports.

Bus bandwidth. PCIe 2.0 runs at 5 GT/s per lane with 8b/10b encoding, about 500 MB/s per lane per direction. PCIe 3.0 runs at 8 GT/s with 128b/130b encoding, about 985 MB/s per lane. One 10 GbE port at line rate needs 1,250 MB/s per direction. So:

• PCIe 3.0 x1 caps one port at about 7.9 Gb/s.
• PCIe 2.0 x4 (about 2,000 MB/s) carries one port at line rate but not two at once.
• PCIe 2.0 x8 or PCIe 3.0 x4 (about 4,000 MB/s) carries a dual port card at full rate.

Consumer boards often wire a full length slot as x4 electrically or share lanes with an M.2 slot. Check what the link trained to with lspci -vv (the LnkSta line).

Driver support. Prefer a chipset whose driver is in the mainline Linux kernel, and in the BSD kernel if your firewall or storage runs BSD. In kernel drivers get fixes through ordinary distribution updates and expose the standard ethtool interfaces.

Queues and offloads. The Linux kernel's scaling documentation explains Receive Side Scaling (RSS): the NIC hashes each packet's addresses and ports (usually a Toeplitz hash) and uses the low bits to index an indirection table that maps flows to receive queues, and so to CPUs. Without RSS every packet lands on one core, the most common reason a 10 GbE link stalls at 3 to 5 Gb/s on a small machine. Checksum, segmentation and receive offloads cut per packet CPU work further.

Used enterprise NICs are popular because they are cheap and well supported. Check that the card is not locked to one server maker's firmware, that it has airflow (many expect a server chassis fan), and that its firmware can still be updated.

Switches

Managed or unmanaged. An unmanaged switch puts every port in one VLAN and has nothing to configure. That is fine for a two host storage link. For VLANs, link aggregation, port statistics or mirroring you need a managed switch with IEEE 802.1Q support (current edition 802.1Q-2022, approved 21 September 2022) and ideally LACP, covered in our link aggregation guide.

Port mix. A common layout is a four to eight port SFP+ switch for servers and uplinks, plus a 1 or 2.5 GbE client switch with one or two SFP+ uplinks. Count the NAS, each hypervisor, the firewall, the client uplink, and two spares.

Fanless or not. Fanless SFP+ switches rely on convection. Put DAC and fibre in them, not hot 10GBASE-T modules.

Security relevant management features. A management VLAN or out of band port, SSH and HTTPS with plain text protocols disabled, text configuration export, firmware still receiving updates with published release notes, syslog, SNMPv3, and per port disable. Our switch manufacturing post explains why switch firmware is a full network operating system with its own patch stream.

Open source switch software. Linux's switchdev driver model and its Distributed Switch Architecture (DSA) subsystem let switch chips appear as ordinary network interfaces configured with ip and bridge. Small open router distributions build on DSA, and open network operating systems run on larger switches. For most homelabs, a managed switch plus Linux bridges on the hosts is enough.

What are coded optics and compatibility issues?

Every SFP+ module and DAC carries an identity in its EEPROM. SFF-8472 places the serial ID at 2-wire address A0h: vendor name in bytes 20 to 35 (ASCII), the vendor's IEEE company ID (OUI) in bytes 37 to 39, and part number in bytes 40 to 55, plus transceiver codes, nominal length, wavelength and power level declarations.

Some switches and NIC drivers read those bytes and refuse modules not on an allow list. The Linux kernel documentation for one widely used 10 GbE driver family documents a module parameter, allow_unsupported_sfp, default 0 (disabled), which "allows unsupported and untested SFP+ modules" as long as the module type is known to the driver. Many switches have no such override.

Third party sellers respond by programming ("coding") each module's identity bytes to match what a given switch expects. That works, but:

• The identity bytes say what the module was coded for, not who made it.
• Support contracts may exclude third party optics.
• A module coded for a switch may be rejected by a NIC, so mixed links may need differently coded ends.
• A switch firmware update can change which modules it accepts. Test a spare after each update.

Favour hardware that accepts standards compliant modules, and confirm each module with ethtool -m after installing it.

How should VLANs and segmentation work at 10 GbE?

The goal is the same as on any home network: separate traffic by trust and route between zones only through a policy you wrote. At 10 GbE, add one rule: bulk storage traffic should be switched inside a VLAN at line rate, not forced through a firewall that cannot route at 10 Gb/s.

Keep storage non routed. Give the NAS and each hypervisor an address on VLAN 40 with no gateway on that interface. Storage is then switched in hardware at line rate, and a device on another VLAN has no route to your storage protocols even if a firewall rule is wrong.

Trunk to hypervisors. Proxmox VE documents two approaches: a "VLAN aware" Linux bridge, where each guest's virtual NIC gets a VLAN tag and the bridge handles it, and a "traditional" method that creates a VLAN device and a bridge per VLAN. The VLAN aware bridge (bridge-vlan-aware yes with a bridge-vids range) is simpler at scale.

Prune trunks. List exactly which VLANs each trunk carries. A NAS port that needs storage and management should not carry IoT.

Where does the firewall sit?

1. Firewall routes everything (router on a stick). One trunk to the firewall carries every VLAN, and all inter-VLAN traffic goes up and back down it. Fully policy controlled, but the firewall must route at the rate you want.
2. Firewall routes user traffic; storage stays local. Storage and migration VLANs are not routed, so heavy traffic never touches the firewall. The right default for most homelabs.
3. A layer 3 switch routes internal VLANs. Line rate routing in hardware with access control lists, while the firewall handles internet and sensitive zones. Switch ACLs are usually stateless and harder to audit, so use this only if you need line rate routing between zones.

Packet rate explains why placement matters. A minimum 64 byte frame plus 8 bytes of preamble and 12 bytes of inter-frame gap is 84 bytes, or 672 bits, on the wire. At 10 Gb/s that is 10,000,000,000 / 672, about 14.88 million frames per second. No small firewall routes that through connection tracking and a full rule set. With 1500 byte packets (1538 bytes on the wire) the rate is about 813,000 per second, which a multi core machine with RSS can handle. A firewall that shows 10 Gb/s in one bulk TCP test may fall far short with many small flows.

What are jumbo frames, and when do they help or hurt?

IEEE 802.3 keeps a maximum untagged frame of 1518 bytes (1522 with an 802.1Q tag), carrying a 1500 byte payload; the 802.3an and 802.3bz objectives both say to "preserve" the minimum and maximum frame size of the current standard. Jumbo frames are a widely supported but non standard extension, most often a 9000 byte MTU.

Worked example: packet rate and efficiency at 10 Gb/s

1500 byte MTU. Each full frame costs 1500 bytes of payload, 14 bytes of Ethernet header, 4 of frame check sequence, 8 of preamble and 12 of gap: 1538 bytes, or 12,304 bits. At 10 Gb/s that is about 812,700 frames per second. A TCP segment carries 1500 minus 20 (IPv4) minus 20 (TCP) minus 12 (timestamps) = 1448 bytes of data, so maximum goodput is 1448 / 1538 × 10 Gb/s, about 9.41 Gb/s.

9000 byte MTU. Each frame is 9038 bytes, or 72,304 bits, so about 138,300 frames per second. TCP payload is 8948 bytes per segment, and goodput is 8948 / 9038 × 10 Gb/s, about 9.90 Gb/s.

Jumbo frames buy about 5 percent more goodput and about 5.9 times fewer packets for the CPU. ESnet's guidance makes the same two points: a lower packet rate "requiring less CPU for packet processing", and faster TCP recovery after loss because recovery scales with segment size. On a clean LAN, the CPU saving is what matters, and NIC segmentation and receive offloads already capture much of it at 1500 bytes.

When jumbo frames hurt

ESnet's MTU page states the core problem: all hosts in a broadcast domain must use the same MTU, and "Ethernet has no way of detecting an MTU mismatch". Symptoms are connections that open and then hang on the first large transfer, SMB or NFS mounts that list directories but stall on copies, and iSCSI sessions that drop under load. Causes:

• Switch frame size below host MTU. The switch silently drops oversize frames. Set the switch maximum (often 9216) above any host MTU.
• One host left at 1500. TCP may cope via a small MSS, but UDP based protocols fail.
• Routing between a jumbo VLAN and a 1500 VLAN. The router must send ICMP Fragmentation Needed (IPv4, RFC 1191) or Packet Too Big (IPv6, RFC 8201). If a firewall drops those, you get a path MTU black hole. RFC 4821's Packetization Layer Path MTU Discovery lets TCP probe for a working size without ICMP, and ESnet recommends net.ipv4.tcp_mtu_probing=1 on jumbo frame hosts (it notes a value of 2 sometimes causes performance problems).
• Overlays. VXLAN, WireGuard and similar tunnels add headers; lower the overlay MTU or raise the underlay.

The safe pattern: jumbo frames only on non routed VLANs, the same value on every host and switch port in that VLAN, routed VLANs left at 1500, and a do not fragment ping test.

How do you test and tune a 10 GbE network?

Measure with iperf3

iperf3, maintained by ESnet, measures TCP, UDP and SCTP throughput between two hosts. From the iperf3 manual:

ESnet's iperf page explains that iperf3 "was originally single threaded", so multi stream tests were often CPU bound on one core, and that iperf 3.16 removed that constraint.

Read the NIC and module with ethtool

From the ethtool manual:

• ethtool -S returns standard and "NIC- and driver-specific statistics". Watch CRC, symbol, missed and no buffer counters; names vary by driver.
• ethtool -m "retrieves and if possible decodes the EEPROM from plugin modules, e.g SFP+, QSFP", plus optical diagnostics when supported: temperature, supply voltage, laser bias, transmit and receive power, which SFF-8472 places in the real time diagnostic registers at A2h bytes 96 to 111.
• ethtool -l and -L show and set channels; "a channel is an IRQ and the set of queues that can trigger that IRQ".
• ethtool -g and -G show and set ring sizes; -c and -C show and set interrupt coalescing (adaptive-rx, rx-usecs, rx-frames); -k and -K handle offloads; -a and -A handle pause frames; --show-fec shows forward error correction.

Tune only what the measurement points at

Interrupt moderation. If a small CPU is pegged by interrupts, raise rx-usecs or enable adaptive coalescing; if latency matters more, lower it. Change one value and rerun.

Multiqueue. Confirm several combined channels with ethtool -l and that their interrupts land on different cores in /proc/interrupts. The kernel scaling document describes the software equivalents too: Receive Packet Steering, "logically a software implementation of RSS", Receive Flow Steering, which steers packets to the CPU running the consuming application, and Transmit Packet Steering. RPS helps NICs with one receive queue.

Rings. If receive drops climb under load, raise the receive ring within the driver maximum (ethtool -G eth0 rx 4096).

TCP buffers. On a LAN the bandwidth delay product is tiny: 10 Gb/s × 0.2 ms / 8 = 250,000 bytes, which default autotuning covers. ESnet's Linux tuning page recommends larger limits (for example net.core.rmem_max = 67108864) for long paths; at 10 Gb/s and 50 ms the BDP is 62.5 MB. It also recommends the fq qdisc and, for a 10G host, pacing with tc qdisc add dev ethN root fq maxrate 8gbit to avoid bursts that overflow switch buffers, and no longer recommends switching congestion control away from the default CUBIC. Inside a homelab, those settings matter mainly for off site replication.

CPU limits on routing

A forwarding firewall does per packet work: receive, connection tracking, rule evaluation, NAT, transmit. Throughput scales with packet rate. Run iperf3 between hosts on two routed VLANs with -P 1 and -P 8 while watching mpstat -P ALL 1 on the firewall. One core at 100 percent with others idle means flows are not spreading (expected for a single flow, since RSS hashes a flow to one queue). All cores busy means you have reached the box's routing capacity.

Step by step: build and verify the network

These steps assume Debian based Linux hosts (a NAS and two hypervisors), a managed SFP+ switch and a separate firewall. Interface names and addresses are examples.

Step 1: plan the topology and write it down

Switch port  Device         Link        Mode      VLANs (tagged)     Untagged
sfp1         NAS            DAC 1 m     trunk     10,40              none
sfp2         hv1            DAC 1 m     trunk     10,20,30,40,50     none
sfp3         hv2            DAC 1 m     trunk     10,20,30,40,50     none
sfp4         firewall       DAC 2 m     trunk     10,20,30,60        none
sfp5         office switch  OM4 30 m    trunk     20,60              none
sfp6..8      unused         none        disabled                     999 (parking)

Keep it in Git with your host configuration. Storage (40) and migration (50) never reach the firewall.

Step 2: install NICs and confirm the bus

lspci -nn | grep -i ethernet
sudo lspci -vv -s 01:00.0 | grep -E 'LnkCap|LnkSta'
# LnkCap is what the card supports; LnkSta is what it negotiated.
ethtool -i enp1s0f0     # driver, version, firmware-version: record these

Step 3: install modules and cables, then check the physical layer

Never look into an optical port or fibre end, and clean LC connectors before inserting them.

ethtool enp1s0f0 | grep -E 'Speed|Duplex|Link detected|Port'
sudo ethtool -m enp1s0f0
mkdir -p ~/net-baseline
ethtool -S enp1s0f0 > ~/net-baseline/enp1s0f0-$(date +%F).txt

For optics, compare receive power with the module's stated range. Very low receive power usually means a dirty connector, swapped strands, or a damaged patch cable.

Step 4: configure the switch

1. Change default credentials and disable plain text management protocols.
2. Create VLANs 10 to 60 and a parking VLAN 999 with no members.
3. Set trunk ports with explicit tagged VLAN lists and no untagged VLAN where the device does not need one.
4. Move unused ports to VLAN 999 and disable them.
5. Set maximum frame size on ports carrying VLANs 40 and 50 to the switch's jumbo maximum.
6. Put the switch management address on VLAN 10 only.
7. Export the configuration as text and commit it next to the plan.

Step 5: configure VLANs and MTU on the NAS

For a quick test, the ip-link manual documents ip link add link DEVICE name NAME type vlan id VLANID, with protocol either 802.1Q or 802.1ad:

sudo ip link set dev enp1s0f0 mtu 9000 up     # parent must be >= any VLAN MTU
sudo ip link add link enp1s0f0 name enp1s0f0.10 type vlan id 10
sudo ip link add link enp1s0f0 name enp1s0f0.40 type vlan id 40
sudo ip link set dev enp1s0f0.10 mtu 1500 up
sudo ip link set dev enp1s0f0.40 mtu 9000 up
sudo ip addr add 192.168.10.5/24 dev enp1s0f0.10
sudo ip addr add 10.40.0.5/24 dev enp1s0f0.40   # storage: no gateway
sudo ip route add default via 192.168.10.1

To persist with systemd-networkd, systemd.netdev defines a [VLAN] section with Id= (0 to 4094) and an MTUBytes= setting, and systemd.network attaches VLANs with VLAN=:

# /etc/systemd/network/10-enp1s0f0.network
[Match]
Name=enp1s0f0
[Link]
MTUBytes=9000
[Network]
VLAN=enp1s0f0.10
VLAN=enp1s0f0.40
LinkLocalAddressing=no

# /etc/systemd/network/20-vlan40.netdev
[NetDev]
Name=enp1s0f0.40
Kind=vlan
MTUBytes=9000
[VLAN]
Id=40

# /etc/systemd/network/30-vlan40.network
[Match]
Name=enp1s0f0.40
[Network]
Address=10.40.0.5/24

Repeat the pair for VLAN 10 with MTUBytes=1500, an address and a gateway. The systemd.network manual also notes that if IPv6 is enabled and the MTU is set below 1280, the minimum for IPv6, it is raised automatically.

Step 6: configure a VLAN aware bridge on each hypervisor

On Proxmox VE, in /etc/network/interfaces:

iface enp1s0f0 inet manual
    mtu 9000

auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp1s0f0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
    mtu 9000

auto vmbr0.10
iface vmbr0.10 inet static
    address 192.168.10.11/24
    gateway 192.168.10.1
    mtu 1500

auto vmbr0.40
iface vmbr0.40 inet static
    address 10.40.0.11/24
    mtu 9000

Add vmbr0.50 the same way for migration, tag each guest NIC with 20, 30 or 60, and narrow bridge-vids to the VLANs you use. Apply changes from the console, not over the link you are changing.

Step 7: verify MTU end to end

ping -M do -s 8972 -c 3 10.40.0.11       # 9000 - 20 IPv4 - 8 ICMP
ping -M do -s 1472 -c 3 192.168.10.11    # 1500 - 28
ping -6 -M do -s 8952 -c 3 fd40::11      # 9000 - 40 IPv6 - 8 ICMPv6

If 8972 fails and 1472 works, something on the path is still at 1500.

Step 8: run iperf3 baselines

iperf3 -s -B 10.40.0.5                                   # on the NAS
iperf3 -c 10.40.0.5 -t 30 -O 3                           # on hv1
iperf3 -c 10.40.0.5 -t 30 -O 3 -P 4
iperf3 -c 10.40.0.5 -t 30 -O 3 -R
iperf3 -c 10.40.0.5 -t 30 -O 3 --bidir
iperf3 -c 10.40.0.5 -t 30 -O 3 -J > ~/net-baseline/hv1-nas-$(date +%F).json
ethtool -S enp1s0f0 | grep -Ei 'err|drop|miss|crc|no_buf'

A clean link shows about 9.4 Gb/s at MTU 1500 and about 9.9 Gb/s at 9000, with near zero retransmits in the Retr column. If one stream gives 6 Gb/s and -P 4 gives 9.4, the single stream is CPU limited. If directions differ widely, check LnkSta and ring drops on the slower receiver. CRC or symbol errors that grow under load point at the physical layer: a marginal DAC, a dirty fibre, or copper outside its specification. Finally, test the routed path from VLAN 20 to VLAN 30 while watching the firewall's CPU.

Step 9: update NIC and switch firmware, and record versions

The LVFS documentation says fwupd "can query supported hardware for the current firmware versions and also deploy new firmware versions", and that the LVFS supplies its update data "in a secure format, allowing the fwupd project to install the update safely":

fwupdmgr get-devices
fwupdmgr refresh
fwupdmgr get-updates
fwupdmgr update

Many NICs and most switches are not on the LVFS. For those, download firmware only from the maker's support site over HTTPS, verify the checksum or signature, read the release notes for security fixes and optics changes, export the configuration, keep console access, update one device at a time, rerun Steps 3 and 8, and record the new version.

Security and maintenance

Keep switch firmware current. Switch firmware is a network operating system with a web server, SSH server, SNMP agent and open source libraries. Subscribe to the maker's advisories, and put a switch that no longer gets updates on your replacement list.

Isolate the management plane. Management interfaces belong on VLAN 10, reachable only from an admin workstation through a firewall rule. Disable HTTP, Telnet and SNMPv1 and v2c; use SSH, HTTPS and SNMPv3. Never expose management to IoT, guests or the internet.

Disable unused ports. Shut them down and park them in a VLAN with no members and no routing.

Avoid default and native VLAN pitfalls. Use explicit tagged lists on trunks and leave the switch's default VLAN unused. When a trunk's untagged VLAN equals an access VLAN, double tagged frames can in some designs reach a VLAN they should not; tag only trunks avoid that class of problem.

Edge protections and logs. Enable loop protection or spanning tree edge guards and storm control on host ports if offered; a looped cable at 10 Gb/s floods the network in seconds. Send syslog to a central host and sync time with NTP.

Keep a firmware inventory.

Monthly: check advisories, compare ethtool -i with the inventory, review switch logs for link flaps, rerun a 30 second iperf3 per link, and compare error counters with the baseline.

A dated timeline of the standards behind a 10 GbE homelab

Common mistakes

1. Buying 10 GbE for a single hard disk. It cannot fill the link; 2.5 GbE may be enough.
2. A slot that cannot feed the NIC. PCIe 3.0 x1 caps one port near 7.9 Gb/s; PCIe 2.0 x4 cannot run two ports at line rate. Check LnkSta.
3. Mixed MTUs on one VLAN. Transfers start, then hang. One MTU per VLAN, switch frame size above it, and a ping -M do test.
4. Routing storage through the firewall. Every byte crosses its CPU. Use a non routed storage VLAN.
5. Blocking all ICMP. Dropping Packet Too Big and Fragmentation Needed breaks path MTU discovery on routed paths.
6. Assuming any module works anywhere. Coded optics checks reject modules. Check documentation and keep a tested spare.
7. 10GBASE-T over unknown cable. Cat 5e and long Cat 6 are outside the specification. Test and watch CRC counters.
8. Hot copper modules in a fanless switch. At 2 W or more each, use DAC and fibre instead.
9. Tuning before measuring. Long path sysctl values rarely help a LAN. Measure, change one thing, measure again.
10. Management on the default VLAN, and unknown firmware versions. Move management to its own VLAN, and keep the inventory current.

FAQ

Is 10 GbE worth it for a homelab?

Yes when a machine regularly moves more than about 110 MB/s: an SSD or multi disk NAS, large backups, live migration, or large AI model files. For a single hard disk or Wi-Fi clients, 2.5 GbE or 1 GbE is the better spend.

Should I use SFP+ or 10GBASE-T RJ45?

SFP+ with DAC inside a rack and fibre between rooms: cooler, cheaper per port, lower latency. 10GBASE-T when you must reuse Cat 6A or connect RJ45 only devices, with more power and heat.

What is a DAC cable?

A twinax copper cable with an SFP+ plug fixed at each end. Passive DACs contain only an identification EEPROM and suit runs of a few metres. They are defined by SFF-8431 Appendix E, not an IEEE PHY.

Can 10GBASE-T run over Cat 6 or Cat 5e?

It is specified for 100 m on Cat 6A and up to 55 m on Cat 6. Cat 5e is not a specified medium. On Cat 5e, 2.5GBASE-T (100 m per IEEE 802.3bz objectives) is the standards based upgrade.

What MTU should I use for jumbo frames?

9000 on hosts, with the switch at its jumbo maximum so it is always larger. Use it only on non routed storage and migration VLANs, on every device in them, and test with ping -M do -s 8972.

Why does iperf3 only show 3 to 5 Gb/s on my 10 GbE link?

Usually a saturated CPU core (try -P 4 on iperf 3.16 or later), too few PCIe lanes, RSS not spreading interrupts, or a firewall or VM without enough CPU in the path. Check LnkSta, ethtool -l and per core CPU.

What are coded SFP+ optics?

Modules whose EEPROM identity bytes (vendor name, OUI and part number at A0h under SFF-8472) are programmed to match what a particular switch expects. Support terms and future firmware may treat them differently.

How do I check SFP+ temperature and optical power on Linux?

Run sudo ethtool -m <interface>. With digital diagnostics support, it shows temperature, voltage, laser bias, and transmit and receive power from the SFF-8472 diagnostic registers at A2h.

How do I update network card firmware on Linux?

If fwupd and the LVFS support the card, run fwupdmgr refresh, fwupdmgr get-updates and fwupdmgr update. Otherwise use the maker's updater, verify the checksum, and record the version from ethtool -i.

Where Safeguard fits

Safeguard does not configure switches, test links, manage NICs, scan device firmware or monitor network traffic, and nothing in the build above depends on it. The honest connection is the software around the network: container images on the hypervisors, small scripts and dashboards that collect iperf3 results or switch statistics, and storage and backup services that pull open source packages every time they are rebuilt.

For code and container images you build or pull yourself, Safeguard's software composition analysis builds a dependency inventory, container scanning lists the package versions inside an image, and SBOM generation records the result in SPDX or CycloneDX. The platform shows which projects contain a given component and version, and vulnerability matching and reachability analysis help decide what to fix first. It is the same "what version runs where" discipline as the firmware inventory above, applied to software.

A rebuilt hypervisor or NAS pulls packages and images again, and the management interfaces and storage services in a homelab ship open source components that receive security fixes on someone else's schedule. Switch and NIC firmware still comes from their makers, but for the open source dependencies and images you run on top, the approach below applies.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

These demos show dependency, SBOM and remediation workflows, not switch configuration, link testing or firmware updates. You can download Safeguard and follow the CLI docs to try them on your own projects.

This demo sets up the Safeguard runner on your own machine, approves it with its device code and scans from the terminal. Pointed at a container image on your hypervisor, such as a dashboard that graphs iperf3 results, the same scan lists the packages inside it and their known vulnerabilities.

Click through it at your own pace, or open it full screen.

This demo is a real CLI session: ask it to fix the lodash CVE in a repository and it reads the finding and advisory, asks before each command, moves lodash to 4.17.21, runs npm install and npm test with your approval, and pushes nothing. The same flow applies to a small tool you wrote to parse iperf3 JSON or collect switch statistics.

Click through it at your own pace, or open it full screen.

This demo connects Claude to the Safeguard MCP server with OAuth and lets you choose which tools Claude can use. You can then ask which of your homelab projects or images ship a given package version and get the answer from your own scan data.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• IEEE 802.3-2022, IEEE Standard for Ethernet (IEEE SA)
• IEEE P802.3ae 10Gb/s Ethernet Task Force
• IEEE P802.3an (10GBASE-T) Task Force
• IEEE 802.3 10GBASE-T Study Group objectives, September 2003
• IEEE P802.3bz 2.5G/5GBASE-T Task Force
• IEEE 802.3 NGEABT objectives, approved 12 March 2015
• IEEE 802.1Q-2022, Bridges and Bridged Networks (IEEE SA)
• SFF-8472 Rev 12.5a, Management Interface for SFP+ (SNIA SFF TWG)
• INF-8074i, SFP Transceiver (SFF Committee)
• ethtool(8) manual page
• ip-link(8) manual page
• systemd.netdev(5) manual page
• systemd.network(5) manual page
• iperf3 project repository (ESnet)
• iperf3: Invoking iperf3 (manual)
• ESnet Fasterdata: iperf2 and iperf3
• ESnet Fasterdata: Linux host tuning
• ESnet Fasterdata: MTU issues
• Linux kernel documentation: Scaling in the Linux networking stack
• Linux kernel documentation: Ethernet switch device driver model (switchdev)
• Linux kernel documentation: Distributed Switch Architecture
• Linux kernel documentation: a 10 GbE NIC driver, including allow_unsupported_sfp
• Proxmox VE: Network Configuration
• LVFS documentation: introduction
• fwupd and the Linux Vendor Firmware Service
• RFC 1191: Path MTU Discovery
• RFC 8201: Path MTU Discovery for IP version 6
• RFC 4821: Packetization Layer Path MTU Discovery

Last reviewed: 24 October 2026. Option names follow the current online ethtool, ip-link, systemd and iperf3 manuals; check the manual pages installed on your own system for the options your versions support.