iperf3 in multi-threading to get the most out of 10GbE: a complete guide

In 10GbE networks, increasingly common in home and professional systems, the bottleneck often shifts from the network card to the CPU . Therefore, adjusting iperf3 to leverage multiple threads can make the difference between achieving 6–8 Gbps and nearing link saturation. This guide will show you, with real-world examples, how to take advantage of multithreading, which versions you need, and what hardware pitfalls to avoid to prevent performance loss.

Beyond simply running a command, success depends on the iperf3 version, system configuration, NIC drivers , and even the PCIe link width . You'll see practical examples (from a mini PC with an Intel N100 to a server with TrueNAS and Proxmox), key iperf3 parameters, best practices for interpreting results, and a diagnostic checklist for when the counter gets stuck.

What is iperf3 and why does it matter in 10GbE?

iperf3 is an open-source tool for measuring bandwidth between two computers using TCP, UDP, or SCTP. It is cross-platform ( Windows , Linux , macOS, BSD, among others) and its code was rewritten from scratch compared to iperf2 to simplify it, add JSON output, and create a modular framework that is easier to integrate into other programs.

Unlike other utilities, it allows you to adjust TCP window size, define data amounts, select protocol, and generate multiple simultaneous connections . For UDP, it measures jitter and signal loss, supports multicast, and can set the target throughput; for TCP, it offers options such as zero-copy, congestion control, and detailed output by interval.

On links of 10 Gbps or higher, a single TCP connection may not be enough to saturate the link; distributing the load across multiple streams and, since iperf3 3.16, taking advantage of multithreading more easily helps to fill the pipe when the CPU or network stack is limited.

Versions: minimum requirements and compatibility

First of all, check the version on both ends with iperf3 -v. Since 3.16 iperf3 simplifies multithreading, reducing the need for Tricks y commands additional features that were common in 40 Gbps environments and above. If your distro is behind (typically in hypervisors or appliances), compiling the latest version is usually the fastest way; and check the Network types in Hyper‑V, VirtualBox, and VMware of your environment.

A real-world example: in an environment with OpenWrt/QWRT and Proxmox VE 8.3, OpenWrt already had a recent version, but Proxmox was outdated . This was resolved by compiling iperf 3.18 from source on the Proxmox host, and from then on, multithreading was enabled normally.

Preparing your 10GbE environment: CPU, NIC, drivers, and PCIe

To achieve peak speeds close to 9.8–9.9 Gbps, Cat6A cables and a 10G switch are not enough . You have to look further down the line: driver version, RSS (Receive Side Scaling), interrupt coalescing, offloads (TSO/GSO/GRO/LRO), and, above all, the PCIe link on the card.

On Windows 10/11, especially with 10G NICs like the ASUS XG-C100C, update the driver, enable RSS, and check if SMB Multichannel is active and audit the network connection in Windows . For file transfers with a NAS (e.g., QNAP with QuTS Hero), the speed will also depend on the storage performance and the NAS's CPU; it's normal to see around 6–8 Gbps if the RAID or ZFS configuration isn't robust, while iperf3 (memory-to-memory) can approach 10GbE.

On Linux and BSD, Verify that the card negotiates the correct PCIe width. A typical case: a client was stuck at ~3.5 Gbps despite having 10GbE because the slot was running degraded to x1 at 5 GT/s. The command lspci -vv showed “LnkSta: Speed ​​5GT/s, Width x1 (degraded)”. After adjusting the BIOS To force the port to x16, the speed jumped to ~9.84 Gbps.

Multithreading mode in iperf3: essential commands

The basic test pattern is simple: launch the server with iperf3 -s at one end and run the client from the other. From here, add parallelism:

• Multiple TCP streams: iperf3 -c IP_SERVIDOR -P 4 generates 4 parallel connections. Adjust -P to your CPU/NIC; on 10GbE, 4–8 is usually sufficient.
• Simultaneous bidirectional: iperf3 -c IP_SERVIDOR -P 4 -d measures upload and download at the same time, useful for validating full duplex.
• Reverse direction (server to client traffic): -R.
• Duration: -t 60 for 60 s tests; rule out starts with -O 3 to avoid TCP slow start.
• Zero-copy: -Z reduces CPU load on Linux.
• CPU Affinity: -A sets the affinity (useful for distributing threads across different cores).
• Binding to interface: -B 10.0.0.2 explicitly choose the outgoing IP/IF (see How to list network interfaces in CMD).

If you work with UDP, controls the flow with -b and check for loss/jitter; although for saturating 10GbE and measuring the system stack, TCP is usually the starting point.

Case Study 1: Intel N100 Mini PC with Two 10GbE

Scenario: Two iKOOCORE R2 Max (one fanless with OpenWrt/QWRT as the server and the other with Proxmox VE as the client), both with dual 10GbE and a quad-core Intel N100 CPU . The first measurement showed ~9.41 Gbps upload speed, but the download speed was only ~8.6 Gbps, and the bidirectional test performed even worse.

HTOP showed the track: a single core at 100% while the rest were idle. Although the total CPU usage was around 30%, a single saturated thread was limiting the effective throughput. To resolve this, the server was started as usual (iperf3 -s) and in the client it was added -P 4 to open 4 streams.

The output, with 10 s intervals and four threads, is more verbose, but The important thing is that the target figure was achieved: ~9.41 Gbps. In the full duplex test with -d, the sums averaged ~9.40 Gbps both ways for one minute, confirming that the CPU was already spreading the load across multiple threads and the NIC was saturating the link.

As a practical note: if the client version is old (as was the case with Proxmox), compiling iperf 3.18 resolved the performance gap and compatibility of recent options on the side that needed it most.

Case Study 2: TrueNAS + Proxmox and the PCIe Bottleneck

Configuration: Server with TrueNAS-13.0-U6 (FreeBSD 13.1) on Supermicro x8/x9 boards, dual Intel Westmere E5645 CPUs and Chelsio T420-CR NICs ; client with Proxmox VE 8.1.3 (Debian 12, kernel 6.5.11), dual E5-2420 v2 CPUs and Chelsio T440-CR NICs . Direct wiring via twinax DAC between hosts.

The first test settled at an average of ~3.55 Gbps, with zero retransmissions and a stable congested window. The CPU barely broke a sweat: ~13–20% usage per iperf3 . The decisive clue came when looking at the PCIe link status: “LnkSta: Speed ​​5GT/s, Width x1 (degraded)”. In other words, the 10G NIC was tied to a PCIe x1 lane at 5 GT/s, limiting the throughput.

After delving into the BIOS, a critical option was found on the Supermicro X9DBU-3F: Advanced → Chipset Configuration → North Bridge → Integrated IO Configuration → IIO 1 IOU3 – PCIe Port . Changing it from “Auto” to “x16” restored the link to correct bandwidth and speed. Result: iperf3 recorded a sustained ~9.84 Gbps, with zero retransmissions and a stable Cwnd connection.

Also, opening two clients against two different ports (-p 5000 y -p 5001) ~19.58 Gbps aggregated were reached. In that dual-port scenario, two threads of the Westmere were running at 70–80% and the Ivy Bridge at 30–40%., consistent with multi-threaded loading by pushing traffic through two 10G interfaces at once.

Interpreting results: what to look for and how to adjust

In the iperf3 output, look at the final average rate and the interval lines (default is 1 s; some examples use 10 s). If you multiply the MBytes transferred by 8 and divide by the interval time , the number should match the reported Gbits/s.

If you see a single saturated thread and the total does not increase, increases -P gradually (4, 6, 8) until the throughput stops growing. If it increases slightly but does not reach 9.8–9.9 Gbps, check offloads and RSS. When there are losses or retransmissions in TCP, evaluate -w (window), -O (skip Boot) and the quality of the physical link.

In two-way (-d), look at the sums and the minute to validate that both directions are close to the theoretical maximum. Large differences between upload and download speeds usually indicate affinity, IRQs, or poorly distributed RSS.

When it's not the CPU: Chronic “6 Gbps” diagnosis

If you're stuck at ~6–7 Gbps (very typical in new setups), rule out these points one by one :

• Old version: 3.7 clients with 3.16 servers work, but upgrade both to benefit from multithreading improvements and fixes.
• PCIe degraded: check width and speed with lspci -vv (Linux) or vendor tools; a x1 at 5 GT/s throttles the 10G NIC.
• RSS disabled: without interrupt sharing, only one core remains at 100%. On Windows, enable it in the driver; on Linux, check ethtool -l and RPS/RFS.
• Offloads and coalescing: TSO/GSO/GRO/LRO and interrupt coalescing reduce CPU load; adjust with ethtool.
• MTU/Jumbo frames: It doesn't always increase throughput, but it helps with weak CPUs. Enable it on both ends and on the switch if it supports it.
• Storage: In NAS tests, the bottleneck may be RAID/ZFS; use iperf3 to isolate the network (memory to memory) and not the disk.
• SMB and multichannel: On Windows, SMB Multichannel speeds up backups; if it's not enabled, files will run slower even if iperf3 clocks in at 9–10 Gbps.

It is also a good idea to adjust system buffers in Linux when searching for sustained peaks: sysctl net.core.rmem_max y net.core.wmem_max High speeds prevent socket bottlenecks; they are not mandatory for 10GbE on low-latency LANs, but they help in demanding scenarios.

Quick installation on Windows, Linux, and macOS

On Windows download the trusted binaries, unzip and run from CMD o PowerShell; If Windows does not detect the network card, see the solutions guide. Open a console, go to the folder and run iperf3.exe -s o iperf3.exe -c IP. Make sure to allow port 5201/TCP on your firewall if you're testing across subnets.

On Linux/macOS, install is usually a command: on Debian/Ubuntu sudo apt-get install iperf3, on RHEL/CentOS sudo yum install iperf3, on macOS with Homebrew brew install iperf3If your system is outdated, compiling from source gives you the latest version.

Useful iperf3 parameters (client and server)

To control the session in detail, these flags save you time :

• General: -p port, --cport client port (>=3.1), -f format (k/K/m/M/g/G), -i interval, -F archive, -B bind, -V verbose, -J JSON, --logfile (>=3.1), -h help, -v version.
• employee: -s server, -D devil, -I PID file.
• Client: -c IP, --sctp SCTP, -u UDP -b bandwidth, -t weather, -n bytes, -k packages, -l buffer length, -P threads, -R reverse, -w TCP window, -M MSS, -N TCP no-delay, -4/-6 IPv4/IPv6, -S T.O.S. -L IPv6 tag, -Z zero-copy, -O skip boot, -T Title, -C congestion algorithm.

Remember -P applies by flow. If you fix -b If UDP or the system imposes socket limits, each thread may hit its own ceiling; experiment with fewer but better-tuned threads.

Impact and Limitations: What iperf3 Measures (and What It Doesn't)

iperf3 is excellent for stress testing the network, but it can flood your LAN if run in a production environment . Avoid long tests during peak hours, and coordinate with other teams if you share trunk links. Its traffic pattern doesn't always replicate actual application loads.

Among its drawbacks, it lacks a built-in graphical interface, requires tweaking parameters , and doesn't automatically save historical data. While it supports many protocols, there are scenarios (e.g., certain IPv6 functionalities or specific multiplexing) where other tools might be a better fit.

Alternatives when you need something else

If iperf3 doesn't fit your needs, these options might save you :

• hyperf2: legacy but useful for old compatibilities; there is classic GUI (jperf).
• netperf: very powerful for TCP/UDP and latency, more technical to handle.
• nuttcp: lightweight for measuring bandwidth/loss without too much complication.
• bwping/bwping6: ICMP to estimate bandwidth when you can't open TCP/UDP (less accurate).
• Speedtest CLI / Fast.com CLI: for public Internet; do not replace iperf3 on LANs.

If your goal is to get the most out of 10GbE, start with iperf3 multithreading: verify version ≥3.16, enable 4–8 threads, check RSS and the PCIe link, and review drivers/offloads . With these minimum requirements, modest systems like an Intel N100 have saturated 10GbE (9.40–9.85 Gbps), and older configurations have gone from 3.5 to 9.8 Gbps after correcting a simple x1 PCIe issue. The key is to distribute the workload across cores, avoid hidden bottlenecks, and read the appropriate metrics to know when to adjust each parameter.

Related articles:

VirtualBox: Complete Guide to Network Types, Uses, and Tricks