iperf3 ku-multi-threading ukuze uthole okuningi ku-10GbE: umhlahlandlela ophelele

Kumanethiwekhi e-10GbE, avame kakhulu ezinhlelweni zasekhaya nezobungcweti, inkinga ivame ukushintsha kusuka ekhadini lenethiwekhi iye ku-CPU . Ngakho-ke, ukulungisa i-iperf3 ukuze kusetshenziswe imicu eminingi kungenza umehluko phakathi kokufinyelela ku-6-8 Gbps kanye nokugcwala kwesixhumanisi okuseduze. Lo mhlahlandlela uzokukhombisa, ngezibonelo zomhlaba wangempela, ukuthi ungayisebenzisa kanjani i-multithreading, yiziphi izinguqulo ozidingayo, nokuthi yiziphi izingibe zehadiwe okufanele uzigweme ukuze uvimbele ukulahleka kokusebenza.

Ngaphandle kokusebenzisa umyalo nje, impumelelo incike kunguqulo ye-iperf3, ukucushwa kwesistimu, abashayeli be -NIC , ngisho nobubanzi besixhumanisi se-PCIe . Uzobona izibonelo ezisebenzayo (kusukela ku-PC encane ene- Intel N100 kuya kuseva ene-TrueNAS ne-Proxmox), amapharamitha ayisihluthulelo e-iperf3, imikhuba emihle yokuhumusha imiphumela, kanye nohlu lokuhlola lokuxilonga lapho ikhawunta ibambeka khona.

Iyini iperf3 futhi kungani ibalulekile ku-10GbE?

I-iperf3 iyithuluzi elivulekile lokulinganisa i-bandwidth phakathi kwamakhompyutha amabili asebenzisa i-TCP, i-UDP, noma i-SCTP. Iyi-cross-platform ( i-Windows , i-Linux , i-macOS, i-BSD, phakathi kwezinye) futhi ikhodi yayo yabhalwa kabusha kusukela ekuqaleni uma iqhathaniswa ne-iperf2 ukuze ibe lula, ingeze umphumela we-JSON, futhi idale uhlaka lwe-modular olulula ukuluhlanganisa kwezinye izinhlelo.

Ngokungafani nezinye izinsiza, ikuvumela ukuthi ulungise usayizi wewindi le-TCP, uchaze inani ledatha, ukhethe iphrothokholi, futhi ukhiqize ukuxhumana okuningi ngasikhathi sinye . Ku-UDP, ilinganisa i-jitter kanye nokulahleka kwesignali, isekela i-multicast, futhi ingamisa i-target throughput; ku-TCP, inikeza izinketho ezifana ne-zero-copy, i-congestion control, kanye ne-detailed output by interval.

Kuzixhumanisi ze-10 Gbps noma ngaphezulu, uxhumano olulodwa lwe-TCP lungase lunganeli ukugcwalisa isixhumanisi; ukusabalalisa umthwalo emifudlaneni eminingi futhi, njengoba iperf3 3.16, ukusebenzisa i-multithreading kalula kusiza ukugcwalisa ipayipi lapho i-CPU noma i-network stack ilinganiselwe.

Izinguqulo: ubuncane bezidingo nokuhambisana

Okokuqala, hlola inguqulo kuzo zombili iziphetho nge iperf3 -v. Njengoba 3.16 iperf3 yenza lula ukufunda okuningi, ukunciphisa isidingo tricks y imiyalo izici ezengeziwe ezazivamile ezindaweni ezingu-40 Gbps nangaphezulu. Uma i-distro yakho ingemuva (imvamisa kuma-hypervisors noma ezintweni zikagesi), ukuhlanganisa inguqulo yakamuva ngokuvamile kuyindlela eshesha kakhulu; futhi hlola Izinhlobo zenethiwekhi ku-Hyper-V, VirtualBox, ne-VMware yendawo yakho.

Isibonelo sangempela: endaweni ene-OpenWrt/QWRT kanye ne-Proxmox VE 8.3, i-OpenWrt yayivele inenguqulo yakamuva, kodwa i-Proxmox yayiphelelwe yisikhathi . Lokhu kwaxazululwa ngokuhlanganisa i-iperf 3.18 kusuka emthonjeni ku-host ye-Proxmox, futhi kusukela ngaleso sikhathi kuqhubeke, i-multithreading yavulwa ngokujwayelekile.

Ilungiselela indawo yakho ye-10GbE: CPU, NIC, abashayeli, ne-PCIe

Ukuze ufinyelele isivinini esiphezulu esiseduze no-9.8–9.9 Gbps, izintambo ze-Cat6A kanye neswishi ye-10G akwanele . Kufanele ubheke phambili emugqeni olandelayo: inguqulo yomshayeli, i-RSS (Receive Side Scaling), ukuphazamisa ukuxhuma, ukulayisha (TSO/GSO/GRO/LRO), kanye, ngaphezu kwakho konke, isixhumanisi se-PCIe ekhadini.

Ku-Windows 10/11, ikakhulukazi ngama-NIC angu-10G njenge-ASUS XG-C100C, buyekeza idrayivu, vumela i-RSS, bese uhlola ukuthi i-SMB Multichannel iyasebenza yini bese uhlola uxhumano lwenethiwekhi ku-Windows . Ngokudluliswa kwamafayela nge-NAS (isb., i-QNAP ene-QuTS Hero), isivinini sizoncika ekusebenzeni kwesitoreji kanye ne-CPU ye-NAS; kuvamile ukubona cishe ama-Gbps angu-6–8 uma ukucushwa kwe-RAID noma i-ZFS kungaqinile, kuyilapho i-iperf3 (inkumbulo-kuya-kumemori) ingasondela ku-10GbE.

Ku-Linux naku-BSD, Qinisekisa ukuthi ikhadi lixoxisana ngobubanzi obulungile be-PCIe. Icala elijwayelekile: iklayenti belibambeke ku-~3.5 Gbps naphezu kokuba ne-10GbE ngenxa yokuthi isikhala besisebenza sehliswe ku-x1 ku-5 GT/s. Umyalo lspci -vv ibonise “LnkSta: Speed ​​​​5GT/s, Width x1 (yehlisiwe)”. Ngemva kokulungisa i- I-BIOS Ukuze uphoqelele imbobo ku-x16, isivinini seqe ku-~9.84 Gbps.

Imodi yokufunda okuningi ku-iperf3: imiyalo ebalulekile

Iphethini yokuhlola eyisisekelo ilula: vula iseva nge iperf3 -s ngakolunye uhlangothi bese usebenzisa iklayenti kolunye. Ukusuka lapha, engeza ukufana:

• Ukusakaza okuningi kwe-TCP: iperf3 -c IP_SERVIDOR -P 4 ikhiqiza ukuxhumana oku-4 okufanayo. Lungisa -P ku-CPU/NIC yakho; ku-10GbE, i-4–8 ngokuvamile yanele.
• I-bidirectional ngesikhathi esisodwa: iperf3 -c IP_SERVIDOR -P 4 -d ilinganisa ukulayisha nokulanda ngesikhathi esifanayo, iwusizo ekuqinisekiseni i-duplex egcwele.
• Isiqondiso esiphendukayo (i-server kuya kuthrafikhi yeklayenti): -R.
• Isikhathi: -t 60 ngokuhlolwa kwe-60s; ukukhipha iqala nge -O 3 ukugwema ukuqala kancane kwe-TCP.
• Ikhophi enguziro: -Z yehlisa umthwalo we-CPU ku-Linux.
• CPU Affinity: -A isetha ukuhambisana (iwusizo ekusabalaliseni imicu phakathi kwamakhora ahlukene).
• Ibophezela esibonakalayo: -B 10.0.0.2 khetha ngokucacile i-IP/IF ephumayo (bona Uzibhala kanjani izixhumanisi zenethiwekhi ku-CMD).

Uma usebenza ne-UDP, ilawula ukugeleza nge -b futhi uhlole ukulahlekelwa/i-jitter; nakuba ngokugcwalisa i-10GbE nokulinganisa isitaki sohlelo, i-TCP ivamise ukuba yindawo yokuqala.

I-Case Study 1: I-Intel N100 Mini PC ene-Two 10GbE

Isimo: Ama-iKOOCORE R2 Max amabili (eyodwa engena-fan ene-OpenWrt/QWRT njengeseva kanti enye ene-Proxmox VE njengeklayenti), zombili zine-10GbE ekabili kanye ne-Intel N100 CPU ene-quad-core . Ukulinganisa kokuqala kubonise isivinini sokulayisha esingu-~9.41 Gbps, kodwa isivinini sokulanda sasingu-~8.6 Gbps kuphela, kanti ukuhlolwa kwe-bidirectional kwenze okubi nakakhulu.

I-HTOP ibonise ithrekhi: umgogodla owodwa ku-100% kuyilapho abanye bengasebenzi. Nakuba ukusetshenziswa kwe-CPU sekukonke bekuku-30%, umucu owodwa ogcwele wawubeka umkhawulo ekusebenzeni okusebenzayo. Ukuxazulula lokhu, iseva yaqalwa njengokujwayelekile (iperf3 -s) futhi kuklayenti yengezwe -P 4 ukuvula imifudlana emi-4.

Okukhiphayo, okunezikhawu ezingu-10 kanye nemicu emine, kune-verbose kakhulu, kodwa Into ebalulekile ukuthi isibalo esihlosiwe sifinyelelwe: ~9.41 Gbps. Ekuhlolweni okuphelele kwe-duplex nge -d, izibalo zaba isilinganiso esingu-~9.40 Gbps ngazo zombili izindlela ngomzuzu owodwa, okuqinisekisa ukuthi i-CPU yayisivele isakaza umthwalo emicu eminingi futhi i-NIC yayigcwalisa isixhumanisi.

Njengenothi elisebenzayo: uma inguqulo yeklayenti indala (njengoba kwakunjalo nge-Proxmox), ukuhlanganisa i-iperf 3.18 kuxazulule igebe lokusebenza kanye nokuhambisana kwezinketho zakamuva ohlangothini oluyidinga kakhulu.

Indaba Eyisibonelo 2: I-TrueNAS + Proxmox kanye ne-PCIe Bottleneck

Ukucushwa: Iseva ene-TrueNAS-13.0-U6 (FreeBSD 13.1) kumabhodi e-Supermicro x8/x9, ama-CPU amabili e-Intel Westmere E5645 kanye nama-NIC e-Chelsio T420-CR ; iklayenti eline-Proxmox VE 8.1.3 (Debian 12, kernel 6.5.11), ama-CPU amabili e-E5-2420 v2 kanye nama-NIC e-Chelsio T440-CR . Ukuxhuma ngqo nge-twinax DAC phakathi kwama-host.

Ukuhlolwa kokuqala kuhlale ku-~3.55 Gbps, kungekho ukudluliselwa kabusha kanye nefasitela eliqinile eliminyene. I-CPU ayikwazanga ukujuluka: ~13–20% ukusetshenziswa kwe-iperf3 ngayinye . Inkomba eqondile yavela lapho kubhekwe isimo sesixhumanisi se-PCIe: “LnkSta: Isivinini 5GT/s, Ububanzi x1 (obonakele)”. Ngamanye amazwi, i-10G NIC yayiboshwe kulayini we-PCIe x1 ku-5 GT/s, okunciphisa ukuphuma.

Ngemva kokuhlola i-BIOS, kwatholakala inketho ebalulekile ku-Supermicro X9DBU-3F: Okuthuthukisiwe → Ukucushwa kwe-Chipset → I-North Bridge → Ukucushwa kwe-IO Okuhlanganisiwe → IIO 1 IOU3 - I-PCIe Port . Ukuyishintsha kusuka ku-“Auto” kuya ku-“x16” kwabuyisela isixhumanisi ku-bandwidth nesivinini esilungile. Umphumela: i-iperf3 iqophe i-~9.84 Gbps eqhubekayo, ngaphandle kokudluliselwa kabusha kanye nokuxhumeka kwe-Cwnd okuzinzile.

Futhi, ukuvula amaklayenti amabili ngokumelene namachweba amabili ahlukene (-p 5000 y -p 5001) ~19.58 Gbps ehlanganisiwe ifinyelelwe. Kuleso simo se-dual-port, imicu emibili ye-Westmere ibisebenza ku-70–80% kanye ne-Ivy Bridge ku-30–40%., okuhambisana nokulayisha okunemicu eminingi ngokuphusha ithrafikhi ezindaweni ezimbili ze-10G ngesikhathi esisodwa.

Ukuhumusha imiphumela: yini okufanele uyibheke nokuthi ungayilungisa kanjani

Ku-output ye-iperf3, bheka isilinganiso sokugcina sesilinganiso kanye nemigqa yesikhala (okuzenzakalelayo kungu-1 s; ezinye izibonelo zisebenzisa amasekhondi ayi-10). Uma uphinda ama-MBytes adluliselwe ngo-8 bese uwahlukanisa ngesikhathi sesikhala , inombolo kufanele ifane nama-Gbits/s abikiwe.

Uma ubona intambo eyodwa egcwele futhi inani linganyuki, ukwanda -P kancane kancane (4, 6, 8) kuze kube yilapho i-output iyeka ukukhula. Uma inyuka kancane kodwa ingafinyeleli ku-9.8–9.9 Gbps, hlola okulayishiwe kanye ne-RSS. Uma kukhona ukulahlekelwa noma ukudluliswa kabusha ku-TCP, hlaziya -w (iwindi), -O (yeqa ibhuthini) kanye nekhwalithi yesixhumanisi esibonakalayo.

Ngezindlela ezimbili (-d), buka izibalo nomzuzu ukuze uqinisekise ukuthi zombili izikhombisi-ndlela ziseduze nomkhawulo wetiyori. Umehluko omkhulu phakathi kwesivinini sokulayisha nokulanda ngokuvamile ubonisa ukuhambisana, ama-IRQ, noma i-RSS esakazwa kabi.

Uma kungeyona i-CPU: Ukuxilongwa Okungapheli kwe-“6 Gbps”

Uma ubambeke ku-~6–7 Gbps (okujwayelekile kakhulu ekusethweni okusha), susa la maphuzu ngalinye ngalinye :

• Uhlobo oludala: Amaklayenti angu-3.7 anamaseva angu-3.16 ayasebenza, kodwa thuthukisa kokubili ukuze uzuze ekuthuthukisweni nasekulungisweni kokufunda okuningi.
• I-PCIe yehlisiwe: hlola ububanzi nesivinini nge lspci -vv (Linux) noma amathuluzi omthengisi; i-x1 ku-5 GT/s icindezela i-10G NIC.
• I-RSS ikhutshaziwe: ngaphandle kokwabelana ngokuphazamiseka, ingqikithi eyodwa kuphela esala ku-100%. Ku-Windows, yivule kumshayeli; ku-Linux, hlola ethtool -l kanye ne-RPS/RFS.
• Ukulayisha nokuhlanganisa: I-TSO/GSO/GRO/LRO nokuphazamisa ukuhlangana kunciphisa umthwalo we-CPU; lungisa nge ethtool.
• Ozimele be-MTU/Jumbo: Ayikhulisi njalo i-throughput, kodwa isiza ngama-CPU abuthakathaka. Inike amandla kuzo zombili iziphetho kanye nakuswishi uma iyisekela.
• Isitoreji: Ezivivinyweni ze-NAS, ibhodlela lingase libe yi-RAID/ZFS; sebenzisa iperf3 ukuhlukanisa inethiwekhi (inkumbulo kumemori) hhayi idiski.
• I-SMB kanye neziteshi eziningi: Ku-Windows, i-SMB Multichannel isheshisa izipele; uma inikwe amandla, amafayela azosebenza kancane ngisho noma i-iperf3 iwashi ngo-9–10 Gbps.

Kungumqondo omuhle futhi ukulungisa amabhafa wesistimu ku-Linux lapho ucinga iziqongo eziqhubekayo: sysctl net.core.rmem_max y net.core.wmem_max Isivinini esiphezulu sivimbela amabhodlela e-socket; aziphoqelekile ku-10GbE kuma-LAN aphansi, kodwa zisiza ezimweni ezinzima.

Ukufakwa okusheshayo ku-Windows, Linux, ne-macOS

Ku-Windows landa kanambambili othenjwayo, vula unzip bese uqalisa I-CMD o I-PowerShell; Uma iWindows ingaliboni ikhadi lenethiwekhi, bona inkomba yezisombululo. Vula ikhonsoli, iya kufolda bese ugijima iperf3.exe -s o iperf3.exe -c IP. Qiniseka ukuthi uvumela imbobo engu-5201/TCP ku-firewall yakho uma uhlola kuwo wonke ama-subnet.

Ku-Linux/macOS, ukufaka ngokuvamile kuwumyalo: ku-Debian/Ubuntu sudo apt-get install iperf3, ku-RHEL/CentOS sudo yum install iperf3, ku-macOS ene-Homebrew brew install iperf3Uma isistimu yakho isiphelelwe yisikhathi, ukuhlanganisa okuvela emthonjeni kukunikeza inguqulo yakamuva.

Amapharamitha e-iperf3 awusizo (iklayenti neseva)

Ukuze ulawule iseshini ngokuningiliziwe, lawa mafulegi akongela isikhathi :

• Jikelele: -p itheku, --cport imbobo yeklayenti (>=3.1), -f ifomethi (k/K/m/M/g/G), -i isikhawu, -F ingobo yomlando, -B bopha, -V i-verbose, -J JSON, --logfile (>=3.1), -h Usizo, -v inguqulo.
• Iseva: -s iseva, -D sathane, -I Ifayela le-PID.
• Ikhasimende: -c IP, --sctp I-SCTP, -u UDP, -b umkhawulokudonsa, -t isimo sezulu, -n amabhayithi, -k amaphakheji, -l ubude bebhafa, -P imicu, -R hlehla, -w Iwindi le-TCP, -M MSS, -N Ukungabambezeli kwe-TCP, -4/-6 IPv4/IPv6, -S KHOHLISA, -L Ithegi ye-IPv6, -Z ikhophi enguziro, -O yeqa ibhuthi, -T Isihloko, -C ukuminyana kwe-algorithm.

Khumbula lokho -P isebenza ngokugeleza. Uma ulungisa -b Uma i-UDP noma isistimu ibeka imingcele yesokhethi, intambo ngayinye ingase ishaye uphahla lwayo; zama ngochungechunge olumbalwa kodwa olushunwe kangcono.

Umthelela kanye Nemikhawulo: Yiziphi Izinyathelo ze-iperf3 (nokuthi Ingenzi)

I-iperf3 ilungele ukuhlola ukucindezeleka kwenethiwekhi, kodwa ingagcwalisa i-LAN yakho uma isebenza endaweni yokukhiqiza . Gwema ukuhlolwa okude ngesikhathi samahora aphezulu, futhi uhlanganyele namanye amaqembu uma wabelana ngezixhumanisi ze-trunk. Iphethini yayo yethrafikhi ayiphindaphindi njalo imithwalo yangempela yohlelo lokusebenza.

Phakathi kwezingqinamba zayo, ayinaso isikhombikubona sezithombe esakhelwe ngaphakathi, idinga ukulungisa amapharamitha , futhi ayilondolozi ngokuzenzakalelayo idatha yomlando. Nakuba isekela amaphrothokholi amaningi, kunezimo (isb., imisebenzi ethile ye-IPv6 noma i-multiplexing ethile) lapho amanye amathuluzi angase afaneleke kangcono.

Ezinye izindlela lapho udinga enye into

Uma i-iperf3 ingafanelani nezidingo zakho, lezi zinketho zingase zikusindise :

• iperf2: ifa kodwa liwusizo ezintweni ezihambisanayo zakudala; kukhona i-GUI yakudala (jperf).
• netperf: inamandla kakhulu ku-TCP/UDP nokubambezeleka, ubuchwepheshe obuningi bokusingathwa.
• nuttcp: Isisindo esincane sokulinganisa umkhawulokudonsa/ukulahlekelwa ngaphandle kwenkinga enkulu.
• bwping/bwping6: I-ICMP yokulinganisa umkhawulokudonsa lapho ungakwazi ukuvula i-TCP/UDP (inembe kancane).
• I-Speedtest CLI / Fast.com CLI: ku-inthanethi yomphakathi; ungashintshi iperf3 kuma-LAN.

Uma umgomo wakho ukuthola okuningi ku-10GbE, qala nge-iperf3 multithreading: qinisekisa inguqulo ≥3.16, vumela izintambo ezingu-4–8, hlola i-RSS kanye nesixhumanisi se-PCIe, bese ubuyekeza abashayeli/ukukhipha amafayela . Ngalezi zidingo ezincane, izinhlelo ezincane njenge-Intel N100 zine-10GbE egcwele (9.40–9.85 Gbps), kanti ukucushwa okudala kusuke ku-3.5 kuya ku-9.8 Gbps ngemuva kokulungisa inkinga elula ye-x1 PCIe. Isihluthulelo ukusabalalisa umthwalo womsebenzi kuwo wonke ama-cores, ugweme izithiyo ezifihliwe, nokufunda amamethrikhi afanele ukuze wazi ukuthi kufanele ulungise nini ipharamitha ngayinye.

I-athikili ehlobene:

I-VirtualBox: Umhlahlandlela ophelele wezinhlobo zenethiwekhi, ukusetshenziswa, namaqhinga