Изначально я писал на «Сетевой инженерии», но мне сказали спросить здесь.
У меня есть 2 сервера, подключенных кабелем 10G между ними напрямую. Мне нужно знать, что соединение хорошее и действительно имеет пропускную способность 10G. Если нет, как я могу проверить, в чем проблема? Я использовал IPERF для проверки пропускной способности.
ОБНОВИТЬ Я настроил jumbo-пакеты, экстремальную частоту прерываний, максимальные значения для буферов приема / передачи. Для IPERF я использовал просто «iperf3.exe -s» для сервера и «iperf3.exe -c IP» для клиента.
По тесту пропускная способность 1,5-1,7 Гбит / с. Означает ли это, что соединение 10G? В чем может быть проблема?
Сервер A - это 64-разрядная версия Windows 8.1 Pro.
Сервер B - это 64-разрядная версия Windows 7 Enterprise.
Это результат, когда ServerA (192.168.10.100) является сервером, а ServerB (192.168.10.200) является клиентом:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.10.200, port 63772
[ 5] local 192.168.10.100 port 5201 connected to 192.168.10.200 port 63773
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 170 MBytes 1.42 Gbits/sec
[ 5] 1.00-2.00 sec 217 MBytes 1.82 Gbits/sec
[ 5] 2.00-3.00 sec 217 MBytes 1.82 Gbits/sec
[ 5] 3.00-4.00 sec 214 MBytes 1.80 Gbits/sec
[ 5] 4.00-5.00 sec 217 MBytes 1.82 Gbits/sec
[ 5] 5.00-6.00 sec 207 MBytes 1.74 Gbits/sec
[ 5] 6.00-7.00 sec 199 MBytes 1.67 Gbits/sec
[ 5] 7.00-8.00 sec 206 MBytes 1.73 Gbits/sec
[ 5] 8.00-9.00 sec 192 MBytes 1.61 Gbits/sec
[ 5] 9.00-10.00 sec 209 MBytes 1.75 Gbits/sec
[ 5] 10.00-10.16 sec 30.3 MBytes 1.63 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.16 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.16 sec 2.03 GBytes 1.72 Gbits/sec receiver
Это результат, когда B (192.168.10.200) - сервер, а A (192.168.10.100) - клиент:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.10.100, port 55565
[ 5] local 192.168.10.200 port 5201 connected to 192.168.10.100 port 55566
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 170 MBytes 1.43 Gbits/sec
[ 5] 1.00-2.00 sec 175 MBytes 1.47 Gbits/sec
[ 5] 2.00-3.00 sec 175 MBytes 1.47 Gbits/sec
[ 5] 3.00-4.00 sec 179 MBytes 1.50 Gbits/sec
[ 5] 4.00-5.00 sec 181 MBytes 1.52 Gbits/sec
[ 5] 5.00-6.00 sec 200 MBytes 1.68 Gbits/sec
[ 5] 6.00-7.00 sec 176 MBytes 1.47 Gbits/sec
[ 5] 7.00-8.00 sec 177 MBytes 1.48 Gbits/sec
[ 5] 8.00-9.00 sec 186 MBytes 1.56 Gbits/sec
[ 5] 9.00-10.00 sec 179 MBytes 1.50 Gbits/sec
[ 5] 10.00-10.04 sec 6.42 MBytes 1.46 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.04 sec 1.76 GBytes 1.51 Gbits/sec receiver
ОБНОВЛЕНИЕ 2 - UDP
Когда 192.168.10.100 - сервер
iperf3.exe -c 192.168.10.100 -u -V
iperf 3.1.3
CYGWIN_NT-6.1 ServerB 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
Time: Tue, 09 Aug 2016 19:58:03 GMT
Connecting to host 192.168.10.100, port 5201
Cookie: ServerB.1470772683.561800.7c902a3e559
[ 4] local 192.168.10.200 port 63675 connected to 192.168.10.100 port 5201
Starting Test: protocol: UDP, 1 streams, 8192 byte blocks, omitting 0 seconds, 1
0 second test
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 3.00-4.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 4.00-5.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 5.00-6.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 6.00-7.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 7.00-8.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 8.00-9.00 sec 128 KBytes 1.05 Mbits/sec 16
[ 4] 9.00-10.00 sec 128 KBytes 1.05 Mbits/sec 16
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datag
rams
[ 4] 0.00-10.00 sec 1.25 MBytes 1.05 Mbits/sec 0.437 ms 0/160 (0%)
[ 4] Sent 160 datagrams
CPU Utilization: local/sender 1.7% (0.9%u/0.8%s), remote/receiver 0.1% (0.0%u/0.
1%s)
Когда 192.168.10.200 - это сервер
iperf3.exe -c 192.168.10.200 -u -V
iperf 3.1.3
CYGWIN_NT-6.3 ServerA 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
Time: Tue, 09 Aug 2016 20:37:11 GMT
Connecting to host 192.168.10.200, port 5201
Cookie: ServerA.1470775031.986727.6f8e913650af9b
[ 4] local 192.168.10.100 port 51834 connected to 192.168.10.200 port 5201
Starting Test: protocol: UDP, 1 streams, 8192 byte blocks, omitting 0 seconds, 1
0 second test
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 152 KBytes 1.24 Mbits/sec 19
[ 4] 1.00-2.00 sec 144 KBytes 1.18 Mbits/sec 18
[ 4] 2.00-3.00 sec 160 KBytes 1.31 Mbits/sec 20
[ 4] 3.00-4.00 sec 160 KBytes 1.31 Mbits/sec 20
[ 4] 4.00-5.00 sec 152 KBytes 1.25 Mbits/sec 19
[ 4] 5.00-6.00 sec 152 KBytes 1.24 Mbits/sec 19
[ 4] 6.00-7.00 sec 152 KBytes 1.25 Mbits/sec 19
[ 4] 7.00-8.00 sec 152 KBytes 1.25 Mbits/sec 19
[ 4] 8.00-9.00 sec 152 KBytes 1.24 Mbits/sec 19
[ 4] 9.00-10.00 sec 144 KBytes 1.18 Mbits/sec 18
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datag
rams
[ 4] 0.00-10.00 sec 1.48 MBytes 1.25 Mbits/sec 0.202 ms 0/189 (0%)
[ 4] Sent 189 datagrams
CPU Utilization: local/sender 0.1% (0.1%u/0.0%s), remote/receiver 0.0% (0.0%u/0.
0%s)
ОБНОВЛЕНИЕ 3 - NTTTCP без -NDL
ServerA
ntttcp.exe -s -m 8,*,192.168.10.200 -l 128k -a 2 -t 15
Copyright Version 5.31
Network activity progressing...
Thread Time(s) Throughput(KB/s) Avg B / Compl
====== ======= ================ =============
0 15.000 65544.533 131072.000
1 15.000 65518.933 131072.000
2 15.000 93585.067 131072.000
3 15.002 63906.146 131072.000
4 15.000 65536.000 131072.000
5 15.000 62310.400 131072.000
6 15.001 81837.744 131072.000
7 15.001 73262.316 131072.000
##### Totals: #####
Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
8371.875000 15.001 8854.867 558.088
Throughput(Buffers/s) Cycles/Byte Buffers
===================== =========== =============
4464.702 17.198 66975.000
DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
12120.992 0.823 30450.170 0.328
Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
991381 149616 9 0 36.026
ServerB
ntttcp.exe -r -m 8,*,192.168.10.200 -rb 2M -a 16 -t 15
Copyright Version 5.31
Network activity progressing...
Thread Time(s) Throughput(KB/s) Avg B / Compl
====== ======= ================ =============
0 15.000 65546.450 60268.990
1 14.999 65523.318 60229.282
2 14.999 93582.889 60659.943
3 14.999 63900.843 61621.870
4 15.000 65536.817 60278.174
5 14.999 62316.071 61141.628
6 14.999 81840.156 60394.032
7 14.999 73264.934 60858.538
##### Totals: #####
Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
8371.320061 14.999 8942.835 558.125
Throughput(Buffers/s) Cycles/Byte Buffers
===================== =========== =============
8930.003 5.314 133941.121
DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
7965.264 8.216 14260.151 4.589
Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
146841 981564 1 0 9.699
UPDATE4 - NTTTCP С -NDL
ServerA
ntttcp.exe -s -m 8,*,192.168.10.200 -l 128k -a 2 -t 15 -ndl
Copyright Version 5.31
Network activity progressing...
Thread Time(s) Throughput(KB/s) Avg B / Compl
====== ======= ================ =============
0 14.999 78059.604 131072.000
1 15.000 76902.400 131072.000
2 14.999 59882.392 131072.000
3 14.999 62485.232 131072.000
4 14.999 72734.449 131072.000
5 15.001 77520.165 131072.000
6 15.000 62702.933 131072.000
7 15.000 62771.200 131072.000
##### Totals: #####
Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
8101.250000 15.000 8912.094 540.083
Throughput(Buffers/s) Cycles/Byte Buffers
===================== =========== =============
4320.667 17.355 64810.000
DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
9966.333 0.974 29515.333 0.329
Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
953174 145579 1 0 35.182
ServerB
ntttcp.exe -r -m 8,*,192.168.10.200 -rb 2M -a 16 -t 15 -ndl
Copyright Version 5.31
Network activity progressing...
Thread Time(s) Throughput(KB/s) Avg B / Compl
====== ======= ================ =============
0 15.000 78054.000 60255.789
1 15.001 76903.606 60839.375
2 15.000 59880.183 61786.888
3 14.999 62483.832 60812.856
4 15.000 72730.333 61516.405
5 14.999 77520.335 60398.365
6 14.998 62701.660 60628.917
7 14.998 62771.436 60585.766
##### Totals: #####
Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
8100.944093 14.999 8943.556 540.099
Throughput(Buffers/s) Cycles/Byte Buffers
===================== =========== =============
8641.583 4.888 129615.105
DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
7912.594 8.003 14043.670 4.509
Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
143914 949785 0 0 8.633
Оказалось, что проблема была связана с тем, как клиенты Windows 7/8 управляют масштабированием окна TCP.
По умолчанию клиентские версии Windows предпочитают относительно небольшое окно TCP, отдавая предпочтение задержке по сравнению с пропускной способностью. При использовании очень быстрого соединения, как в этом случае, установка автоматической настройки на нормальный режим может привести к значительному увеличению пропускной способности.
Чтобы установить автоматическую настройку на более разумное значение, вы можете использовать команду netsh interface tcp set heuristics disabled. Во всяком случае, посмотрите Вот Чтобы получить больше информации.