Administrace komentářů

Ahoj, snazim se zjistit pricinu male propustnosti tunelu GRE over IPSec v LTE modemu Conel SPECTRE v3. Na druhe strane je jak GRE, tak IPSec terminovan na malem Ciscu 871.

V modemu je dost osekany Linux (kernel 3.12.10) s busyboxem a minimem diagnostickych nastroju. Konfigurace je provedena prostrednictvim nastroju weboveho rozhrani modemu, ale ty v podastate pouzivaji standardni "ip" pro vytvoreni GRE tunnelu a pluto jako IKE daemona (ale jako implementaci IPSecu to pouziva kernelovy NETKEY). Aby to nebylo tak jednoduche, tak ma modem v ramci LTE dynamicky pridelovanou neverejnou IP adresu, takze se pouziva jeste NAT-T. Pro registraci GRE endpointu je pouzity OpenNHRP.

Za modemem na jedne strane a za Ciscem na druhe jsou dva pocitace, na nichz pouzivam iperf v UDP rezimu pro mereni propustnosti. Kdyz se posilaji data ze stany Cisca na stranu modemu, tak dosahnu cca 8Mbps, v opacnem smeru mi ale jede prenos pouze cca 500Kbps. Pri komunikaci v ramci Internetu (mimo IPSec tunnel) dosahuji pres modem rychlosti cca 22Mbps download a 17Mbps upload.

Co jsem se snazil vyloucit:

1. Vliv MTU - Jeste kdyz jsem delal pokusy jen pomoci TCP (scp), tak jsem zkousel zmensit primo MTU na rozhrani komunikujicich pocitacu - bez vlivu. Pro pokusy s iperf pouzivam parametr -l 1200, ktery generuje rovnou mensi datagramy.
2. Vliv vykonu CPU na modemu pro sifrovani. Standardne pouzivam AES-256/SHA1, zkousel jsem pouzit i AES-128/MD5 a DES/MD5, ale prakticky bez zmeny propustnosti. CPU je tam takoveto:

   processor	: 0
   model name	: ARMv7 Processor rev 2 (v7l)
   Features	: swp half fastmult vfp edsp neon vfpv3 tls vfpd32 
   CPU implementer	: 0x41
   CPU architecture: 7
   CPU variant	: 0x3
   CPU part	: 0xc08
   CPU revision	: 2
   
   Hardware	: Generic AM33XX (Flattened Device Tree)
   Revision	: 0000
   Serial		: 0000000000000000
   

   Vysledek "openssl speed aes-256-cbc" na modemu je navic radove srovnatelny s mym intel laptopem:

   # openssl speed aes-256-cbc
   Doing aes-256 cbc for 3s on 16 size blocks: 3019223 aes-256 cbc's in 2.98s
   Doing aes-256 cbc for 3s on 64 size blocks: 811793 aes-256 cbc's in 2.98s
   Doing aes-256 cbc for 3s on 256 size blocks: 210625 aes-256 cbc's in 2.98s
   Doing aes-256 cbc for 3s on 1024 size blocks: 53233 aes-256 cbc's in 2.98s
   Doing aes-256 cbc for 3s on 8192 size blocks: 6672 aes-256 cbc's in 2.99s
   OpenSSL 1.0.2h  3 May 2016
   built on: reproducible build, date unspecified
   options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr) 
   compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
   The 'numbers' are in 1000s of bytes per second processed.
   type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
   aes-256 cbc      16210.59k    17434.48k    18093.96k    18292.14k    18279.94k
   

3. Vliv QoS v siti operatora (O2). Z PC za modemem jsem navazal IPSec VPN spojeni na VPN koncentrator v siti hned vedle Cisco routeru. Stejnou cestou a stejnym protokolem jsem dosahl rychlosti cca 12Mbps v obou smerech.
4. Spatne parametry ip stacku pro UDP - spis uz ze zoufalstvi jsem zkousel zvednout velikosti pro /proc/sys/net/ipv4/udp_* a pro /proc/sys/net/core/[rw]mem* ,ale nepozoroval jsem zadny efekt.
5. Problemy na strane Cisco routeru. Nainstaloval jsem si virtualni stroj s Fedorou, ve kterem jsem se pokusil replikovat setup z modemu - GRE+IPSec(NETKEY)+OpenNHRP. Z nej jsem dosahl rychlosti cca 10Mbps v obou smerech. Cisco v tu dobu narazi na svuj CPU limit.
6. Jen tak, mimo soutez, jsem zkousel propustnost samotneho GRE tunelu nba modemu. Nastavil jsem ho pouze mezi modemem a PC pripojenym po LAN. Propustnost byla cca 50Mbps.

gre1      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:xxx.xxx.xxx.xxx  P-t-P:xxx.xxx.xxx.xxx  Mask:255.255.255.255
          UP POINTOPOINT RUNNING  MTU:1472  Metric:1
          RX packets:45 errors:0 dropped:0 overruns:0 frame:0
          TX packets:704 errors:30 dropped:4547 overruns:0 carrier:30
          collisions:0 txqueuelen:0 
          RX bytes:6599 (6.4 KB)  TX bytes:768264 (750.2 KB)
usb0      Link encap:Ethernet  HWaddr DE:AD:BE:EF:00:00  
          inet addr:/modem_ip/  Bcast:100.255.255.255  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1121 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:176854 (172.7 KB)  TX bytes:897120 (876.0 KB)

No.     Time           Source                Destination           Protocol Length Info
    115 0.098272       /modem_ip/         /cisco_ip/          ESP      1326   ESP (SPI=0x1a975dd3)

Frame 115: 1326 bytes on wire (10608 bits), 300 bytes captured (2400 bits)
Ethernet II, Src: de:ad:be:ef:00:00 (de:ad:be:ef:00:00), Dst: 02:50:f3:00:00:00 (02:50:f3:00:00:00)
Internet Protocol Version 4, Src: /modem_ip/ (/modem_ip/), Dst: /cisco_ip/ (/cisco_ip/)
User Datagram Protocol, Src Port: ipsec-nat-t (4500), Dst Port: ipsec-nat-t (4500)
UDP Encapsulation of IPsec Packets
Encapsulating Security Payload
    ESP SPI: 0x1a975dd3 (446127571)
    ESP Sequence: 3807

No.     Time           Source                Destination           Protocol Length Info
    116 0.000393       /modem_ip/         /cisco_ip/          ESP      1326   ESP (SPI=0x1a975dd3)

Frame 116: 1326 bytes on wire (10608 bits), 300 bytes captured (2400 bits)
Ethernet II, Src: de:ad:be:ef:00:00 (de:ad:be:ef:00:00), Dst: 02:50:f3:00:00:00 (02:50:f3:00:00:00)
Internet Protocol Version 4, Src: /modem_ip/ (/modem_ip/), Dst: /cisco_ip/ (/cisco_ip/)
User Datagram Protocol, Src Port: ipsec-nat-t (4500), Dst Port: ipsec-nat-t (4500)
UDP Encapsulation of IPsec Packets
Encapsulating Security Payload
    ESP SPI: 0x1a975dd3 (446127571)
    ESP Sequence: 3808

No.     Time           Source                Destination           Protocol Length Info
    117 0.000181       /modem_ip/         /cisco_ip/          ESP      1326   ESP (SPI=0x1a975dd3)

Frame 117: 1326 bytes on wire (10608 bits), 300 bytes captured (2400 bits)
Ethernet II, Src: de:ad:be:ef:00:00 (de:ad:be:ef:00:00), Dst: 02:50:f3:00:00:00 (02:50:f3:00:00:00)
Internet Protocol Version 4, Src: /modem_ip/ (/modem_ip/), Dst: /cisco_ip/ (/cisco_ip/)
User Datagram Protocol, Src Port: ipsec-nat-t (4500), Dst Port: ipsec-nat-t (4500)
UDP Encapsulation of IPsec Packets
Encapsulating Security Payload
    ESP SPI: 0x1a975dd3 (446127571)
    ESP Sequence: 3899

No.     Time           Source                Destination           Protocol Length Info
    118 0.103109       /modem_ip/         /cisco_ip/          ESP      1326   ESP (SPI=0x1a975dd3)

Frame 118: 1326 bytes on wire (10608 bits), 300 bytes captured (2400 bits)
Ethernet II, Src: de:ad:be:ef:00:00 (de:ad:be:ef:00:00), Dst: 02:50:f3:00:00:00 (02:50:f3:00:00:00)
Internet Protocol Version 4, Src: /modem_ip/ (/modem_ip/), Dst: /cisco_ip/ (/cisco_ip/)
User Datagram Protocol, Src Port: ipsec-nat-t (4500), Dst Port: ipsec-nat-t (4500)
UDP Encapsulation of IPsec Packets
Encapsulating Security Payload
    ESP SPI: 0x1a975dd3 (446127571)
    ESP Sequence: 3905

Podle toho mi to pripada, jakoby se modemu podarilo nejak zasifrovat a pripravit data pro ESP packet, ale problem nastal az pri encapsulaci do UDP a data se mu nakonec nepodarilo odeslat. Nevim ale, kam se dal podivat, co proverit, pripadne nastavit.

Moje google-fu me zradilo, jedine podobne zoufale lidi jsem nasel u mikrotiku, jinak nic

Nemate s tim prosim nekdo zkusenost, nenapada vas neco?

Diky.