***: km_ has joined #arpnetworks
ziyourenxiang has joined #arpnetworks
perlgod has quit IRC (Read error: Connection reset by peer)
perlgod has joined #arpnetworks
KILLALLHUMANS01 has quit IRC (Ping timeout: 264 seconds)
tabthorpe has joined #arpnetworks
KILLALLHUMANS01 has joined #arpnetworks
ziyourenxiang has quit IRC (Ping timeout: 240 seconds)
nznzx2 has joined #arpnetworks up_the_irons: brycec: yeah that is nice ***: ziyourenxiang has joined #arpnetworks
ziyourenxiang has quit IRC (Ping timeout: 240 seconds)
reardencode has quit IRC (Read error: Connection reset by peer)
reardencode has joined #arpnetworks perlgod: does anyone know the networking specs for the arp thunder instances? ive been troubleshooting abysmal openvpn performance for the last few hours
looks like throughput is limited to 10MB/sec up and down
i managed to get >0.15mbps by disabling all the hardware offload options in freebsd or vtnet0 mercutio: perl: are you using pf?
freebsd has some bugs with pf and checksum perlgod: yeah, i am using pf mercutio: you might find your speed goes up magically when you disable pf
well not now that you've disabled the offload probably. perlgod: interesting...but...i love pf :( mercutio: fix the bugs then!
when i was reading about it it sounded like it was non-trivial to fix perlgod: the weird thing is, over the vpn, my upload speed is great but download is awful mercutio: so one sided issue
so disabling the offload fixed it?
i might have some notes somewhere from what i found... perlgod: disabling the offload definitely helped. i get like 2mbps download instead of 0.15mbps
but, suppoedly i should get (close to) 100mbps mercutio: yeah
hangon i'll check perlgod: testing this with a combo of iperf and transfering an ubuntu iso back and forth from my thunder instance.
interestingly, either speedtest.net has banned arpnetworks, or arpnetworks has blacklisted speedtest.net it seems mercutio: are you using ipv4 or ipv6?
turning csum, gso, tso off on host improves freebsd performance a little perlgod: i believe my openvpn uses ipv4 only
dont have any issues with normal tcp traffic. mercutio: oh hangon
are you using normal port perlgod: 1194 mercutio: there might be the additional thing of the udp rate limit perlgod: no good? mercutio: but that's at 5 megabit
nah that port is exempt perlgod: actually mercutio: http://support.arpnetworks.com/kb/main/is-there-a-firewall-filter-rate-limit-or-similar-device-applied-to-my-traffic perlgod: when i had my metal instance....
i remember i had to file a ticket
and garry was like
i remember now
1194 is supposed to be exempt from udp rate limit, but garry had to disable the rate limit for my whole block bc the exemption didnt work mercutio: was your openvpn performance higher with metal? or weren't you using freebsd? perlgod: maybe that exemption got lost in the ether
i was using freebsd both times mercutio: it's 5 megabit rather than 2 megabit though perlgod: i got ~80mbps once garry disabled the rate limit for my whole IP block
h/o let me run a speed test
getting around 3.5 mbps down, 20 mbps up mercutio: you could try iperf -b 5m -c <your home ip> -P with iperf -s -i 1 -u on your home ip perlgod: does the rate limit only apply to outbound traffic? mercutio: well with a port forwrad too probably
yeah perlgod: that seems like the issue then
the missing 1.5 mbps is probably overhead
but anyway, pf + vnet0 + hw offloading = problems ? mercutio: yeah
but i think there's a separate hw offload oddity without pf too
are you using -tso -txcsum -rxcsum? perlgod: yes
ok so
iperf3 has a -R option, so i didnt need to do port forward mercutio: oh cool perlgod: definitely hitting 5mbps rate limit
let me manually set port to 1194 and try again mercutio: https://www.freebsd.org/security/advisories/FreeBSD-EN-16:02.pf.asc
that appears that some of the pf checksum stuff was fixed perlgod: pf seems not very well maintained
on freebsd. mercutio: but i noticed this after that...
it's not just that it's badly maintained. it was forked, with some very odd changes that make it hard to integrate upstream changes
so they've stayed at an old version
now they've added their own SMP work etc in
creating even more complications
the whole freebsd network side is a bit iffy. there's memory fragmentation issues with large mtus too perlgod: ok, i ran iperf over port 1194 udp, still hitting the 5mbps rate limit
so it must be the same issue i had 2 years ago mercutio: 1194 as source? perlgod: yeah
[SUM] 0.00-10.00 sec 5.96 MBytes 5.00 Mbits/sec 3.508 ms 4086/8404 (45%) receiver mercutio: and fine with 4megabit? perlgod: couldnt parse that last question mercutio: if you go down to 4 megabit for sending speed does it work fine perlgod: ah, one sec
i can only hit the 5mbps rate limit by telling iperf to use multiple threads
wait nvm, bad copypasta from stackexchange
yeah, no packet loss if i limit to 4mbps mercutio: sounds like it's that then perlgod: iperf3 -c beastie.c0ffee.net -u -R -p 1194 -b 4m mercutio: send a ticket in perlgod: [ 7] 0.00-10.00 sec 4.82 MBytes 4.04 Mbits/sec 0.361 ms 0/3492 (0%) receiver
will do
mercutio - worth it to learn IPFW ? ive been using pf for years but if it cant even checksum properly it might be time to switch mercutio: i think the netbsd one is ok
but i haven't looked into it much, i am used to pf perlgod: yeah, the syntax is so good. mercutio: FreeBSD does not use Linux® IPTABLES for its firewall. Instead, FreeBSD offers a choice of three kernel level firewalls:
just to confuse you :)
is ipfilter the netbsd one?
seems not
hmm
https://en.wikipedia.org/wiki/NPF_(firewall) BryceBot: NPF (firewall) :: NPF is a BSD licensed stateful packet filter, a central piece of software for firewalling. It is comparable to iptables, ipfw, ipfilter and PF. NPF is developed on NetBSD. History NPF was primarily written by Mindaugas Rasiukevicius. NPF first appeared in the NetBSD 6.0 release in 2012. Features NPF is designed for high performance on SMP systems and for easy extensibility. It supports various forms of Network... mercutio: it seems there was talk of importing npf but it never happened
i'm actually not sure why netbsd isn't more popular. it has a lot of benefits over freebsd... perlgod: dont send me down the rabbit hole of learning another bsd distro mercutio: heh mike-burns: But if you are going to learn another BSD, I recommend OpenBSD! You might like their packet filter software ... perlgod: ive toyed around with it. the ACPI support is a lot better for laptops etc
installed TrueOS on an old thinkpad i have. their lumina desktop is pretty nice, cool to have a BSD-first DE
also crazy that its just one dude hacking away at it mercutio: enlightenment had one person hacking on it, then other people joined in and development slowed
sometimes it's easier to make progress on your own dne: hell is other people