brycec: yeah that is nice does anyone know the networking specs for the arp thunder instances? ive been troubleshooting abysmal openvpn performance for the last few hours looks like throughput is limited to 10MB/sec up and down i managed to get >0.15mbps by disabling all the hardware offload options in freebsd or vtnet0 perl: are you using pf? freebsd has some bugs with pf and checksum yeah, i am using pf you might find your speed goes up magically when you disable pf well not now that you've disabled the offload probably. interesting...but...i love pf :( fix the bugs then! when i was reading about it it sounded like it was non-trivial to fix the weird thing is, over the vpn, my upload speed is great but download is awful so one sided issue so disabling the offload fixed it? i might have some notes somewhere from what i found... disabling the offload definitely helped. i get like 2mbps download instead of 0.15mbps but, suppoedly i should get (close to) 100mbps yeah hangon i'll check testing this with a combo of iperf and transfering an ubuntu iso back and forth from my thunder instance. interestingly, either speedtest.net has banned arpnetworks, or arpnetworks has blacklisted speedtest.net it seems are you using ipv4 or ipv6? turning csum, gso, tso off on host improves freebsd performance a little i believe my openvpn uses ipv4 only dont have any issues with normal tcp traffic. oh hangon are you using normal port 1194 there might be the additional thing of the udp rate limit no good? but that's at 5 megabit nah that port is exempt actually http://support.arpnetworks.com/kb/main/is-there-a-firewall-filter-rate-limit-or-similar-device-applied-to-my-traffic when i had my metal instance.... i remember i had to file a ticket and garry was like i remember now 1194 is supposed to be exempt from udp rate limit, but garry had to disable the rate limit for my whole block bc the exemption didnt work was your openvpn performance higher with metal? or weren't you using freebsd? maybe that exemption got lost in the ether i was using freebsd both times it's 5 megabit rather than 2 megabit though i got ~80mbps once garry disabled the rate limit for my whole IP block h/o let me run a speed test getting around 3.5 mbps down, 20 mbps up you could try iperf -b 5m -c -P with iperf -s -i 1 -u on your home ip does the rate limit only apply to outbound traffic? well with a port forwrad too probably yeah that seems like the issue then the missing 1.5 mbps is probably overhead but anyway, pf + vnet0 + hw offloading = problems ? yeah but i think there's a separate hw offload oddity without pf too are you using -tso -txcsum -rxcsum? yes ok so iperf3 has a -R option, so i didnt need to do port forward oh cool definitely hitting 5mbps rate limit let me manually set port to 1194 and try again https://www.freebsd.org/security/advisories/FreeBSD-EN-16:02.pf.asc that appears that some of the pf checksum stuff was fixed pf seems not very well maintained on freebsd. but i noticed this after that... it's not just that it's badly maintained. it was forked, with some very odd changes that make it hard to integrate upstream changes so they've stayed at an old version now they've added their own SMP work etc in creating even more complications the whole freebsd network side is a bit iffy. there's memory fragmentation issues with large mtus too ok, i ran iperf over port 1194 udp, still hitting the 5mbps rate limit so it must be the same issue i had 2 years ago 1194 as source? yeah [SUM] 0.00-10.00 sec 5.96 MBytes 5.00 Mbits/sec 3.508 ms 4086/8404 (45%) receiver and fine with 4megabit? couldnt parse that last question if you go down to 4 megabit for sending speed does it work fine ah, one sec i can only hit the 5mbps rate limit by telling iperf to use multiple threads wait nvm, bad copypasta from stackexchange yeah, no packet loss if i limit to 4mbps sounds like it's that then iperf3 -c beastie.c0ffee.net -u -R -p 1194 -b 4m send a ticket in [ 7] 0.00-10.00 sec 4.82 MBytes 4.04 Mbits/sec 0.361 ms 0/3492 (0%) receiver will do mercutio - worth it to learn IPFW ? ive been using pf for years but if it cant even checksum properly it might be time to switch i think the netbsd one is ok but i haven't looked into it much, i am used to pf yeah, the syntax is so good. FreeBSD does not use Linux® IPTABLES for its firewall. Instead, FreeBSD offers a choice of three kernel level firewalls: just to confuse you :) is ipfilter the netbsd one? seems not hmm https://en.wikipedia.org/wiki/NPF_(firewall) NPF (firewall) :: NPF is a BSD licensed stateful packet filter, a central piece of software for firewalling. It is comparable to iptables, ipfw, ipfilter and PF. NPF is developed on NetBSD. History NPF was primarily written by Mindaugas Rasiukevicius. NPF first appeared in the NetBSD 6.0 release in 2012. Features NPF is designed for high performance on SMP systems and for easy extensibility. It supports various forms of Network... it seems there was talk of importing npf but it never happened i'm actually not sure why netbsd isn't more popular. it has a lot of benefits over freebsd... dont send me down the rabbit hole of learning another bsd distro heh But if you are going to learn another BSD, I recommend OpenBSD! You might like their packet filter software ... ive toyed around with it. the ACPI support is a lot better for laptops etc installed TrueOS on an old thinkpad i have. their lumina desktop is pretty nice, cool to have a BSD-first DE also crazy that its just one dude hacking away at it enlightenment had one person hacking on it, then other people joined in and development slowed sometimes it's easier to make progress on your own hell is other people