[03:11] *** km_ has quit IRC (Ping timeout: 255 seconds) [03:13] *** km_ has joined #arpnetworks [03:58] *** ziyourenxiang has joined #arpnetworks [04:46] *** perlgod has quit IRC (Read error: Connection reset by peer) [04:51] *** perlgod has joined #arpnetworks [07:54] *** KILLALLHUMANS01 has quit IRC (Ping timeout: 264 seconds) [07:55] *** tabthorpe has joined #arpnetworks [07:56] *** KILLALLHUMANS01 has joined #arpnetworks [08:44] *** ziyourenxiang has quit IRC (Ping timeout: 240 seconds) [12:37] *** nznzx2 has joined #arpnetworks [13:48] brycec: yeah that is nice [14:55] *** ziyourenxiang has joined #arpnetworks [15:14] *** ziyourenxiang has quit IRC (Ping timeout: 240 seconds) [15:17] *** reardencode has quit IRC (Read error: Connection reset by peer) [15:17] *** reardencode has joined #arpnetworks [17:52] does anyone know the networking specs for the arp thunder instances? ive been troubleshooting abysmal openvpn performance for the last few hours [17:53] looks like throughput is limited to 10MB/sec up and down [17:53] i managed to get >0.15mbps by disabling all the hardware offload options in freebsd or vtnet0 [17:59] perl: are you using pf? [17:59] freebsd has some bugs with pf and checksum [17:59] yeah, i am using pf [18:00] you might find your speed goes up magically when you disable pf [18:00] well not now that you've disabled the offload probably. [18:00] interesting...but...i love pf :( [18:00] fix the bugs then! [18:00] when i was reading about it it sounded like it was non-trivial to fix [18:00] the weird thing is, over the vpn, my upload speed is great but download is awful [18:01] so one sided issue [18:01] so disabling the offload fixed it? [18:01] i might have some notes somewhere from what i found... [18:02] disabling the offload definitely helped. i get like 2mbps download instead of 0.15mbps [18:02] but, suppoedly i should get (close to) 100mbps [18:02] yeah [18:02] hangon i'll check [18:03] testing this with a combo of iperf and transfering an ubuntu iso back and forth from my thunder instance. [18:03] interestingly, either speedtest.net has banned arpnetworks, or arpnetworks has blacklisted speedtest.net it seems [18:05] are you using ipv4 or ipv6? [18:07] turning csum, gso, tso off on host improves freebsd performance a little [18:07] i believe my openvpn uses ipv4 only [18:07] dont have any issues with normal tcp traffic. [18:08] oh hangon [18:08] are you using normal port [18:08] 1194 [18:08] there might be the additional thing of the udp rate limit [18:08] no good? [18:08] but that's at 5 megabit [18:08] nah that port is exempt [18:08] actually [18:08] http://support.arpnetworks.com/kb/main/is-there-a-firewall-filter-rate-limit-or-similar-device-applied-to-my-traffic [18:08] when i had my metal instance.... [18:08] i remember i had to file a ticket [18:08] and garry was like [18:09] i remember now [18:09] 1194 is supposed to be exempt from udp rate limit, but garry had to disable the rate limit for my whole block bc the exemption didnt work [18:09] was your openvpn performance higher with metal? or weren't you using freebsd? [18:09] maybe that exemption got lost in the ether [18:09] i was using freebsd both times [18:09] it's 5 megabit rather than 2 megabit though [18:10] i got ~80mbps once garry disabled the rate limit for my whole IP block [18:10] h/o let me run a speed test [18:10] getting around 3.5 mbps down, 20 mbps up [18:11] you could try iperf -b 5m -c -P with iperf -s -i 1 -u on your home ip [18:11] does the rate limit only apply to outbound traffic? [18:11] well with a port forwrad too probably [18:11] yeah [18:11] that seems like the issue then [18:12] the missing 1.5 mbps is probably overhead [18:12] but anyway, pf + vnet0 + hw offloading = problems ? [18:13] yeah [18:13] but i think there's a separate hw offload oddity without pf too [18:14] are you using -tso -txcsum -rxcsum? [18:14] yes [18:14] ok so [18:15] iperf3 has a -R option, so i didnt need to do port forward [18:15] oh cool [18:15] definitely hitting 5mbps rate limit [18:15] let me manually set port to 1194 and try again [18:15] https://www.freebsd.org/security/advisories/FreeBSD-EN-16:02.pf.asc [18:16] that appears that some of the pf checksum stuff was fixed [18:16] pf seems not very well maintained [18:16] on freebsd. [18:16] but i noticed this after that... [18:16] it's not just that it's badly maintained. it was forked, with some very odd changes that make it hard to integrate upstream changes [18:17] so they've stayed at an old version [18:17] now they've added their own SMP work etc in [18:17] creating even more complications [18:17] the whole freebsd network side is a bit iffy. there's memory fragmentation issues with large mtus too [18:18] ok, i ran iperf over port 1194 udp, still hitting the 5mbps rate limit [18:18] so it must be the same issue i had 2 years ago [18:18] 1194 as source? [18:18] yeah [18:18] [SUM] 0.00-10.00 sec 5.96 MBytes 5.00 Mbits/sec 3.508 ms 4086/8404 (45%) receiver [18:18] and fine with 4megabit? [18:19] couldnt parse that last question [18:19] if you go down to 4 megabit for sending speed does it work fine [18:19] ah, one sec [18:21] i can only hit the 5mbps rate limit by telling iperf to use multiple threads [18:22] wait nvm, bad copypasta from stackexchange [18:22] yeah, no packet loss if i limit to 4mbps [18:23] sounds like it's that then [18:23] iperf3 -c beastie.c0ffee.net -u -R -p 1194 -b 4m [18:23] send a ticket in [18:23] [ 7] 0.00-10.00 sec 4.82 MBytes 4.04 Mbits/sec 0.361 ms 0/3492 (0%) receiver [18:23] will do [18:24] mercutio - worth it to learn IPFW ? ive been using pf for years but if it cant even checksum properly it might be time to switch [18:24] i think the netbsd one is ok [18:24] but i haven't looked into it much, i am used to pf [18:25] yeah, the syntax is so good. [18:25] FreeBSD does not use Linux® IPTABLES for its firewall. Instead, FreeBSD offers a choice of three kernel level firewalls: [18:25] just to confuse you :) [18:25] is ipfilter the netbsd one? [18:26] seems not [18:27] hmm [18:27] https://en.wikipedia.org/wiki/NPF_(firewall) [18:27] NPF (firewall) :: NPF is a BSD licensed stateful packet filter, a central piece of software for firewalling. It is comparable to iptables, ipfw, ipfilter and PF. NPF is developed on NetBSD. History NPF was primarily written by Mindaugas Rasiukevicius. NPF first appeared in the NetBSD 6.0 release in 2012. Features NPF is designed for high performance on SMP systems and for easy extensibility. It supports various forms of Network... [18:28] it seems there was talk of importing npf but it never happened [18:30] i'm actually not sure why netbsd isn't more popular. it has a lot of benefits over freebsd... [18:33] dont send me down the rabbit hole of learning another bsd distro [18:34] heh [18:34] But if you are going to learn another BSD, I recommend OpenBSD! You might like their packet filter software ... [18:35] ive toyed around with it. the ACPI support is a lot better for laptops etc [18:36] installed TrueOS on an old thinkpad i have. their lumina desktop is pretty nice, cool to have a BSD-first DE [18:36] also crazy that its just one dude hacking away at it [18:37] enlightenment had one person hacking on it, then other people joined in and development slowed [18:37] sometimes it's easier to make progress on your own [23:37] hell is other people