[00:10] *** gizmoguy has quit IRC (Ping timeout: 245 seconds) [00:11] *** gizmoguy has joined #arpnetworks [00:12] *** Hien has quit IRC (Ping timeout: 264 seconds) [00:13] *** solj has quit IRC (Ping timeout: 264 seconds) [00:15] *** phlux has quit IRC (Ping timeout: 264 seconds) [00:15] *** staticsafe-znc has quit IRC (Ping timeout: 264 seconds) [00:17] *** SpeedBus has quit IRC (Ping timeout: 245 seconds) [00:17] *** staticsafe-znc has joined #arpnetworks [00:18] *** Hien has joined #arpnetworks [00:19] *** gizmoguy has quit IRC (Ping timeout: 264 seconds) [00:19] *** gizmoguy has joined #arpnetworks [00:19] *** solj has joined #arpnetworks [00:20] *** SpeedBus has joined #arpnetworks [00:23] *** phlux has joined #arpnetworks [00:25] *** SpeedBus has quit IRC (Ping timeout: 245 seconds) [00:27] *** SpeedBus has joined #arpnetworks [00:46] *** robonerd has quit IRC (Read error: Connection reset by peer) [00:47] *** robonerd has joined #arpnetworks [00:51] *** robonerd has quit IRC (Read error: Connection reset by peer) [00:52] *** robonerd has joined #arpnetworks [01:12] *** LT has joined #arpnetworks [01:48] *** robonerd has quit IRC (Read error: Connection reset by peer) [01:49] *** robonerd has joined #arpnetworks [03:55] *** staticsafe-znc has quit IRC (Ping timeout: 265 seconds) [03:56] *** staticsafe has quit IRC (Ping timeout: 265 seconds) [03:57] *** xales has quit IRC (Ping timeout: 246 seconds) [03:59] *** staticsafe has joined #arpnetworks [04:00] *** staticsafe-znc has joined #arpnetworks [04:51] mnathani: no i mean VPS customers [06:45] i tried it in a vps once, it works fine - alias configuration is a PITA vs freebsd or openbsd though [07:27] *** dj_goku_ has quit IRC (Ping timeout: 248 seconds) [07:42] *** mjp_ has quit IRC (Ping timeout: 265 seconds) [07:43] *** abthorpet has joined #arpnetworks [07:43] *** mhoran1 has joined #arpnetworks [07:43] *** ChanServ sets mode: +o mhoran1 [07:44] *** [FBI] has quit IRC (Ping timeout: 265 seconds) [07:44] *** [FBI] starts logging #arpnetworks at Thu Feb 13 07:44:23 2014 [07:44] *** [FBI] has joined #arpnetworks [07:44] *** mhoran has quit IRC (Ping timeout: 265 seconds) [07:45] *** tabthorpe has quit IRC (Ping timeout: 265 seconds) [08:54] What do you mean? Clicking buttons in a GUI versus just typing it in pf.conf? 06:43:51 < m0unds> i tried it in a vps once, it works fine - alias configuration is a PITA vs freebsd or openbsd though [08:54] In which case, I'd say that writing rules is similarly PITA :p [09:01] brycec: no, when i messed with it somewhat recently, interface aliases don't work the way i expected them to (the way they work in freebsd or openbsd) [09:02] Do you mean "interface groups" as they're termed in pfSense? [09:02] the UI element does some weird stuff with aliases that wasn't clear [09:02] i don't remember, but it felt kinda counterproductive [09:02] at any rate, i only looked because i hadn't used it since like 2007 [09:02] heh [09:03] Well I'm happy to say pfSense has come a long, long way in 7 years :p [09:03] and the thing that seemed like it should be alises wasn't [09:03] well, they're still on teh same release tree and the ui has some awful 90s UX to it that they need to get rid of asap :P [09:03] the notification thing at the top in teh default theme is awful [09:03] s/teh/the [09:03] the notification thing at the top in the default theme is awful [09:04] i looked at m0n0 last night because i hadn't used it since probably 2003, and they just moved up to freebsd 8.3 last month, haha [09:04] I don't think I follow what you mean by same release tree - were you expecting them to just scrap everything and start over? [09:04] Nice, they're catching up to pfSense [09:04] no, they were working on "2.0" forever [09:04] IIRC that 8.3 is due to the NanoBSD base. [09:05] True, but 1.2.3 was stable up until ~2 years ago [09:05] *stable/current [09:05] That 2.0 was effectively a total rewrite [09:06] yeah, so i guess it was 2 years ago that i messed with it, haha [09:06] because it was 1.2.3 (i still have the disk image on my kvm server at home) [09:07] "Version 1.0 of the software was released on October 4, 2006.[5] Version 2.0 was released on September 17, 2011,[6] with updates 2.0.1 to 2.0.3 between then and 2013, and version 2.1 was released on September 15, 2013." [09:08] Or you can peruse http://www.pfsense.org/about-pfsense/versions.html [09:09] 1.2.3 released Dec 2009, 2.0.1 released Dec 2011 [09:09] at least they're somewhat consistent [09:17] Good news, up_the_irons, ARP was not on the list of the top 24 networks http://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack [09:18] It didn't make the list at all! https://docs.google.com/spreadsheet/ccc?key=0AhuvvqAkGlindHFtS0pJa0lYZGNlLXNONWtlY01qanc&usp=sharing#gid=0 [09:30] actually, i did try it more recently - i was trying to figure out how i knew about the thing at the top. i tried it on my small arp vps after i migrated stuff to the bigger arp vps [09:30] i do have a 1.2.3 disk image locally though [09:30] which i don't remember doing anything with [09:31] at any rate, maybe it was a quirk with the vps having a single if or something, or i could just be remembering something that doesn't exist, hahaha [09:32] haha, chinanet. go figure. [09:33] srsly [09:33] and ovh [09:33] yeah [09:33] color me surprised [09:33] * brycec colours in m0unds [09:33] what color is surprised? [09:34] *** LT has quit IRC (Quit: Leaving) [09:35] m0unds: mauve [09:35] eeewwwww [09:35] at least it's not taupe [09:35] or chartreuse [09:35] Oh sweet, I'm getting hit with buttloads of DNS queries... [09:35] * m0unds throws DNS queries at brycec [09:36] I'm guessing my own DOS reflection [09:36] an attack of my very own. On a cable modem. woo [09:36] sweet [09:36] I wonder how this IP was choosen... I don't really use it for anything, and it's not in DNS anywhere [09:37] are you connected to IRC w/it? [09:37] or bot connected from it or something? [09:37] the only time i ever had a box get ddosed was when i was connected to a provider's IRC channel w/v4 address and some kid decided to start ddosing people connected to the channel [09:37] nope and nope [09:38] It's the external IP of a router, but all traffic goes out a different IP [09:38] huh. [09:38] is the address SWIP'd to another business or something? [09:38] dun think so [09:39] you must just be lucky [09:39] holy crap, TWC actually filled in the business information [09:41] (protip: always smart to whois your own addresses once in awhile) [09:42] ANY? . [09:42] that's the query [09:42] whee [09:42] * brycec wonders who to complain at [09:45] this is probably odd question but does arp ratelimit dns or anything [09:45] UDP inbound as I recall [09:45] to 5mbps [09:46] $ time host lightning.net >/dev/null [09:46] real 0m8.197s [09:46] $ time host nac.net >/dev/null [09:46] real 0m2.327s [09:46] ouch [09:46] real 0m0.946s [09:46] using the official arp resolvers, and i don't get this kind of response time elsewhere even hitting the same authoritative ns [09:46] close to 1 second seems incredibly weird/bad as is [09:47] hazardous: for google.com real 0m0.161s [09:47] yeah that seems cached maybe, idk [09:47] And nac.net now gives me real 0m0.027s [09:47] i sometimes have periods of time when it's somewhat acceptable and periods of time where anything takes forever [09:48] http://support.arpnetworks.com/kb/main/is-there-a-firewall-filter-rate-limit-or-similar-device-applied-to-my-traffic [09:48] outbound UDP traffic is rate-limited to 5mbps [09:48] I wonder if up_the_irons / ARP's resolvers are exempt from that, and/or being overwhelming it [09:49] wow that was terrible grammar [09:49] try again [09:49] hahah [09:49] yeah i dunno, i just did host google.com and the A records returned instantly [09:49] then it hung for 3-4 seconds before returning the mx part [09:50] huh. [09:50] was just gonna say maybe it wasn't cached but if it's hanging on mx recs, dunno [09:50] the prev query, i mean [09:50] since it returned google quickly [09:52] yeah i have absolutely no idea [09:52] http://pastebin.aquilenet.fr/?1328915b0bfb2488#y8PHFMpIH159tKAd5vFItNbggiBNrLCLq4fJmGtx7oE= [09:52] wat [10:05] oh man this dns ddos is terrible, how will i deal with the 50kbps of traffic pouring in??? [10:07] Silliest. DOS. Ever. [10:33] omgz, you better mitigate it [10:57] brycec: ARP wasn't in the list ;) [11:01] What is the purpose of running pfSense (a firewall) on a VPS with only one interface? [11:02] * m0unds shrugs [11:03] i did it to screw around; you can still use it as a gateway or firewall in front of another vps if you really wanted to [11:03] you don't /have/ to have separate interfaces unless you're gonna nat [11:16] And even then, there are vlans ;) (or vlans inside of vlans, on ARP) [11:16] yes [12:03] vlans inside of vlans on ARP only work if you use svlan(4) (IEEE 802.1AD) [12:04] and then only if you're unblocked and permitted to do so [12:04] I know from experience! [12:04] now don't try to do svlan(4) on vio(4) on current, something about cksums kills it [12:04] current openbsd that is [12:53] Does anyone know how much it costs to Akamaize a website? (Serve it using Akaamai's CDN) ? [12:58] i've never seen $ amounts, but i've also never heard 'affordable' mentioned alongside their name [12:59] might look into edgecast or cachefly too [12:59] Ofcourse it would depend on the size of the site / complexity beign served as well as actual traffic / bandwidth [13:00] s/beign/being [13:00] Ofcourse it would depend on the size of the site / complexity being served as well as actual traffic / bandwidth [13:01] i googled and found pricing from an akamai partner showing a $4/GB x 500GB/mo commit [13:01] and that was 'cheap' [13:02] so approx $2000/month for that setup? [13:02] yea [13:03] later clarification showed $200/mo @ per GB [13:03] and $375/mo for 500GB [13:04] http://www.cachefly.com/pricing.html [13:11] m0unds: thans [13:11] s/thans/thanks [13:11] m0unds: thanks [13:15] cloudflare.com is free if you don't need SSL [13:21] yeah, cloudflare [13:21] 's free stuff is good if you just want the cdn features [13:22] indeed it does sound good [14:02] i'm not sure what i like better [14:02] allow(Mail).to receive(:all).and_return(@msgs) [14:02] allow(Mail).to receive(:all) { @msgs } [14:03] http://www.pantz.org/software/pf/use_freebsd_10_as_a_pf_firewall.html [14:05] I like the #and_return. It's more clear what it's doing. [14:06] m0unds: i prefer cahefly to akamai as a user [14:06] I save the block syntax for when I really need a block. [14:06] akamai is often terrible performance with "cache misses" [14:06] akamai has closer nodes to me than cachefly, but cachefly's average performance is way up. [14:07] mike-burns: ah [14:07] it's harder to test cache miss performance though. you used to be able to send ?1 ?2 ?3 ?4 etc to get an uncached version of stuff to test... but that seemed to stop working [14:08] and with the number of akamai nodes, unless you're huge there are likely to be lots of cache misses [14:12] cloudflrae is terrible i reckon [14:14] CloudFlare should get on Any2 so I can peer with them [14:14] Akamai is, so is EdgeCast [14:14] i don't think it'd make meuch difference? [14:15] cloud flare is in san jose atm isn't it? [14:15] i can't think of any domains that use it off hand [14:15] NO, peering with ARP makes ALL THE DIFFERENCE IN THE WORLD. Get it right mercutio ;) [14:15] well i doubt there's much traffic being pushed to there, and with multiple upstreams ... incoming ins't likely to saturate [14:16] and hardly any users on arp are likely to be pulling large files off cloudflare [14:16] but cloudflare pulls lots of files from arp [14:16] oh [14:16] ok i didn't realise that [14:16] :) [14:17] some dedi customers use CF heavily [14:17] mercutio - anyone hosted at (mt) is using cloudflare by default [14:17] yeah, shared hosting co's are starting to do that [14:17] we have some here [14:18] mt? [14:19] media temple [14:21] (mt)® is Media Temple. Not sure what "mt" is. :) [14:22] me either. But I'm pretty sure a RandalSchwartz is a smartass :P [14:22] i've only noticed cloudflare when sites have problems [14:22] i suppose that's one of the problems with those things [14:22] ditto [14:22] some sites only shift to cloudflare when they're getting ddos'ed [14:23] or have load issues [14:23] and if a site's going slow and you trace and it says cloudflare.. [14:23] insightcruises.com had it enabled for a week, then they screwed up the DNS, and different people were getting different pages or even A records pointing to nowhere [14:24] O_o [14:24] so we ripped it all out, and haven't gone back [14:24] might have been early growing pains [14:24] who's using it right now? [14:25] presume these guys: https://www.cloudflare.com/case-studies [14:25] RandalSchwartz: you coming to any js.la meetups soon? it's kinda hard to believe the last one where we had a chance to chat was over a year ago! (christmas before last) [14:25] imgur will do [14:26] RandalSchwartz: or perhaps that was devops.la... hmmm [14:26] *ladevops [14:26] interesting - http://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack [14:26] * up_the_irons brain is dying [14:26] RandalSchwartz: that was pasted earlier [14:27] my client before this one was directly across the street from my hotel and a bar. I stopped going to meetups. :) [14:27] now I'm working in santa monica, and I have to actually drive around, so I can start doing meetups again. [14:27] plus, it'll give me an excuse to drink a bit less. I was a bit out of control because I wasn't driving for 5 days at a time. [14:27] o_o [14:28] oh cloudflare do have node in los angeles [14:28] they would be stupid not to [14:28] and the ?1 ?2 trick works [14:30] they've got lots of pops [14:30] cache hit time in la, 20 msec, cache miss time 500 msec [14:31] cache hit time in nz, 85 msec, cache miss time 530 msec [14:31] not bad [14:31] pretty consistent anyways [14:31] yeh i seen way worse from akamai [14:31] with > 1 second. [14:31] gross [14:31] it's only 15k file though [14:32] it seemed to get more consistent when i did it more [14:32] my friend put their blog on the free cloudflare plan and saw spam comments just stop altogether - i think just as a side effect of cloudflare's anti-bot stuff [14:32] imgur still loaded images slow [14:32] main imgur.com site isn't cdn'ed, jsut their images [14:33] right [14:33] img.imgur.com [14:33] nah i.imgur.com [14:33] img.imgur.com is too [14:33] oh ok [14:33] imgur is quite reliable [14:34] imgur++ [14:34] i don't normaly look at it [14:34] http://imgur.com/gallery/oxn6ZMh [14:34] but this is funny [14:34] heh [14:37] damn now i want to test akamai [14:38] but they using https [14:42] akamai is 56 msec hit, 1 second miss for apple 56k image [14:43] and 17 msec hit, 260 msec miss from la [14:43] i suppose that not so bad, i suppose it depends wehre origin servers are [14:44] still even for 56k 1 second miss seems kind of bad to me [14:53] Just setup my first site to site vpn in a lab, but I can't seem to ping from a subnet behind the vpn to a subnet on the other site. [14:53] did you enable forwarding? [14:53] The reverse ping works just fine. (pfSense with Openvpn btw) [14:53] Did you set the routing? [14:54] Non-overlapping subnets? [14:54] Firewall? [14:54] firewall - allow any/any [14:54] non-overlapping subnets yup [14:54] traceroute, tcpdump [14:54] the server config section had local subnets and remote subnets [14:54] i assume pfsense would enable forwarding :) [14:54] client only had remote subnets [14:55] client side can ping server side, but not reverse [14:59] how foolish of me, twice in as many days now, Windows Firewall got the better of me and dropped my icmp packets [15:00] oh that's normal [15:03] lolol [15:03] now I need to redo it to verify if all that config I did was necessary [15:04] @mnathani [15:04] Have you checked whether Windows Firewall is enabled and dropping packets? [15:04] adding extra route statement etc [15:04] 425 results found. Here's #265 Dec 19, 2013 16:43:30 requests are going through, but not being cached I dont think [15:05] lol BryceBot / brycec [15:05] You know what they say, if you use something 3 times, you sohuld buy it/script it. [15:05] Well, this was only twice but I'm proactive ;) [15:10] Ugh - had some beans and rice (normally skipped) with my mexican dish for lunch [15:10] now I'm in a carb crash fog :( [15:10] forgotten how bad these are [15:11] RandalSchwartz: yeah they suck [15:11] yeah, rice tends to knock me on my ass too [15:13] i think i'm gonna adjust my pf ruleset a bit - i got lazy and didn't define macros or anything so the rules are ugly as hell [15:15] when done.. var=..\nvar=..\n\ntable..\ntable..\n\nmatch..\nmatch..\n\nblock..\npass..\n done! ;-) [15:19] y'll are lazy [15:19] oh i don't quite know how the y thing works [15:20] i don't use macros with pf either [15:20] y'all are lazy' [15:20] well you get the idea I think... You need 2 or 3 separators [15:20] it's a regex like any other [15:21] oh [15:22] ugh - this office has horrible "free" stuff in the kitchen. either sweetened beverages ("vitamin water", ugh), or decaffinated teas. [15:22] and all sorts of "hearthealthywholegrain" snacks [15:22] it's sad how much I've found out that this stuff is all crap in the past 18 months [15:23] (well not exactly a regex... it's a PCRE verb, with sed expression syntax) [15:23] PCRE is a misnomer :) [15:26] i like macros because it keeps things readable - granted, i'm likely the only one who will ever see it, it's still nice to keep it concise [15:26] i do the same w/network hw configs on gear that actually supports macro type functionality (junos) [15:39] *** m0unds has quit IRC (Quit: reloading) [15:40] hmm cloudflare aren't on coresite [15:41] *** mnathani_web has joined #arpnetworks [15:41] so best hope is for bgp collective to spread to equinix [15:42] are there network issues atm [15:42] I cant ping arpnetworks.com or my VPS [15:42] weird [15:42] traceroute? [15:43] sec. [15:43] I cannot ping arpnetworks.com [15:43] http://pastebin.com/wZehQBW7 [15:43] here is mtr [15:43] I can ping 2.v.freedaemon.com (which is on the 1gbps ports) [15:44] blame nlayer? [15:44] traceroute is taking a while [15:44] i can't resolve arpnetworks.com, weird [15:44] oh wait yes i can [15:44] 208.79.89.243 [15:44] i'm on arpnewtorks fine if anyone wants a traceroute in other direciton [15:44] vl5.s1.lax.arpnetworks.com is giving packet loss [15:44] arpnetworks v6 is working, arpnetworks v4 is not for ns1 and ns2.arpnetworks.com [15:45] ah [15:45] just started responding [15:45] ge0-arpnet.cust.lax07.mzima.net gives packet lsos too [15:45] i wonder what happend [15:46] its back now [15:46] oh [15:46] i never lost connectivity :) [15:47] lsos! [15:48] Also showing IPv4 loss from INSIDE ARP https://smokeping.cobryce.com/?target=ARP.ThisGW [15:48] more accurately, packet loss on my own vlan to the router [15:49] guessing router crash? [15:49] nice [15:49] but not for me [15:49] are you on gigabit? [15:49] Don't think so? [15:49] could it be BIRD related? [15:49] i'm not pinging the link address [15:49] don't think so [15:50] hardly anyone goes over any2ix on the new range [15:50] i think it's proably nlayer [15:50] fwiw I can ping arpnetworks.com and my VPS both. As far as I can tell from smokeping, there _was_ a brief outage but it's back up now. [15:50] actually why would link address stop working then [15:51] only likely cause, something unplugged or reset [15:53] wow you have a lot of sites in smokeping :) [15:54] About 250 probes [15:54] i like the overlayed thing [15:54] how much bandwidth does that use? [15:54] 212 hosts to be precise [15:55] overlayed... When there's a whole bunch of hosts on one graph? like for the IRC networks? [15:55] yeah [15:55] * brycec asks graphs.arpnetworks.com how much b/w is used [15:56] A surprising amount. [15:56] Reportedly 6.71GB in the last 24 horus [15:56] 20 ping every 60 seconds [15:56] all that ICMP [15:56] 212 sites [15:57] mercutio: 500byte packet size too [15:57] brycec: would you mind sharing your hosts file for smokeping? [15:57] ahy are you doing 500 byte packet size [15:57] i was going to ask that but he has some local stuff in it too [15:57] mercutio: Because it's better than the default 5000 :P [15:57] frguly isn't 500 [15:58] err 5000 it's like 72 [15:58] According to the docs it's 5000 [15:58] where [15:58] http://oss.oetiker.ch/smokeping/doc/smokeping_config.en.html [15:58] oh wtf [15:58] that's example value [15:58] which isnt' the default, but why are they suggesting that [15:58] er http://oss.oetiker.ch/smokeping/probe/FPing.en.html [15:59] i'd suggest trying 32 [15:59] and see if graphs look bsaically the same [15:59] You have in the past :P [15:59] and identify issues just the same [15:59] And if the docs don't say "default" but do say "example" what am I to assume? [15:59] well i thought i'd do it again [15:59] ikr [16:00] They aren't the world's best docs :/ [16:00] Change made [16:00] just in time for 4pm on the dot [16:02] hmm.. up_the_irons have you considered making graphs of transit links visible to users? [16:02] Ah, 56 is the default. RTFS :/ [16:02] hmm [16:02] i wonder why i saw 72 then [16:02] oih maybe it was 76 [16:02] i htink 56 doesn't include the IP header size of 20 bytes [16:03] probably [16:03] and actually it leaves it up to fping now that I read right [16:03] just set it to 32 :) [16:03] "Default is 56, as in ping." [16:03] mercutio: 15:58:49 < brycec> Change made [16:03] less load on the network [16:03] oh you did [16:03] But I like chewing up ~7GB/day [16:04] heh [16:04] ~2MB/minute, from 5 monitoring hosts [16:04] thing is it's every destination network [16:04] and the more people use larger packets etc the more people think it's a good idea to block icmp [16:04] which is annyoing [16:05] I thought I'd dialed back the default to 10%, I was a Good Guy :p [16:05] dumb docs [16:05] some of the nlnog ring nodes seem to have stopped responding [16:05] staticsafe: if you're talking about my graphs, some never started responding :P (hurricane, I'm looking at you...) [16:05] ah [16:06] I wanted to give them time to fix themselves :p [16:06] *** m0unds has joined #arpnetworks [16:06] your smokeping responds so slowly [16:06] i dunno if it's cos it's swapping, or because you have so many hosts [16:06] what do you mean? [16:06] like clicking things is slow [16:07] And no swapping [16:07] oh maybe cos it's https [16:07] I like the idea of the NLNOG ring [16:07] me too [16:09] (mmm first page load after I restar the fastcgi, now THAT is slow) [16:09] mine are slower than they could be on arp cos not using fastcgi [16:10] brycec: it was pretty quick for me via comcast fwiw [16:10] it seems your pages take ~2.5 seconds to generate [16:10] Thanks m0unds [16:10] half a second or less for me [16:10] it was 2.6 seconds from arp with curl [16:10] weird [16:10] time curl -v 'https://smokeping.cobryce.com/?target=Internet.NLNOGRING' > /dev/null [16:10] is what i was doing [16:10] it's a bit up and down, now it's 1.4 seconds from arp [16:11] 2.47 to generate for me [16:11] from arp? [16:11] .947s for me from comcast in nm [16:11] images only 250ms each [16:11] haha [16:11] mercutio: from my desk [16:11] i just got a bad gateway error [16:11] damnit who broke smokeping [16:11] from nginx [16:11] ahahahahah [16:11] Sometimes the fastcgi crashes... [16:11]