***: gizmoguy has joined #arpnetworks
Hien has quit IRC (Ping timeout: 264 seconds)
solj has quit IRC (Ping timeout: 264 seconds)
phlux has quit IRC (Ping timeout: 264 seconds)
staticsafe-znc has quit IRC (Ping timeout: 264 seconds)
SpeedBus has quit IRC (Ping timeout: 245 seconds)
staticsafe-znc has joined #arpnetworks
Hien has joined #arpnetworks
gizmoguy has quit IRC (Ping timeout: 264 seconds)
gizmoguy has joined #arpnetworks
solj has joined #arpnetworks
SpeedBus has joined #arpnetworks
phlux has joined #arpnetworks
SpeedBus has quit IRC (Ping timeout: 245 seconds)
SpeedBus has joined #arpnetworks
robonerd has quit IRC (Read error: Connection reset by peer)
robonerd has joined #arpnetworks
robonerd has quit IRC (Read error: Connection reset by peer)
robonerd has joined #arpnetworks
LT has joined #arpnetworks
robonerd has quit IRC (Read error: Connection reset by peer)
robonerd has joined #arpnetworks
staticsafe-znc has quit IRC (Ping timeout: 265 seconds)
staticsafe has quit IRC (Ping timeout: 265 seconds)
xales has quit IRC (Ping timeout: 246 seconds)
staticsafe has joined #arpnetworks
staticsafe-znc has joined #arpnetworks up_the_irons: mnathani: no i mean VPS customers m0unds: i tried it in a vps once, it works fine - alias configuration is a PITA vs freebsd or openbsd though ***: dj_goku_ has quit IRC (Ping timeout: 248 seconds)
mjp_ has quit IRC (Ping timeout: 265 seconds)
abthorpet has joined #arpnetworks
mhoran1 has joined #arpnetworks
ChanServ sets mode: +o mhoran1
[FBI] has quit IRC (Ping timeout: 265 seconds)
[FBI] starts logging #arpnetworks at Thu Feb 13 07:44:23 2014
[FBI] has joined #arpnetworks
mhoran has quit IRC (Ping timeout: 265 seconds)
tabthorpe has quit IRC (Ping timeout: 265 seconds) brycec: What do you mean? Clicking buttons in a GUI versus just typing it in pf.conf? 06:43:51 < m0unds> i tried it in a vps once, it works fine - alias configuration is a PITA vs freebsd or openbsd though
In which case, I'd say that writing rules is similarly PITA :p m0unds: brycec: no, when i messed with it somewhat recently, interface aliases don't work the way i expected them to (the way they work in freebsd or openbsd) brycec: Do you mean "interface groups" as they're termed in pfSense? m0unds: the UI element does some weird stuff with aliases that wasn't clear
i don't remember, but it felt kinda counterproductive
at any rate, i only looked because i hadn't used it since like 2007 brycec: heh
Well I'm happy to say pfSense has come a long, long way in 7 years :p m0unds: and the thing that seemed like it should be alises wasn't
well, they're still on teh same release tree and the ui has some awful 90s UX to it that they need to get rid of asap :P
the notification thing at the top in teh default theme is awful
s/teh/the BryceBot: <m0unds> the notification thing at the top in the default theme is awful m0unds: i looked at m0n0 last night because i hadn't used it since probably 2003, and they just moved up to freebsd 8.3 last month, haha brycec: I don't think I follow what you mean by same release tree - were you expecting them to just scrap everything and start over?
Nice, they're catching up to pfSense m0unds: no, they were working on "2.0" forever brycec: IIRC that 8.3 is due to the NanoBSD base.
True, but 1.2.3 was stable up until ~2 years ago
*stable/current
That 2.0 was effectively a total rewrite m0unds: yeah, so i guess it was 2 years ago that i messed with it, haha
because it was 1.2.3 (i still have the disk image on my kvm server at home) brycec: "Version 1.0 of the software was released on October 4, 2006.[5] Version 2.0 was released on September 17, 2011,[6] with updates 2.0.1 to 2.0.3 between then and 2013, and version 2.1 was released on September 15, 2013."
Or you can peruse http://www.pfsense.org/about-pfsense/versions.html
1.2.3 released Dec 2009, 2.0.1 released Dec 2011
at least they're somewhat consistent
Good news, up_the_irons, ARP was not on the list of the top 24 networks http://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack
It didn't make the list at all! https://docs.google.com/spreadsheet/ccc?key=0AhuvvqAkGlindHFtS0pJa0lYZGNlLXNONWtlY01qanc&usp=sharing#gid=0 m0unds: actually, i did try it more recently - i was trying to figure out how i knew about the thing at the top. i tried it on my small arp vps after i migrated stuff to the bigger arp vps
i do have a 1.2.3 disk image locally though
which i don't remember doing anything with
at any rate, maybe it was a quirk with the vps having a single if or something, or i could just be remembering something that doesn't exist, hahaha
haha, chinanet. go figure. brycec: srsly
and ovh m0unds: yeah
color me surprised -: brycec colours in m0unds m0unds: what color is surprised? ***: LT has quit IRC (Quit: Leaving) brycec: m0unds: mauve m0unds: eeewwwww
at least it's not taupe
or chartreuse brycec: Oh sweet, I'm getting hit with buttloads of DNS queries... -: m0unds throws DNS queries at brycec brycec: I'm guessing my own DOS reflection
an attack of my very own. On a cable modem. woo m0unds: sweet brycec: I wonder how this IP was choosen... I don't really use it for anything, and it's not in DNS anywhere m0unds: are you connected to IRC w/it?
or bot connected from it or something?
the only time i ever had a box get ddosed was when i was connected to a provider's IRC channel w/v4 address and some kid decided to start ddosing people connected to the channel brycec: nope and nope
It's the external IP of a router, but all traffic goes out a different IP m0unds: huh.
is the address SWIP'd to another business or something? brycec: dun think so m0unds: you must just be lucky brycec: holy crap, TWC actually filled in the business information
(protip: always smart to whois your own addresses once in awhile)
ANY? .
that's the query
whee -: brycec wonders who to complain at hazardous: this is probably odd question but does arp ratelimit dns or anything brycec: UDP inbound as I recall
to 5mbps hazardous: $ time host lightning.net >/dev/null
real 0m8.197s
$ time host nac.net >/dev/null
real 0m2.327s brycec: ouch
real 0m0.946s hazardous: using the official arp resolvers, and i don't get this kind of response time elsewhere even hitting the same authoritative ns
close to 1 second seems incredibly weird/bad as is brycec: hazardous: for google.com real 0m0.161s hazardous: yeah that seems cached maybe, idk brycec: And nac.net now gives me real 0m0.027s hazardous: i sometimes have periods of time when it's somewhat acceptable and periods of time where anything takes forever brycec: http://support.arpnetworks.com/kb/main/is-there-a-firewall-filter-rate-limit-or-similar-device-applied-to-my-traffic
outbound UDP traffic is rate-limited to 5mbps
I wonder if up_the_irons / ARP's resolvers are exempt from that, and/or being overwhelming it
wow that was terrible grammar m0unds: try again
hahah hazardous: yeah i dunno, i just did host google.com and the A records returned instantly
then it hung for 3-4 seconds before returning the mx part m0unds: huh.
was just gonna say maybe it wasn't cached but if it's hanging on mx recs, dunno
the prev query, i mean
since it returned google quickly hazardous: yeah i have absolutely no idea
http://pastebin.aquilenet.fr/?1328915b0bfb2488#y8PHFMpIH159tKAd5vFItNbggiBNrLCLq4fJmGtx7oE=
wat brycec: oh man this dns ddos is terrible, how will i deal with the 50kbps of traffic pouring in???
Silliest. DOS. Ever. m0unds: omgz, you better mitigate it up_the_irons: brycec: <phew> ARP wasn't in the list ;) mnathani: What is the purpose of running pfSense (a firewall) on a VPS with only one interface? -: m0unds shrugs m0unds: i did it to screw around; you can still use it as a gateway or firewall in front of another vps if you really wanted to
you don't /have/ to have separate interfaces unless you're gonna nat brycec: And even then, there are vlans ;) (or vlans inside of vlans, on ARP) m0unds: yes toddf: vlans inside of vlans on ARP only work if you use svlan(4) (IEEE 802.1AD)
and then only if you're unblocked and permitted to do so
I know from experience!
now don't try to do svlan(4) on vio(4) on current, something about cksums kills it
current openbsd that is mnathani: Does anyone know how much it costs to Akamaize a website? (Serve it using Akaamai's CDN) ? m0unds: i've never seen $ amounts, but i've also never heard 'affordable' mentioned alongside their name
might look into edgecast or cachefly too mnathani: Ofcourse it would depend on the size of the site / complexity beign served as well as actual traffic / bandwidth
s/beign/being BryceBot: <mnathani> Ofcourse it would depend on the size of the site / complexity being served as well as actual traffic / bandwidth m0unds: i googled and found pricing from an akamai partner showing a $4/GB x 500GB/mo commit
and that was 'cheap' mnathani: so approx $2000/month for that setup? m0unds: yea
later clarification showed $200/mo @ per GB
and $375/mo for 500GB
http://www.cachefly.com/pricing.html mnathani: m0unds: thans
s/thans/thanks BryceBot: <mnathani> m0unds: thanks RandalSchwartz: cloudflare.com is free if you don't need SSL m0unds: yeah, cloudflare
's free stuff is good if you just want the cdn features toddf: indeed it does sound good up_the_irons: i'm not sure what i like better
allow(Mail).to receive(:all).and_return(@msgs)
allow(Mail).to receive(:all) { @msgs } staticsafe: http://www.pantz.org/software/pf/use_freebsd_10_as_a_pf_firewall.html mike-burns: I like the #and_return. It's more clear what it's doing. mercutio: m0unds: i prefer cahefly to akamai as a user mike-burns: I save the block syntax for when I really need a block. mercutio: akamai is often terrible performance with "cache misses"
akamai has closer nodes to me than cachefly, but cachefly's average performance is way up. up_the_irons: mike-burns: ah mercutio: it's harder to test cache miss performance though. you used to be able to send ?1 ?2 ?3 ?4 etc to get an uncached version of stuff to test... but that seemed to stop working
and with the number of akamai nodes, unless you're huge there are likely to be lots of cache misses
cloudflrae is terrible i reckon up_the_irons: CloudFlare should get on Any2 so I can peer with them
Akamai is, so is EdgeCast mercutio: i don't think it'd make meuch difference?
cloud flare is in san jose atm isn't it?
i can't think of any domains that use it off hand up_the_irons: NO, peering with ARP makes ALL THE DIFFERENCE IN THE WORLD. Get it right mercutio ;) mercutio: well i doubt there's much traffic being pushed to there, and with multiple upstreams ... incoming ins't likely to saturate
and hardly any users on arp are likely to be pulling large files off cloudflare up_the_irons: but cloudflare pulls lots of files from arp mercutio: oh
ok i didn't realise that up_the_irons: :)
some dedi customers use CF heavily RandalSchwartz: mercutio - anyone hosted at (mt) is using cloudflare by default up_the_irons: yeah, shared hosting co's are starting to do that
we have some here mercutio: mt? up_the_irons: media temple RandalSchwartz: (mt)® is Media Temple. Not sure what "mt" is. :) brycec: me either. But I'm pretty sure a RandalSchwartz is a smartass :P mercutio: i've only noticed cloudflare when sites have problems
i suppose that's one of the problems with those things brycec: ditto mercutio: some sites only shift to cloudflare when they're getting ddos'ed
or have load issues
and if a site's going slow and you trace and it says cloudflare.. RandalSchwartz: insightcruises.com had it enabled for a week, then they screwed up the DNS, and different people were getting different pages or even A records pointing to nowhere staticsafe: O_o RandalSchwartz: so we ripped it all out, and haven't gone back
might have been early growing pains mercutio: who's using it right now? RandalSchwartz: presume these guys: https://www.cloudflare.com/case-studies up_the_irons: RandalSchwartz: you coming to any js.la meetups soon? it's kinda hard to believe the last one where we had a chance to chat was over a year ago! (christmas before last) mercutio: imgur will do up_the_irons: RandalSchwartz: or perhaps that was devops.la... hmmm
*ladevops RandalSchwartz: interesting - http://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack -: up_the_irons brain is dying up_the_irons: RandalSchwartz: that was pasted earlier RandalSchwartz: my client before this one was directly across the street from my hotel and a bar. I stopped going to meetups. :)
now I'm working in santa monica, and I have to actually drive around, so I can start doing meetups again.
plus, it'll give me an excuse to drink a bit less. I was a bit out of control because I wasn't driving for 5 days at a time. staticsafe: o_o mercutio: oh cloudflare do have node in los angeles staticsafe: they would be stupid not to mercutio: and the ?1 ?2 trick works m0unds: they've got lots of pops mercutio: cache hit time in la, 20 msec, cache miss time 500 msec
cache hit time in nz, 85 msec, cache miss time 530 msec brycec: not bad
pretty consistent anyways mercutio: yeh i seen way worse from akamai
with > 1 second. m0unds: gross mercutio: it's only 15k file though
it seemed to get more consistent when i did it more m0unds: my friend put their blog on the free cloudflare plan and saw spam comments just stop altogether - i think just as a side effect of cloudflare's anti-bot stuff mercutio: imgur still loaded images slow
main imgur.com site isn't cdn'ed, jsut their images m0unds: right
img.imgur.com mercutio: nah i.imgur.com m0unds: img.imgur.com is too mercutio: oh ok staticsafe: imgur is quite reliable
imgur++ mercutio: i don't normaly look at it
http://imgur.com/gallery/oxn6ZMh
but this is funny staticsafe: heh mercutio: damn now i want to test akamai
but they using https
akamai is 56 msec hit, 1 second miss for apple 56k image
and 17 msec hit, 260 msec miss from la
i suppose that not so bad, i suppose it depends wehre origin servers are
still even for 56k 1 second miss seems kind of bad to me mnathani: Just setup my first site to site vpn in a lab, but I can't seem to ping from a subnet behind the vpn to a subnet on the other site. mercutio: did you enable forwarding? mnathani: The reverse ping works just fine. (pfSense with Openvpn btw) brycec: Did you set the routing?
Non-overlapping subnets?
Firewall? mnathani: firewall - allow any/any
non-overlapping subnets yup brycec: traceroute, tcpdump mnathani: the server config section had local subnets and remote subnets mercutio: i assume pfsense would enable forwarding :) mnathani: client only had remote subnets
client side can ping server side, but not reverse
how foolish of me, twice in as many days now, Windows Firewall got the better of me and dropped my icmp packets mercutio: oh that's normal brycec: lolol mnathani: now I need to redo it to verify if all that config I did was necessary brycec: @mnathani BryceBot: Have you checked whether Windows Firewall is enabled and dropping packets? mnathani: adding extra route statement etc BryceBot: 425 results found. Here's #265 Dec 19, 2013 16:43:30 <mnathani > requests are going through, but not being cached I dont think mnathani: lol BryceBot / brycec brycec: You know what they say, if you use something 3 times, you sohuld buy it/script it.
Well, this was only twice but I'm proactive ;) RandalSchwartz: Ugh - had some beans and rice (normally skipped) with my mexican dish for lunch
now I'm in a carb crash fog :(
forgotten how bad these are up_the_irons: RandalSchwartz: yeah they suck m0unds: yeah, rice tends to knock me on my ass too
i think i'm gonna adjust my pf ruleset a bit - i got lazy and didn't define macros or anything so the rules are ugly as hell toddf: when done.. var=..nvar=..nntable..ntable..nnmatch..nmatch..nnblock..npass..n done! ;-) mercutio: y'll are lazy
oh i don't quite know how the y thing works
i don't use macros with pf either brycec: y'all are lazy'
well you get the idea I think... You need 2 or 3 separators
it's a regex like any other mercutio: oh RandalSchwartz: ugh - this office has horrible "free" stuff in the kitchen. either sweetened beverages ("vitamin water", ugh), or decaffinated teas.
and all sorts of "hearthealthywholegrain" snacks
it's sad how much I've found out that this stuff is all crap in the past 18 months brycec: (well not exactly a regex... it's a PCRE verb, with sed expression syntax) RandalSchwartz: PCRE is a misnomer :) m0unds: i like macros because it keeps things readable - granted, i'm likely the only one who will ever see it, it's still nice to keep it concise
i do the same w/network hw configs on gear that actually supports macro type functionality (junos) ***: m0unds has quit IRC (Quit: reloading) mercutio: hmm cloudflare aren't on coresite ***: mnathani_web has joined #arpnetworks mercutio: so best hope is for bgp collective to spread to equinix mnathani_web: are there network issues atm
I cant ping arpnetworks.com or my VPS mercutio: weird
traceroute? mnathani_web: sec. toddf: I cannot ping arpnetworks.com mnathani_web: http://pastebin.com/wZehQBW7
here is mtr toddf: I can ping 2.v.freedaemon.com (which is on the 1gbps ports) mercutio: blame nlayer? mnathani_web: traceroute is taking a while meingtsla: i can't resolve arpnetworks.com, weird
oh wait yes i can mercutio: 208.79.89.243
i'm on arpnewtorks fine if anyone wants a traceroute in other direciton
vl5.s1.lax.arpnetworks.com is giving packet loss toddf: arpnetworks v6 is working, arpnetworks v4 is not for ns1 and ns2.arpnetworks.com meingtsla: ah toddf: just started responding mercutio: ge0-arpnet.cust.lax07.mzima.net gives packet lsos too
i wonder what happend mnathani_web: its back now mercutio: oh
i never lost connectivity :) RandalSchwartz: lsos! brycec: Also showing IPv4 loss from INSIDE ARP https://smokeping.cobryce.com/?target=ARP.ThisGW
more accurately, packet loss on my own vlan to the router
guessing router crash? mercutio: nice
but not for me
are you on gigabit? brycec: Don't think so? mnathani_web: could it be BIRD related? mercutio: i'm not pinging the link address
don't think so
hardly anyone goes over any2ix on the new range
i think it's proably nlayer brycec: fwiw I can ping arpnetworks.com and my VPS both. As far as I can tell from smokeping, there _was_ a brief outage but it's back up now. mercutio: actually why would link address stop working then brycec: only likely cause, something unplugged or reset mercutio: wow you have a lot of sites in smokeping :) brycec: About 250 probes mercutio: i like the overlayed thing
how much bandwidth does that use? brycec: 212 hosts to be precise
overlayed... When there's a whole bunch of hosts on one graph? like for the IRC networks? mercutio: yeah -: brycec asks graphs.arpnetworks.com how much b/w is used brycec: A surprising amount.
Reportedly 6.71GB in the last 24 horus mercutio: 20 ping every 60 seconds staticsafe: all that ICMP mercutio: 212 sites brycec: mercutio: 500byte packet size too staticsafe: brycec: would you mind sharing your hosts file for smokeping? mercutio: ahy are you doing 500 byte packet size
i was going to ask that but he has some local stuff in it too brycec: mercutio: Because it's better than the default 5000 :P mercutio: frguly isn't 500
err 5000 it's like 72 brycec: According to the docs it's 5000 mercutio: where brycec: http://oss.oetiker.ch/smokeping/doc/smokeping_config.en.html mercutio: oh wtf
that's example value
which isnt' the default, but why are they suggesting that brycec: er http://oss.oetiker.ch/smokeping/probe/FPing.en.html mercutio: i'd suggest trying 32
and see if graphs look bsaically the same brycec: You have in the past :P mercutio: and identify issues just the same brycec: And if the docs don't say "default" but do say "example" what am I to assume? mercutio: well i thought i'd do it again
ikr brycec: They aren't the world's best docs :/
Change made
just in time for 4pm on the dot mercutio: hmm.. up_the_irons have you considered making graphs of transit links visible to users? brycec: Ah, 56 is the default. RTFS :/ mercutio: hmm
i wonder why i saw 72 then
oih maybe it was 76
i htink 56 doesn't include the IP header size of 20 bytes brycec: probably
and actually it leaves it up to fping now that I read right mercutio: just set it to 32 :) brycec: "Default is 56, as in ping."
mercutio: 15:58:49 < brycec> Change made mercutio: less load on the network
oh you did brycec: But I like chewing up ~7GB/day mercutio: heh brycec: ~2MB/minute, from 5 monitoring hosts mercutio: thing is it's every destination network
and the more people use larger packets etc the more people think it's a good idea to block icmp
which is annyoing brycec: I thought I'd dialed back the default to 10%, I was a Good Guy :p
dumb docs staticsafe: some of the nlnog ring nodes seem to have stopped responding brycec: staticsafe: if you're talking about my graphs, some never started responding :P (hurricane, I'm looking at you...) staticsafe: ah brycec: I wanted to give them time to fix themselves :p ***: m0unds has joined #arpnetworks mercutio: your smokeping responds so slowly
i dunno if it's cos it's swapping, or because you have so many hosts brycec: what do you mean? mercutio: like clicking things is slow brycec: And no swapping mercutio: oh maybe cos it's https staticsafe: I like the idea of the NLNOG ring brycec: me too
(mmm first page load after I restar the fastcgi, now THAT is slow) mercutio: mine are slower than they could be on arp cos not using fastcgi m0unds: brycec: it was pretty quick for me via comcast fwiw mercutio: it seems your pages take ~2.5 seconds to generate brycec: Thanks m0unds m0unds: half a second or less for me mercutio: it was 2.6 seconds from arp with curl
weird
time curl -v 'https://smokeping.cobryce.com/?target=Internet.NLNOGRING' > /dev/null
is what i was doing
it's a bit up and down, now it's 1.4 seconds from arp brycec: 2.47 to generate for me mercutio: from arp? m0unds: .947s for me from comcast in nm brycec: images only 250ms each m0unds: haha brycec: mercutio: from my desk mercutio: i just got a bad gateway error brycec: damnit who broke smokeping mercutio: from nginx m0unds: ahahahahah brycec: Sometimes the fastcgi crashes... mercutio: hmm mayb eit is cos too many people using it m0unds: womp womp brycec: (And yes from nginx, because nginx is servingup the fastcgi) mercutio: how many fastcgi processes does it run? m0unds: 1100 brycec: mercutio: Just the one process, because just the one cgi
Unless you're asking total on the system... In which case still just one. mercutio: i think i have two
yeh two running as www-data
and one running as smpokeping
i wonder if that slows it down more for me
cos i so far away
oh images don't go via smokeping brycec: (graphs is showing a reduction in b/w woo)
Yes. Those images are built on page load up_the_irons: mercutio: not considered it mercutio: i understand there may be some reluctance to
up_the_irons: any idea what happened aerlier? up_the_irons: brycec: on v4 it would not be a router crash
s1.lax#sh ver
...
s1.lax uptime is 4 years, 51 weeks, 4 days, 4 hours, 33 minutes m0unds: lol up_the_irons: mercutio: no idea m0unds: what chassis is that one? up_the_irons: 4500 m0unds: cool staticsafe: dat uptime m0unds: my vss pair (2x6509E) is at 3y49w2d5h37m up_the_irons: nice :) m0unds: yup, pretty solid
our IS dept did a core upgrade to redundant 7009s - those have been horrible mercutio: i think the longer you deal with IT the more you decide that everything is terrible m0unds: 15 years in and i can't say i have that opinion at all mercutio: really? m0unds: some stuff is really bad, some stuff isn't as bad, some stuff is good up_the_irons: yeah me either mercutio: smtp, spam up_the_irons: i've found some stuff to be very good mercutio: oh right, but the conditions
get worse and worse
bugs get more obscure etc
i suppsoe the problem with things like routers is traffic volume goes up and up m0unds: right mercutio: if 100 gigabit connectinos were standard it wouldn't be such an issue
but tehy're new, and require new investment etc
and heaps of "background" stuff to make it work
i suppose one advantage of bandwidth going up over time is it's going to get harder and harder to sniff traffic brycec: Heh, exactly at 4pm, my usage drops considerably https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-02-13_16-26-15.png mercutio: do the graphs look the same? -: brycec hasn't looked brycec: probably, yes mercutio: yeh basically brycec: I don't have side-by-side 500vs32, but seems right mercutio: although you'd havd a few outages
to this tunnel brycec: huh?
English please. mercutio: ther's gaps in your grpahs brycec: yes when I restart smokeping
Well 15:39 was the ARP outage mercutio: i don't think it's that
https://smokeping.cobryce.com/images/Internet/NLNOGRING/doruknet01ringnlnognetv4~vps1_last_3600.png
it looks like what smokeping does when it can't keep up brycec: That's chunkhost though... those have always been shitty mercutio: oh
it doesn't say loss though brycec: I know mercutio: are you doing dns lookups? brycec: For whatever reason, that slave just doesn't keep up
some mercutio: smokeping can have issues with graphs if it doesn't do all the hosts in the right time period brycec: Yeah I know mercutio: and i've seen it happen due to partial issues with connectivity to some hosts, making dns timeout before -: brycec wishes smokeping bad better, or any logging mercutio: it has logging
it tells you hwen it can't keep up brycec: I only see logs when I start/stop mercutio: weird
are you using rrdcached?
i was just checking my logs and saw some stuff from rrdcached brycec: Last I heard, smokeping doesn't support rrdcached staticsafe: yea it doesn't mercutio: hmm maybe i was using that for cacti brycec: (I use rrdcached for munin stuffs though, but that's isolated) -: staticsafe needs to completely redo his smokeping setup staticsafe: i got rid of munin entirely
I just use zabbix now brycec: Ah that's why you wanted my configs
staticsafe: there's nothing special or unique to my Targets config, it's just long mercutio: i'd like to see a standard set of test sites myself staticsafe: brycec: yea, just want it to because I'm lazy :P brycec: sure I'll sanitize and toss it on sprunge.us staticsafe: ty brycec: mercutio: that's my dream with nlnog
That I can use them as standard test sites mercutio: ahh staticsafe: brycec: my config isn't even monitoring v6 atm mercutio: well i hvae google, gmail facebook twitter slashdot, nytimes, anandtech bbc, guardian godaddy, ubuntu archive wikipedia, staticsafe: which is unacceptable mercutio: which are some goodish sites to test
but like guardian hops around
ubuntu archive always give packet loss when there is a new release
and hmm i didn't comma right at all
so it can make sense to do from multipel sources, to check whether it's the destination site or an in general thing brycec: staticsafe: As requested http://sprunge.us/ZOhN staticsafe: brycec: ty brycec: It needs cleanup... I was inebriated at one point and forgot how Config::Grammar inherits staticsafe: heh
config inheritance is my favorite thing
nagios... -: brycec was thinking fall-through, hence calling SlavesV4/6 over and over and over brycec: (but it's actually parent/child inheritance) staticsafe: # Boy this is getting annoying.
hehe brycec: it really was! mercutio: heh
so basically you're scraping brycec: scraping?
referring to my regex? mercutio: to get the destination sites brycec: As opposed to inventing sites? :P mercutio: or did you manually do mirrors?
yeh
there's a lot of text brycec: The scraping was not automated mercutio: looks like you're relying on dns too -: staticsafe dumps all existing smokeping data mercutio: i wnoder if fpign can have a cache for dns
and use prior data if it cant' do a lookup brycec: mercutio: I'd copy from a webpage into my text editor, apply the regex to mold into target configs, and paste that into Targets staticsafe: mercutio: run a local resolver? brycec: mercutio: Yes, some sites don't have fixed IP's. mercutio: staticsafe: you could still hit expiring ttls staticsafe: yep mercutio: unless you use unbound with prefetch hmm staticsafe: which is fine brycec: (or are known to change their IP from time to time) mercutio: you could still hit it though
static: not if the site goes down
the problem is you don't want to fping to wait forever for dns
s/to// BryceBot: <mercutio> the problem is you don't want fping wait forever for dns mercutio: oh
i just wanted to kill the first one brycec: too bad :P mercutio: so fping can lose all the results
does it assume /g ? brycec: yes
Because that's what PHP's preg_replace does staticsafe: looking at my traffic graphs and my machines are so idle :( mercutio: heh
you should host wikileaks mirror brycec: Is wikileaks still a "thing"? :p mercutio: no idea staticsafe: mercutio: or a tor relay :P mercutio: gah no not tor
argh brycec: up_the_irons has stated that he drops you at the first abuse report mercutio: weird my vm just spiked to 20kbit bnadwidth brycec: Tor seems liek an unnecessary risk mercutio: it looks funyn on the graph staticsafe: brycec: only exits get abuse reports
not relays mercutio: but then 20kbit isn't much brycec: true staticsafe: but i wouldn't run it on ARP anyways mercutio: i average 2kbit/sec inbound
and 1.68kbit/sec outbound staticsafe: seed some linux ISOs too I guess mercutio: i suspect most vm users don't use much bandwidth staticsafe: agreed mercutio: otherwise the ntp thing wouldn't ahve been as obvious
it's probably like 10% of the users use 90% of the bandwidth kind of thing -: staticsafe nods mnathani: Another pfSense issue, I have 4 interfaces: WAN, LAN, OPT1 and OPT2
LAN and OPT1 can ping 8.8.8.8
but OPT2 can not
identical firewall rules
OPT2 can ping LAN and OPT1 though brycec: @mnathani BryceBot: mnathani: Have you checked whether Windows Firewall is enabled and dropping packets?
432 results found. Here's #204 Dec 06, 2013 21:22:38 <mnathani > Anyone know of a method to Auto-BCC a copy of all outgoing mail to a specific address from within the Gmail Web Interface? mnathani: packet capture shows pings leaving, but no reply
lol dont think 8.8.8.8 is behind a windows firewall
lol though brycec: Suuuure
opt1 and opt2 setup identically? routable subnets? mnathani: yup brycec: And just for kicks, swap (assign) them and see what happens mnathani: after swapping, the vm that could ping can no longer ping and the vm that could not ping can ping now brycec: Good
Just wanted to confirm it wasn't something besides configuration
and that the vm's were confi'd right mnathani: would you reset pfsense interface assignments at this point brycec: If it didn't matter, I'd leave that and focus on the issue mnathani: found it
it had a gateway assigned to it that needed to be removed
from within the pfsense interface brycec: awesome
lol why would you set the gateway on the internal interface? :p mnathani: "It made sense at the time" ... ***: dj_goku has joined #arpnetworks
dj_goku has quit IRC (Ping timeout: 260 seconds)
dj_goku has joined #arpnetworks
jcv has quit IRC (Ping timeout: 265 seconds)
jcv has joined #arpnetworks brycec: (Oh good, I just checked and the attempted DNS DOS on me subsided :D) m0unds: whew, that was close
http://1drv.ms/1c4utnA
i had a live wasp crawling its way across the floor in my server room brycec: wasps are a "NOPE!" for me
practically send me running m0unds: haha, i captured it in a cup and took it outside
this guy was super lethargic because of how cold it is in that room brycec: oh shit... I just discovered my carp backup's ntpd is open. damnit damnit. m0unds: YOU'RE PART OF THE PROBLEM brycec: not according to monlist
at least, a very tiny part m0unds: haha
someone in another channel said they had a supermicro board w/ntpd + monlist running on its ipmi interface -: brycec decides it's easier to firewall rather than edit the conf brycec: lolol mercutio: brycec: you mean it tells you the time? brycec: I mean it lists its recent peers, but it's a short list mercutio: oh right
all the open to monlist hosts got blocked
even the ones with shorter lists brycec: I'm talking about a non-ARP system mercutio: yeh on arp i mean m0unds: (we were here yesterday when up_the_irons said he was gonna do it) mercutio: i dunno if people trying to ddos differntiate
oh right
was that eysterday m0unds: yea, or the day before
the days all sort of blur together -: brycec shakes his fist at pfSense mercutio: openntpd hasn't taken off as much as openssh did brycec: (pfSense has "pass in quick" rules to explicitly allow NTP access on all interfaces at the top :( grr) m0unds: whoa, really? brycec: yep
pfctl -sr http://sprunge.us/Vijb
Ahahaha http://translate.google.com/#auto/en/%E8%87%AA%E5%8B%95%E3%82%A2%E3%82%AF%E3%82%BB%E3%82%B9%E3%81%AE%E5%A0%B4%E5%90%88%E3%81%AF%E3%80%81%E9%9B%BB%E5%AD%90%E3%83%A1%E3%83%BC%E3%83%AB%E3%82%92%20ntp-scan%40puck.nether.net%20%E3%81%B8%E3%81%8F%E3%81%A0%E3%81%95%E3%81%84
"For automatic access, please fart ntp-scan@puck.nether.net e-mail"
Reviewing some tcpdumps, I see people trying to use my (correctly configured) router to NTP DDOS. Yay for it being setup properly at least
wow, quite a bit in fact mnathani: what can you do using the command line / ssh with pfSense that can't be done from the web interface? brycec: dd? :P
well even that can be done from the web
On account of Diagnostics->Command
mnathani: I find it easier to dump pfctl info, run tcpdumps and other diag tools mnathani: I guess I meant in terms of managing / configuring the firewall brycec: I leave all management and configuration to the web ui, unless I lock myself out. Since all changes get made to an XML config, best to leave it in the capable, tested, properly-formatting hands of the GUI
Though I suppose watching pflog could count as managing mnathani: what distro is it easiest to setup / resolve dependencies of smokeping? brycec: mnathani: Debian, I'd say
At least for slaves...
I just launch an VZ container, apt-get install --no-install-recommends smokeping ; service smokeping stop ; $EDITOR /etc/default/smokeping /etc/smokeping/secrets ; set permissions ; service start smokeping
19... 1000 packets dumped, 19 different "sources" -: brycec takes a larger sample mnathani: brycec: thanks brycec: np
also, wtf my VPS load spiked to 33
christ on a cracker mnathani: 00:20:50 up 6 days, 8:50, 1 user, load average: 0.00, 0.00, 0.00
00:21:09 up 17 days, 0 min, 1 user, load average: 0.08, 0.10, 0.07
my 2 ARP VPSen brycec: @uptime host BryceBot: host uptime: 140 days, 22 hours, 35 minutes, and 48.429999999702 seconds. brycec: My one
(and loadavg is settling back to the .2 range mercutio: brycec: so much more bw you doing an hour with smaller ping size? brycec: mercutio: Funny you should ask, I was just looking at that https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-02-13_21-31-44.png
You can clearly see when I added a bunch of hosts with size=500, and when I dropped that back down mercutio: iyou're probably still doing a lot of pps
were you at defaults prior?
i didn't quite get that
whther you'd shifted to 500 byte packets today
or ages ago
there are still gaps
i don't think fping does have a way to cache dns lookups
you can use ps to find out what command line it's calling and cut and paste the fping
and do one ping, to figure out long it's waiting on dns
but would need to do it during some kind of outage to know for eusre how much it impacting
comcast.net if you ping that has dns ttl of 30 seconds for instance
so isn't likely to be cached between polls ***: kevr1 has joined #arpnetworks
kevr1 has quit IRC (Quit: WeeChat 0.4.3)
mnathani_web has quit IRC (Quit: Page closed)
xales has joined #arpnetworks