#arpnetworks 2014-02-12,Wed

mercutio

was there just a network problem?

brycec

not afaik

up_the_irons

not that i saw

brycec

Why, would you like one?

mercutio

hah i think nodeping is having issues
how can thinks go up without going down

brycec

hahaha
they went down, but the down notification was lost due to things being down?

mercutio

oh it's changing numbers rnadomly

https://smokeping.cobryce.com/?target=Internet.HENet.Europe.HEtserv11ams1v4 https://smokeping.cobryce.com/?target=Internet.HENet.Europe.HEtserv28waw1v4

up_the_irons

like a BOSS

[00:26]

mercutio

so my cellphone don't group

[00:26]

brycec

Is that like blinking lights randomly, to look busy?

[00:26]

mercutio

heh
cloud core routers have a lcd
i mean seriously, if a router is working who wants to look at a lcd

[00:28]

up_the_irons

srsly

[00:29]

brycec

I can see how nodeping might be having some issues actually

both showing massive loss to ARP in the last few minutes

[00:29]

mercutio

i'm not impressed by nodeping
oh wow
bah he tunnels
omg i wish amazon wouldn't email me about stuff i stuck in my cart
it emails me all the time
there seemed to be a little bit of a dip
but it wsa more liek total outage for a bit

[00:30]

brycec

(returning to normal)

[00:32]

mercutio

then ping came back less than before the outage
so i wonder if where was some hw chnage somewhere?
actually it's not total outage it's 19/20 packets dropped
but to my mind that means total outage :)

[00:32]

brycec

brycec sticks that in the SLA fine-print -- "service still available, does not count as downtime"
:P

[00:33]

mercutio

oh weird minimum ping went interesting
suggesting los angels to nz
rather than la -> sj -> nz
i wodner if any2ix fell over temp
i don't think there's any way to view sessions ?

[00:33]

up_the_irons

but are those graphs measuring tunnels?
seems like a lot of moving parts there

[00:36]

mercutio

gah why is my irc suddenly laggy
it's wihtin nz
i think it's probably ddos
gah

[00:36]

brycec

up_the_irons: my graphs are just from the VPS to the IP in the graph title. no tunneling

up_the_irons

ah ok

brycec

And if there's any way I can make that clearer, let me know

up_the_irons

cuz the list on the left says tunnels

brycec

I'll rename it ;)

up_the_irons

OK:)

dne

staticsafe: the smokeping webapp can be run as a PSGI app, if that helps :)

[00:45]

brycec

brycec is running his as fastcgi

dne

dne runs his using starman

brycec

Otherwise all this attention would be crushing me :)
@imdb Starman

BryceBot

[MOVIE] Title: Starman | Year: 1984 | Rating: 7.0 | Genre: Adventure, Drama, Romance | IMDB Link: http://imdb.com/title/tt0088172

mercutio

mm
my whole dsl died

brycec

ouch

mercutio

ikr

brycec

You didn't really need it though, the DSL that is, right? I mean... there are plenty of sheep.

dne

brycec: http://search.cpan.org/dist/Starman/

mercutio

haha

brycec

dne: yeah, that was #7 in my Google

mercutio

i dunno what happened yet
or how many people dropped, but it wasn't just me i know
my ssh stayed up :)

brycec

I want to add historical traceroutes to my smokeping... But I can't decide on a user interface :(

[00:50]

mercutio

damnit i think it dying again :(

[00:51]

brycec

up_the_irons: one other note: the v6 connectivity on Chunkhost, Comcast, and TWC is provided by HE tunnels. So the data may be "skewed" a bit :(

[00:52]

up_the_irons

brycec: ok

[00:53]

brycec

(Sorry, not much I can do about "native" v6 connectivity on those)

[00:54]

up_the_irons

yeah
lots of packet loss on HE lon2 and ams1 nodes right now

[00:57]

mercutio

apparently that ddos attack thing was bigger in europe than the US
i didn't really notice anything with that ddos fwiw
other than people talking about it :)

[01:01]

brycec

y'all are the only I've heard of it :p

BryceBot

<mercutio> i ofap oetppfip fp e kitgpebiu pi p:)

brycec

damnit perl

mercutio

what

mike-burns

Did we just get Klingon'ed?

brycec

y// in perl meets sed's "use any symbol as a delineator"

mercutio

haha

brycec

so y''
But I'm blaming perl for y//. <3 sed

up_the_irons

y'all are smokin' crack

[01:05]

BryceBot

<brycec> ButaI'mabrcmingapekrafokay//.a<3ased

[01:05]

up_the_irons

YES

[01:05]

brycec

well done. Now just lie on your side, the paramedics are on their way

[01:06]

..... (idle for 21mn)

up_the_irons

my gawd, the mail gem rocks
https://github.com/mikel/mail

[01:27]

ant

woah...that was a lot of scrollback

[01:38]

mercutio

sometimes the channel is idle for a whole day basically..
not recently though

[01:39]

up_the_irons

yeah, lately there's been a lot of activity

[01:41]

ant

some more lines and it had maxed out my scrollback buffer

[01:42]

up_the_irons

haha

[01:43]

mercutio

i found weechat is automatically logging
makes it easier to search for urls..

[01:45]

brycec

lol

[01:47]

.......... (idle for 48mn)

up_the_irons: Is there some reason I can ping my gw 174.136.103.129 but traceroutes fail? (apologies if it's a dumb question)
up_the_irons: I also have weirdness trying to traceroute to arpnetworks.com (I get just the first hope 174.136.103.129 and everything else times out)

[02:35]

up_the_irons

brycec: hmmm not sure

[02:36]

brycec

(ipv4 btw)
v6 seems fine

[02:36]

up_the_irons

paste mtr report

[02:37]

brycec

mmkay
hm, mtr is happy, traceroute is not.
brycec is up way past bedtime :/
Makes me think I should be using mtr instead of traceroute in my logging
up_the_irons: yep, traceroute -I works (ICMP ECHO)

[02:37]

mercutio

traceourte is udp
oh
i'm slow

[02:40]

brycec

heh
me too
apparently

[02:41]

up_the_irons

i always thought traceroute used icmp
ya learn something new every day!
ok, udp by default

[02:41]

ant

using udp allows one to use traceroute as a regular user (at least under linux)

[02:43]

brycec

and without setuid'ing
as mtr is

[02:43]

ant

yeah

[02:44]

mercutio

it does on windows
i think

[02:46]

up_the_irons

so, interesting:
regular 'traceroute', for v4, uses UDP
but for v6, uses ICMP6
wait, n/m
i'm wrong
UDP for both

[02:47]

mercutio

heh
i usuaully use mtr --report btw bryce
it works pretty well

[02:48]

up_the_irons

brycec: anyway, the answer is, iptables on arpnetworks.com server is blocking something. i can see the traceroute if i disable it.

[02:49]

mercutio

cos often you want more than 3 pings to a destination even when doing a traceroute
if there's an actual issue..
if you just want to know the path, then mtr --report --report-cycles=1 is probably faster
traceroute takes ages while it times out
the only problem with mtr --report is that it cuts off the end of hostnames
and some hostnames don't map back to ip's
but you can use -n as well to get around that
often when tehre are actual outages, there'll be looping etc, or changed paths
while bgp reconverges
also for some reason mtr --report can often drop a single packet

[02:49]

brycec

mercutio: actually my mtr line for logging is: mtr --report --report-wide --report-cycles 1 -o "N " google.com

mercutio

so if there's 10% packet loss it doesn't relaly mean anything

brycec

--report-wide ;) 02:49:10 < mercutio> the only problem with mtr --report is that it cuts off the end of hostnames

mercutio

oh i haven't tried report-wide
it still doesn't fix the issue that some things don't resolve

brycec

I'm not concerned about actual ping times or loss in this case, just looking to record the route.
heh

mercutio

ok
i mean forwrad lookup
it does reverse lookup but some things only work in one direction
traceroute shows both

brycec

traceroute also does AS lookup :)

mercutio

true
but you shouldn't need that

Yep. http://www.itnews.com.au/News/372033,worlds-largest-ddos-strikes-us-europe.aspx

brycec

true, but I think it's neat

[02:54]

mercutio

whois.radb.net/whois: Servname not supported for ai_socktype
hmm
wonder why my -A option isn't working
in theory asn support could be added to mtr

[02:54]

................ (idle for 1h16mn)

ant

i'm wondering where's the issue here: http://pastebin.com/nQDqXDuc . is it ntt? or telia?

[04:11]

up_the_irons

ant: i can reach it over ntt
from London

[04:15]

ant

hm..
brb lunch

[04:16]

ree

[04:28]

up_the_irons

reee

[04:29]

ant

ok. now i totally suspect telia
when i try it with nagios.teuchert.org (same machine, just another ip address from the same /64) instead of wnd1 everything is fine.
and i already had such issues in the past

[04:31]

up_the_irons

ouch

[04:32]

ant

now the issue is gone

[04:39]

........ (idle for 35mn)

***

toddf has quit IRC (Quit: leaving)

[05:14]

......... (idle for 42mn)

toddf has joined #arpnetworks
ChanServ sets mode: +o toddf

[05:56]

....................... (idle for 1h53mn)

abthorpet has joined #arpnetworks
mjp_ has joined #arpnetworks
KDE_Perr1 has joined #arpnetworks
DaCa_ has joined #arpnetworks
KDE_Perry has quit IRC (*.net *.split)
DaCa has quit IRC (*.net *.split)
tabthorpe has quit IRC (*.net *.split)
mjp has quit IRC (*.net *.split)

[07:49]

.................. (idle for 1h28mn)

brycec

[09:25]

............ (idle for 56mn)

toddf

note that cloudfare serves both coinbase and btc-e

[10:21]

brycec

Among many others :p

[10:23]

.................. (idle for 1h26mn)

***

ant has quit IRC (Ping timeout: 245 seconds)

[11:49]

ant has joined #arpnetworks

[12:02]

................. (idle for 1h24mn)

pjs has quit IRC (Read error: Connection reset by peer)
pjs has joined #arpnetworks
pjs is now known as Guest53036

[13:26]

mercutio

brycec: news articles were a bit skimp on the ddos

[13:38]

brycec

Not surprising

[13:38]

mercutio

itnews.com.au is the first one i read

[13:38]

brycec

It was linked off /.

[13:39]

mercutio

a lot of others were like 12 hours later or osmething
ahh ok someone posted to nznog
which is like nanog but for new zealand with less posts :)

[13:39]

............... (idle for 1h10mn)

***

RandalSchwartz has joined #arpnetworks

[14:49]

RandalSchwartz

weird. Why won't my arp server connect to freenode?

[14:49]

m0unds

any specific server?

[14:49]

RandalSchwartz

tried a few

m0unds

i'm connected now via ipv6
to asimov.freenode.net

RandalSchwartz

I'll have to try again later

m0unds

wait, no
i'm on v4
haha

***

m0unds has quit IRC (Quit: reload)
m0unds has joined #arpnetworks