#arpnetworks 2015-09-05,Sat

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***Guest88801 is now known as easymac
easymac is now known as Guest55375
[00:48]
............. (idle for 1h0mn)
Guest55375 is now known as easymac
easymac is now known as Guest45837
[01:49]
............. (idle for 1h1mn)
Guest45837 is now known as easymac
easymac is now known as Guest61474
[02:50]
............. (idle for 1h0mn)
Guest61474 is now known as easymac
easymac is now known as Guest79080
[03:50]
............. (idle for 1h0mn)
Guest79080 is now known as easymac
easymac is now known as Guest90147
[04:51]
............. (idle for 1h0mn)
Guest90147 is now known as easymac
easymac is now known as Guest69670
[05:52]
.... (idle for 16mn)
staticsafemercutio: you around? [06:08]
mercutioyip
what's up
[06:08]
staticsafemercutio: my v6 prefix's announce disappeared sometime last night, as far as I can see the prefix is exported on my end. Can you take a look?
2620:98:4001::/48
[06:09]
mercutiook
yeah that is curious
i can see 4000 but not 4001
but 4000 isn't going via arp
[06:09]
staticsafei can reach it from my nagios instance indicating some peers are seeing it
(softlayer)
[06:14]
mercutioso he.net isn't seeing it [06:14]
staticsafeyeah [06:14]
mercutiois the /48 just advertised via ape
arp even
[06:15]
staticsafeyeah [06:15]
mercutiowith having no path via a supporting /32 or such
i think there's some route object issues with your subnet getting out in general
and he.net's been having issues
hmm
so 2620:98:4000::/48 isn't advertised by arp
[06:15]
staticsafeno, its advertised elsewhere [06:21]
mercutioyeah it seems to be advertised anyway, due to the way the filters are setup
this needs fixing :)
do you want the ability to advertise 4000 still?
[06:22]
....... (idle for 31mn)
***Guest69670 is now known as easymac
easymac is now known as Guest71394
[06:53]
............. (idle for 1h1mn)
Guest71394 is now known as easymac
easymac is now known as Guest55703
[07:54]
............. (idle for 1h1mn)
Guest55703 is now known as easymac
easymac is now known as Guest30476
[08:55]
............. (idle for 1h1mn)
Guest30476 is now known as easymac
easymac is now known as Guest13865
[09:56]
............. (idle for 1h0mn)
Guest13865 is now known as easymac
easymac is now known as Guest97469
[10:56]
............. (idle for 1h0mn)
Guest97469 is now known as easymac
easymac is now known as Guest78531
[11:57]
............. (idle for 1h1mn)
Guest78531 is now known as easymac
easymac is now known as Guest1206
[12:59]
............. (idle for 1h0mn)
Guest1206 is now known as easymac
easymac is now known as Guest36637
[13:59]
............. (idle for 1h0mn)
Guest36637 is now known as easymac
easymac is now known as Guest85919
[15:00]
............. (idle for 1h1mn)
Guest85919 is now known as easymac
easymac is now known as Guest89094
[16:01]
............. (idle for 1h1mn)
Guest89094 is now known as easymac
easymac is now known as Guest46254
[17:02]
...... (idle for 28mn)
sjackso has quit IRC (Quit: leaving) [17:30]
...... (idle for 26mn)
m0unds has quit IRC (Quit: WeeChat 1.1.1)
m0unds has joined #arpnetworks
[17:56]
Guest46254 is now known as easymac
m0unds has quit IRC (Quit: stupid tmux)
[18:02]
mercutiotmux is great, dunno what he's talking about :) [18:03]
***easymac is now known as Guest91502 [18:03]
staticsafehrm, my prefix is gone again [18:04]
mercutiooh? [18:04]
staticsafeyeah [18:04]
mercutiodamn hangon
should be back
trying to fix these filters up better
but maybe i shoudl wait until a bit later
[18:04]
staticsafe:o [18:07]
brycecThe only time tmux has ever fouled me up beyond recovery was when I upgraded tmux and couldn't reattach to an existing session running the previous binary. (And even then, the interim solution was to downgrade) [18:15]
m0unds_i'm having refresh issues w/tmux
on one particular client, not on another
so it's annoying as hell
so..in this instance, annoying and not so great
[18:16]
brycecWhat exactly is a refresh issue in tmux? [18:16]
m0unds_when you switch between windows and it redraws the content in it
like..if text is scrolling or something
[18:16]
brycecMmhm
never had an issue that redraw/switching windows didn't fix
[18:16]
m0unds_switch to the other window, slow redraw
switch back, slow redraw
it's irritating
[18:17]
brycecI blame IPv6 >.> [18:17]
m0unds_it's like i'm running it over a serial connection [18:17]
brycec<.< [18:17]
m0unds_haha, seems legit [18:17]
mercutiothat sounds like a terminal issue
are you using putty or something
[18:17]
m0unds_doesn't matter [18:17]
mercutiooh hangon i've seen something like that before m0unds. [18:17]
m0unds_i've connected to it different ways, behaves the same regardless [18:17]
brycecin light of recent network issues it's legit. (though not to worry, not seeing anything at this moment :) ) [18:17]
m0unds_it's not on arp :) [18:18]
mercutiosome terminal was insanely slow with radeon view drivers.
s/view/video/
[18:18]
BryceBot<mercutio> some terminal was insanely slow with radeon video drivers. [18:18]
mercutioand i seem to remember tmux did something that made it slower [18:18]
m0unds_this client (a raspi) is fine [18:18]
mercutioprobably something weird with the way it does background colours or some other less common thing [18:18]
brycecprotip: Don't PM brycec, he won't see it for at least 20minutes :p [18:21]
mercutioheh
i used to have huge issues with notion and radeon video drivers.
whenever i resized windows using line drawing it was painfully slow
the fallback path for unaccelerated stuff is shocking.
[18:21]
brycecfwiw mercutio, trace from my home to ARP looks to be taking the same route as it has before. But at the moment, mostly clean. Dropped 3 out of 300 packets. [18:22]
mercutioit's the path from arp to you that may be different atm [18:22]
brycecmmkay I'll look in a sec :)
I can see the same nightly IPv6 abuse on ARP starting again in Smokeping https://smokeping.cobryce.com/?target=ARP.Upstream6 (that's from my ARP VPS to ARP's v6 router [i believe])
[18:23]
mercutioheh
there is another change to come
[18:24]
brycecdun dun DUN [18:25]
mercutioyeah the nightly abuse is pretty bad
it seemed to start a few minutes late tonight
[18:25]
brycecI do see packet loss the first hop after ARP (ge-101-0-0-13.r04.lsanca03.us.bb.gin.ntt.net) (my mtr to the feed:dead address) [18:26]
mercutioi saw 98% loss to that hop
i think it's gone some chronic icmp deprioritisation
s/gone/got/
[18:26]
BryceBot<mercutio> i think it's got some chronic icmp deprioritisation [18:26]
brycec80%+ over the last 2 minutes [18:26]
mercutiothe hop one after was fine though right [18:27]
brycecYeah could be (I'm well aware of that pifall)
Right
[18:27]
mercutioyeah [18:27]
brycecAnd there's another ntt.net hop that's lossy, but through to the destination is clear.
mercutio: I don't suppose ARP can warn the customer that's beating up the v6 router or anything, can they?
[18:27]
mercutiofor doing traffic? [18:29]
brycecFor disrupting service? [18:29]
mercutioyeah i'm not sure
it's not like they're ddosing
[18:30]
brycec(Or maybe ARP just needs to get the Cisco v6 router going...) [18:30]
mercutiojust heavy usage.
improving the router situation is important medium to long term
not having dropouts is important short term
[18:30]
brycecThe Acceptable Use Policy section 2.2 and 2.3 could be broadly interpreted as being violated :P
But like I've said before, I don't care too much, I just connect over ipv4 and carry on.
[18:31]
mercutioheh [18:33]
brycec(even if I would be happier with v6... and not running up my bandwidth quota :p) [18:33]
mercutioi'm sure you're nowhere near your quota [18:33]
brycecwhoa I've used 25% of my monthly quota actually
208.68/800GB
but that's also normal
[18:34]
mercutio25%
nothing to worry about
[18:34]
brycecfor me. Not exceptionally high due to all my terminal traffic now going over v4 :P [18:34]
mercutioheh [18:34]
brycecIf I was worried, I'd tunnel through my employer's dedicated with 12TB or whatever it is
(Yep, 12TB of bandwidth)
[18:34]
mercutioit's 10TB on arp metal [18:35]
brycecWe have 12TB
And 504GB used this month
[18:35]
mercutioheh [18:35]
brycecAll-time we've used only 3.4TB [18:36]
mercutiowow
i think i've used more than that personally
[18:36]
brycecI thought so too personally, but my own yearly usage was <2TB
And in our defence, we weren't using that host until July
[18:37]
mercutio56.11GB/year [18:38]
brycecmercutio: can you confirm which is inbound/outbound on the graphs - is inbound internet->VPS/dedi? (or is it relative to the switch port, inbound is host -> switch)
brycec is wondering why he has a 200Mbps spike"in"
*spike "inbound"
[18:38]
mercutioi think it's reversed [18:39]
brycecokay that would make sense to me then (uploading a chunk of data) [18:39]
mercutiobecause it's monitoring the router rather than the host
but don't hold me to that :)
[18:39]
brycecIf you really didn't want to be held accountable for that remark, should've started your message with [OFF] :P
Now we have logs....
[18:40]
mercutioor just say i think
rather than i know :)
[18:40]
brycecstatus update on my home HE -> ARP VPS mtr: 17 dropped out of 1500.
So, pretty quiet
[18:41]
mercutioi'll be back in a bit
sweet that sounds way better
i'll check it agian in a bit :)
[18:41]
brycecbrycec reconnects over ipv6
we'll see how this goes... assuming I'm at my tmux to notice
[18:42]
..... (idle for 21mn)
***Guest91502 is now known as easymac
easymac is now known as Guest36744
[19:03]
............ (idle for 55mn)
m0unds has joined #arpnetworks [19:59]
Guest36744 is now known as easymac
easymac is now known as Guest44560
[20:05]
.......... (idle for 45mn)
brycecblargh dropped a few packets just then
but still better than the other day
[20:50]
m0undsyea, was pretty bad [20:51]
mercutiohmm i'm seeing a bit of bursts of jitter up to like 25 msec [20:52]
m0undsi think it's happened a few times in the last mo or so [20:52]
mercutiothen the next second it's fine
when you say drop a few packets you mean like jittery ssh not responding quickly then fine?
[20:52]
brycecbrycec only noticed it because he was trying to change tmux windows at the time and it just wouldn't respond [20:53]
mercutioor like it feels like it's dropping out. [20:53]
brycecmercutio: I mean mtr says packets were dropped
And also ssh was non-responsive
[20:53]
mercutioa few in a row? [20:53]
brycecHard to say
Yes
[20:53]
mercutiowell m0unds noticed it too [20:54]
brycec(thank you mtr "Display Mode") [20:54]
mercutioheh
yeah when i use mtr i can glance at it cos i see the colour change
although it wasn't on screen atm
[20:54]
brycecI'm seeing 4 packets here and there over the last 90 seconds (12 total)
brycec goes back to abusing Syncthing
[20:54]
m0undshaha [20:55]
mercutiothat seems like heaps
ahh
i'm seeing loss to www.he.net
but no loss to the gateway
actually i'm not sure if it's loss to or from
[20:55]
brycecI should add, mercutio, the the loss I'm seeing is on my home HE tunnel -> ARP VPS, starting at the ntt hop (one and only, second to last hop) and continuing to the last hop of my VPS
Another burst of loss there and there
[20:56]
mercutioyeah i'm still seeing loss to www.he.net
i'm trying to find a host site
for some reason uk vultr seems down :(
i thought that maybe on he
[20:56]
brycec10 packets in a row just then :(
Cloudflare has native IPv6 IIRC
could test to them
[20:58]
mercutiocloudflare doesn't cache my file properly
i was hoping to use iperf to test direction
[20:58]
brycecI figured you were just pinging/mtr'ing [20:58]
mercutioi was testing with vultr before
but that was hitting gtt
[20:58]
brycecah [20:59]
JC_Dentonanyone here using freenas? [21:00]
mercutiomy route over ipv4 ius hitting he.net to vultr uk
so hopefully it'll help :)
weird it's hitting trit for this route
[21:00]
brycecJC_Denton: Yes [21:01]
m0undswhen it was worst for me was when my v6 route from home traversed he
when it goes over just ntt, it's fine
[21:01]
mercutiom0unds: i think there's at least double issues atm
and one potential issue :)
[21:02]
m0undsdaily, haha [21:02]
brycecmy conclusion the other day, m0unds, was that it is a particular HE-NTT handoff that is over-burdened. [21:02]
mercutioi mean of separate issues [21:02]
m0undsyeah [21:02]
JC_Dentonbrycec: know off the top of your head if the ZFS setup on it supports POSIX ACLs? [21:02]
m0undsbrycec: yeah, that's how it looked to me [21:02]
mercutiom0unds: but he.net direct was screwed too right?
i mean brycec
[21:02]
m0undsno [21:02]
brycecJC_Denton: hm no I don't [21:02]
m0undsbut my route changed off he->ntt before i could really dig into it at all [21:03]
brycecmercutio: can you clarify what you mean by HE direct? [21:03]
mercutiohe.net to he.net
when not hitting ntt was also breaking
[21:03]
brycecJC_Denton: IIRC xacl support is in the underlying ZFS (because FreeBSD) but I never tested it.
mer
DAMMIT IPV6
brb
[21:03]
m0undshahahahah [21:04]
brycecI'm back to v4. Trying to chat with y'all is impossible :p [21:04]
mercutiothis trit route is like 10 megabit tcp to uk [21:04]
m0undsneed moar missed keystrokes pls [21:04]
brycecmercutio: How would I test HE-HE? ARP traffic always crossed NTT. [21:05]
mercutiooh really? [21:05]
brycecm0unds: bless SSH/TCP it always kept up. [21:05]
m0undsbrycec: are you hitting lax2.he.net?
when you're enroute to arp via your tunnel
[21:05]
***Guest44560 is now known as easymac [21:05]
brycecm0unds: yes [21:05]
m0undscheck this out [21:05]
bryceclax2.he -> lsanca.ntt [21:05]
m0undshttp://pastebin.com/eLLZEMkP [21:06]
brycec(and it's that lsanca hop that drops) [21:06]
mercutiothis is ntt->cogent and level3->trit and seems ok in both directions [21:06]
***easymac is now known as Guest39524 [21:06]
m0undsthat's LAX2-> comcast in NM, and LAX2 -> arp [21:06]
brycec(hence my conclusion that it's the lax2he->lsanca.ntt hop that is saturated) [21:06]
m0undsyup [21:06]
mercutio2001:418:0:5000::52c is the ntt hop i was seeing drop when it was fine before [21:07]
brycecI see 52d in my current mtr [21:07]
mercutioprobably the other end of such :)
but yeah that alone seems to mean nothing
[21:07]
brycecor could be another parallel route [21:08]
mercutiocos 2001:418:0:2000::1ee is fine [21:08]
brycec(that hop doesn't even appear in my mtr) [21:08]
mercutiohah why is he.net looking glass showinghigh pings for every hop
this is outbound from arp over ntt
[21:09]
m0undslax1->lax2 is fine
lax2->lax1 is fine (he to he)
it's that ntt hop borking stuff
because he->comcast on ipv6 is shitty (goes via ntt) and he->arp is shitty (also goes via ntt iirc)
[21:09]
mercutio2001:470:0:6c::1
what's your trace to that ip like?
[21:11]
brycecfor me, sea1.ipv6.he.net -> sea1.he.nt -> sjc2.he.net -> lax2.he.net no loss (from my home tunnel)
(obviously)
[21:13]
mercutioyeah and that's screwed from arp [21:13]
brycecAnd from ARP https://dl.dropboxusercontent.com/u/3167967/screenshot_2015-09-05_21-11-23.png [21:14]
m0unds216.218.226.238 tracerouting to this via v4 looks awful, goes through any2ix
(tunnel server in seattle)
[21:19]
mercutiois there something special that makes www.he.net special? [21:21]
m0undswhat do you mean? [21:21]
mercutiowith loss [21:21]
m0undsis it not bad?
it doesn't go through any2ix
[21:21]
mercutioit is bad [21:22]
m0undsnm
it does
[21:22]
mercutiobut it seems to be bad suddenly [21:22]
m0undsjust no rdns on the hop
any2ix to HE sucks from arp
every single one i've tried
[21:22]
mercutioso i'm wondering if it's rate limit
so it's always been bad?
and ntt->he is bad
[21:22]
m0undsno, don't think it always has [21:24]
mercutiobut for quite a while? [21:25]
m0undsmaybe, i don't usually see he on my route
but when i saw tons of pkt loss w/unreliable ssh the other night, it involved he
and that ntt hop brycec mentioned
and any2ix on either the return or other path
[21:25]
mercutioi'm looking for more looking glasse
i foundm something that's slow, but it's even slower over ipv4
and fine from linux, go figure
[21:26]
m0undshuh [21:37]
mercutiohe.net's giving issues again
i'ev found dns server as something beter to ping
[21:37]
.... (idle for 18mn)
and it's he.net incoming rather than outgoing that's the issue [21:56]
***Guest39524 is now known as easymac
easymac is now known as Guest13147
[22:06]
........... (idle for 54mn)
mnathani2_s/beter/better [23:01]
BryceBot<mercutio> i'ev found dns server as something better to ping [23:01]
mercutioheh [23:01]
mnathani2_hey where did the 2 come from? [23:01]
mercutioi got my own tunnel
can test easy now :)
[23:01]
***mnathani2_ is now known as mnathani_ [23:01]
mercutioprobably from another netsplit
when it says so much loss with mtr, it's actually single direction
[23:01]
mnathani_there, thats better [23:02]
mercutioit's traffic into arp via he.net that's the issue [23:02]
mnathani_via any2ix? [23:02]
mercutioand that ntt hop is return path, where arp sends traffic out, so it's not the issue
yeah
[23:02]
mnathani_congestion? [23:03]
mercutioi don't think it's actually congestion.
i'm not sure what it is though
it's like suddenly packets stop coming through
and then it's fine again
[23:03]
***Guest13147 is now known as easymac
easymac is now known as Guest52803
[23:08]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)