mercutio: if it's not tweaking every little site but only the bigger providers then it's simpler config wise
and if it's that the as path length is the same for both, it could hop randomly if the weight is the same ***: jcv has quit IRC (Quit: leaving)
jcv has joined #arpnetworks
jcv has quit IRC (Client Quit)
jcv has joined #arpnetworks
ziyourenxiang has joined #arpnetworks
jcv has quit IRC (Quit: leaving)
jcv has joined #arpnetworks
ziyourenxiang has quit IRC (Quit: ziyourenxiang) m0unds: up_the_irons: 14.04 here, but mine are in enforce mode ***: novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks
novae has quit IRC (Changing host)
novae has joined #arpnetworks acf_: up_the_irons: fwiw, I emailed the NTT NOC a long time ago
idk if you've seen it
http://paste.unixcube.org/k/bb55b1 m0unds: ... http://dogeon.org/ ***: Amfy has joined #arpnetworks
Mifta has joined #arpnetworks
Mifta has left
toddf has quit IRC (Ping timeout: 252 seconds)
toddf has joined #arpnetworks
ChanServ sets mode: +o toddf acf_: it seems that the return path of level3 (level3 -> arp) goes through gblx and trit
http://paste.unixcube.org/k/e0c7ac
mercutio: I would think that since ARP and level3 are peered that it wouldn't go through trit m0unds: acf_: i've seen people mention that it's only outbound l3 right now
http://pastebin.com/k3meCyuJ acf_ - from a couple days ago ***: novae has quit IRC (Ping timeout: 245 seconds) acf_: cool, thanks, didn't see that m0unds: sure thing, had to dig through my buffer to find it. wondered if i'd just imagined seeing it :) ***: novae has joined #arpnetworks brycec: I too recall up_the_irons stating it's outbound only now, that ARP isn't yet announcing.
A nice gradual turn-up m0unds: http://urbanvivant.com/wp-content/uploads/2010/12/turnip.jpg
yay. my headphone amp was repaired and is on its way back to me.
onboard audio makes me sad ***: httpduck1 has joined #arpnetworks
httpducks has quit IRC (Write error: Broken pipe) up_the_irons: m0unds: turns out i had the security driver set to "none" in the config; so commenting that out fixed the prob
acf_: i haven't announced our routes to level3 yet
it's all outbound
oh, all you guys already said outbound... -: up_the_irons needs to read more scrollback before responding staticsafe: :) up_the_irons: acf_: read your email; so i take it Verizon didn't respond when you contacted them? m0unds: up_the_irons: ah, gotcha
up_the_irons: noticed any difference in CPU load on the updated hosts? up_the_irons: slightly lower brycec: I was working to counter that last night with a git-gc on a 1.5GB repo... Then again, i'm not on an updated host yet up_the_irons: haha acf_: up_the_irons: yep. basically Verizon won't talk to you unless you're a transit customer or something
that's been my experience with them anyway up_the_irons: aholes acf_: yeah
I don't think they could/would have told me any more than NTT anyway though mercutio: up_the_irons: how do the as path lengths compare with verizon/comcast/etc? up_the_irons: mercutio: as path lengths of what? mercutio: i do wonder if a lot are the samw
verizon ip's
umm
whatrever acf is tracing to
comcast.net is a good comcast example apparently
i can't remember acf's traceroute site :) acf_: http://kremvax.acfsys.net/smokeping.cgi?target=Remote.verizon-lsanca mercutio: that's a router
verizon.net
you shouldn't really make assumptions about routers cos of deprioritisation etc acf_: verizon.net actually takes a different return path mercutio: hmm acf_: if you look at the last 360 days graph mercutio: but yeah i do wonder if as path length is the same
hmm i have some verizon ip i'm smokepinging from nz -: acf_ goes to plug in DSL modem mercutio: hmm there's loss atm acf_: there's always loss mercutio: from arp
but not from nz acf_: yep
oh mercutio: yeah from nz it's going via alter.net
which is verizon right? acf_: yep
what is going via alter.net? mercutio: from my home
oh sweet kvr15 update is happening :)
i wonder whose ip this was
ae-0.verizonbusiness.lsanca03.us.bb.gin.ntt.net
the packet loss seems to start there acf_: same from arp mercutio: this is from arp
from home there's no loss :) acf_: ah, ok ***: grepidemic has quit IRC (Read error: Connection reset by peer)
mus1cb0x has joined #arpnetworks mus1cb0x: how can i find out if my vps has acpi enabled? mercutio: i think they all do
unless it's dos or something
what OS? mus1cb0x: fbsd 9.2 mercutio: in openbsd it shows up in dmesg
yeh that should be fine
did you get emailed too? :) mus1cb0x: yeh
i guess we're server brothers mercutio: heh up_the_irons: LOL mercutio: he did say he was doing two servers mus1cb0x: i'm on 14 up_the_irons: i'm actually doing three now mercutio: im'm on 15
oh up_the_irons: i have some extra time next week, so might as well bust out the upgrades mercutio: up_the_irons: is it getting easier? mus1cb0x: 14 > 15 m0unds: neighbors then up_the_irons: mercutio: it IS actually.. last few upgrades have went very smoothly, even faster than my estimated time in the maint advisory ***: grepidemic has joined #arpnetworks mercutio: i always overestimate times myself
just in case anything goes wrong
and because i usually underestimate by default up_the_irons: yeah mercutio: we can't use virtio until there's a knob though right?
well without manual support request mus1cb0x: you can modify the host OS without it affecting the disk partitions of the vpss? mercutio: mus1cb0x: that's normal for virtualisation hosts.
the two are disconnected mus1cb0x: i wonder how deep you can nest that before things go weird
imaging a tight loop programmatically creating nested guest environments m0unds: some will run openvz inside kvm
qemu mercutio: ad infinitium i imagine mus1cb0x: you could send a chinese hacker into a mirrorverse of insanity mercutio: you'd probably run out of memory though m0unds: the extra hw abstraction can help you work around bad hw support in kernels compatible w/openvz mercutio: qemu doesn't need privileges for emulation
it also supports non intel cpus m0unds: yes, i run qemu on an opteron server mercutio: probably quite slowly.
m0unds: no, i mean like it can run on powerpc etc
and can emulate 68k etc m0unds: oh, non x86 mercutio: xen is using qemu too
but qemu is actually supporeting lots of weird stuff, it's damn cool in a way m0unds: not a big fan of xen-pv mus1cb0x: bhyve is getting some nice upgrades this summer mercutio: xen has a whole lot of new modes now
like hybrid stuff
modern cpus are not so quick in pv mode m0unds: yeah, but lots of hosts that run xen environments won't run w/hvm or whatever mercutio: yeah there's somethign in between too m0unds: so you're stuck with linux or obnoxiously unstable bsd mercutio: which is better
well yeh i chose arp becauase of kvm and openbsd
err
kvm and ipv6 m0unds: yep mercutio: because i wanted to run openbsd mus1cb0x: by openbsd i assume you mean freebsd m0unds: right mercutio: i don't use ipv6 though mus1cb0x: ;> mercutio: oh location was theo ther reason mus1cb0x: haha mercutio: i'm in new zealand, los angeles and san jose are the only two fsat places from here really m0unds: i have a silly amount of unused referral credit w/another provider that i won't use until i can get bsd stable in their env mercutio: well low latency
i've run freebsd on xen fwiw m0unds: it runs fine in hvm mercutio: and openbsd on vmware
it runs in pv mode too m0unds: it's terrible in xen-pv when your only option is pvgrub to bootstrap bsd
it's awful mercutio: but it runs better in hvm mode
i was using pygrub i think?
it was on opensolaris
which doesn't have pvgrub , and is xen 3.4
oh having it's own cp is good too
i am kind of anti solusvm m0unds: yeah, solusvm is awful
the way it handles assigning ipv6 addresses is hilarious
you don't get a routable /64 or whatever, you get X number of addresses
so...8 or 16
or whatever the host chooses mus1cb0x: mercutio: i'd like to visit nz some day
m0unds: sounds very mcdonalds mercutio: mus1cb0x: heh
m0unds: it's also got lots of funky mysql and iptables stuff
if you want to do thigns like limit people to their own ip it's complicated by solusvm getting in the way with iptables
which means most solusvm hosts allow spoofing
most are just huge bridge domains too
so you get arps for a whole lot of random ip's acf_: Comcast cable kind of does the same thing
you see lots of arps for random IPs mercutio: can you spoof? m0unds: i think that's just the nature of cable networks acf_: no m0unds: no mercutio: how many /24s is it? acf_: well, my netmask is /24 mercutio: cable network in nz overloaded acf_: and if I try to ping things in the same /24 mercutio: and routers couldn't handle the number of arp requests
there was no standard router provided acf_: the cable network gateway will spoof its mac address for the ARP replies mercutio: but it meant lots of people's internet died completely in evening
yeah i had /24 on cable
it was nice that you could just plug it into a pc acf_: same with Verizon DSL here kind of mercutio: set an ip and gateway and you were off
verizon is vdsl now right? acf_: except Verizon caches your MAC address for DHCP, so you can't change computers without calling them first
or spoofing the MAC of course mercutio: with some shitty router that everyone puts their own one after acf_: no, ADSL
2 mercutio: oh weird
i'm sure i had a friend who went comcast -> verizon -> comcast
comcast was higher latency apparently acf_: that's kind of hard to believe mercutio: and only like 30 megabit
he was in kansas acf_: Verizon DSL sucks crap m0unds: i think vz can offer vdsl2 if you're close to their gear mercutio: oh m0unds: like..sub 1000ft loop length mercutio: i meant verizon was oops
sorry! acf_: I'm less than that from their CO mercutio: and comcast was 50 megabit i think? m0unds: old dslams maybe in that facility
where in the US are you? acf_: central coast of california
93455
old GTE territory m0unds: huh, weird - business i used to work at had vdsl service in huntington beach area ca mercutio: heh i have friend in san jose, and he said comcast was pretty bad
but it's basically comcast or at&t
i suppose comcast is better than at&t m0unds: comcast is in the middle of doubling residential speeds again acf_: at least they actually upgrade their network sometimes mercutio: he has heaps of jitter
and really high pings to cogent
it's about 50 msec pings to san jose from san jose
over cogent acf_: peering congestion you think? mercutio: this was a while ago m0unds: the routing is probably goofy mercutio: before the netflix peering
routing looked fine
it's probably the netflix/cogent/comcast dispute m0unds: wouldn't have surprised me if it went SJC>LAX>Cogent>SJC or something mercutio: for some reasno people call it the netflix issue
but it affects other traffic too acf_: my mtr to cogentco.com: http://paste.unixcube.org/k/efacc8 mercutio: nah it didn't m0unds: yeah mercutio: it did over 3mb/sec to los angeles though
on 25 megabit plan
mci
ok it wasn't that broken m0unds: hahaha mercutio: i've seen that from cogent to at&t
oh that's to cogent's website
try sj.meh.net.nz acf_: via nlayer mercutio: oh acf_: http://paste.unixcube.org/k/ea49f6 mercutio: ok that's bbetter
it was going via comcast before m0unds: cogent denver pop -> albuquerque, nm (460mi) is 55ms mercutio: maybe they stopped sending via cogent when netflix paid money m0unds: cogent LAX pop -> albuquerque = 27ms
800mi mercutio: hmm that ping is still kind of high though
what do you get to arp? m0unds: 28ms mercutio: he got lower ping than that to arp acf_: http://paste.unixcube.org/k/7cabf5 mercutio: i don't really know what santamaria is
he was in santa clara? acf_: right next to santa barbara mercutio: but they prob not close acf_: right in the middle of lax and sfo m0unds: nice area of ca mercutio: oh acf_: my Verizon traffic goes via lax m0unds: i've spent a fair amount of time in ventura & carpinteria for work mercutio: so the issue is that there's no peering there
and it has to go to sj/la
or both acf_: and my Comcast trafic goes via sj
yeah mercutio: weird google says santa marisa is in crazil
santa maria brazil
i can't type m0unds: http://binged.it/1i9ZuWW BryceBot: http://binged.it/1i9ZuWW -> https://www.bing.com/maps/?v=2&where1=Santa Maria, CA&form=LMLTSN&cp=34.95372009277341~-120.4355697631836&lvl=12&sty=r&encType=1 mercutio: oh i see where it is now
that does look nice
it looks samll though m0unds: i like that area of ca a lot acf_: it's pretty nice :) m0unds: last time i went was last june, it was 65 every day
so nice mercutio: does it rain there? acf_: we're in a pretty bad drought right now
so not much mercutio: that's the most disturbing thing about california to me acf_: yeah, it is pretty bothersome
it was a lot better some years ago mercutio: wow
it rains over 100 times more here than there m0unds: a good friend of mine used to work for the UCSB NOC a few years back mercutio: 1,212.4 inches per year, compared to 14.01 inches per year m0unds: haha mercutio: so yeah it seems a ltitle disoncerting :) acf_: http://goo.gl/Bw1jei
the local CO BryceBot: http://goo.gl/Bw1jei -> https://www.google.com/maps/@34.883954,-120.418628,3a,75y,91.38h,85.04t/data=!3m4!1e1!3m2!1scvaM7PWqCG7UQ9E4kfPquQ!2e0 m0unds: wait..
1,212 inches = 100ft of rain mercutio: northern california rains more? m0unds: haha mercutio: tahts' per year m0unds: mm
wikipedia shows inches in parenthesis
1212.4mm mercutio: oh it's precipitation vs rainfall
is that the same thing m0unds: yes
the parenthesis is 47.732" mercutio: oh i fucked up didn't i
yes m0unds: backwards, lol mercutio: they swapped it aruond ***: mus1cb0x has left "WeeChat 0.4.2" mercutio: i was trying to use inches for US audience :/ m0unds: yeah, i think because you're metric and we're lame mercutio: sorry
well i expected wikipedia to be constant
ok that difference isnt' so bad
it's just 4x as much m0unds: we get 9.45 in/yr mercutio: i'd rather 14 :/ m0unds: you and i both mercutio: i dunno m0unds: haha mercutio: haha m0unds: i'm also in the desert mercutio: nice
when i was younger i thought it'd be cool to get a job in the US
but you need work visa and to work for one company for ages :/ m0unds: my house is at ~1800m above sea level mercutio: my friend did it by shifting within the same company to a diff country
but it ook him two attempts m0unds: yeah, it's a pain mercutio: there wsa some paperwork error or something m0unds: i have a friend who's finishing up her fellowships in the US as a pediatric orthopedic surgeon
and they're still not going to let her stay, she has to return to brazil at the end of the year mercutio: and you can't fix it you have to wait until next year? m0unds: yeah, it's a bureaucratic mess mercutio: damn m0unds: legal immigration into the us is gross mercutio: i wonder how he managed it m0unds: another friend got married in canada, and it took 6 months for his wife to legally immigrate to the US
and ~$5k in attorney fees mercutio: ouch
he must have done it properly because he bought a house m0unds: and hours upon hours at US consulates for meetings with immigration clerks and stuff mercutio: apparently there's ths really exepensive grooming thing for houses when they sell
and therew as like a video walk through and stuff
that doesn't happen here at all
but it was actualyl cheaper than i thought it'd be m0unds: huh mercutio: they show this video of the house
going through all the rooms
with music
and panning m0unds: hahaha, yeah, i've seen that before mercutio: and everything looks so neat and tidy
and sterile m0unds: yeah, they "stage" the house
sometimes they do it empty, sometimes they rent furniture and set it up so you can get an idea of what it'd look like if you lived there mercutio: oh that's the term yes ***: tellnes has quit IRC (Ping timeout: 252 seconds)
tellnes has joined #arpnetworks m0unds: oh, great
my smokeping graph looks like acf_'s verizon one
hahaha
yeah, seeing roughly 2x the latency at this hop 10. ge0-arpnet.cust.lax07.mzima.net
looks like it's been like this daily since the 3rd staticsafe: https://twitter.com/j_smedley/status/475023041106169856 :O BryceBot: TWITTER: looking good for a Waterson and Mattherson merge. (Fri Jun 06 21:14:53 +0000 2014, retweeted 50 times) mnathani: did anyone not on kvr15 get a maintenance announcement for kvr15 ? milki: i didnt notice anything mercutio: kvr14 is goiung down too
and somethjing else acf_: m0unds: mind revealing the source destination network/host?
/s/source destination/source/ mercutio: (i'm on kvr15 so don't know if they all say that) acf_: m0unds are you on Comcast?
http://kremvax.acfsys.net/smokeping.cgi?target=Local.phoenix
that is rather upsetting
crap now comcast <-> nlayer is bad...
or is it comcast <-> ntt?
my forward path is via nlayer
and return is via ntt
from Comcast's perspective
hmm if you're seeing loss at ge0-arpnet.cust.lax07.mzima.net
it makes me think ntt
because ntt becomes the return path at that hop
definitely NTT: http://paste.unixcube.org/k/65f903 m0unds: sorry, was commuting home - i'm on comcast biz
it's nlayer inbound from comcast -> arp
and return path is via ntt
and i think it's ntt
it stops being problematic at precisely midnight mdt acf_: more peering problems I assume... m0unds: guessing so
starts at the same time
ends at the same time acf_: I'll email NTT again
not that it'll help any m0unds: are you seeing this stuff starting at around 1530ish PDT?
it starts for me at 1630 mdt, ends at midnight +/- 5minutes acf_: yepyep
http://kremvax.acfsys.net/smokeping.cgi?epoch_start=1402038000;hierarchy=;epoch_end=1402124340;target=Local.phoenix;displaymode=n;start=2014-06-06%2000%3A00;end=2014-06-06%2023%3A59;Generate%21=Generate%21 m0unds: mine's just as smooth before that too, haha
very little jitter acf_: I've been noticing problems with voip lately
probably that m0unds: ugh acf_: up_the_irons: maybe you should see this
strange
Level3 appears to have a shorter AS path to Comcast than NTT
http://paste.unixcube.org/k/83497d
vs
http://paste.unixcube.org/k/6f24b0
or it's the same... I think I misread
but Level3 looks like NTT when it's congestion-free ***: novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks mercutio: acf: i found something from last year on google with ntt/verizon :/
it's probably the same acf_: meaning the issue has been ongoing for at least a year? mercutio: that's not as path
the AS path should end in 7922
with the network prior, like 3356 appearing
where 3356 is level3
like i can see routes for 701 and 3356
can't see what arp sees :/ acf_: hmm ok mercutio: and 701/703 is verizon
need a looking glass
well don't need, but would mean coudl see what they see
hmm
so verizon have 703 and 701 in the AS Path
and level 3 just has 3356
but it's interesting that it's both comcast and verizon having issues
i wonder if they're working together
because they both had netflix issues too
in a similar fashion ***: BryceBot has quit IRC (Excess Flood)
BryceBot has joined #arpnetworks