[01:22] *** adoyee has joined #arpnetworks [01:25] hi,all [01:40] hi [01:41] I can't login my VPS :( [01:42] it got hacked? [01:42] have you tried using vnc? [01:43] I can't ping it. can you help me to ping it? [01:45] we have some professional pingers in the channel [01:46] but yeah, does the VNC console let you in? [01:46] oh, really? I think my VPN was blocked by the fireworks [01:46] what? [01:47] firewall ? [01:47] ok what is your ip [01:47] for your vps [01:47] he's in china [01:47] so if he was vpn'ing via it, then it may have randomly got blocked [01:47] 174.136.103.90 [01:47] ah. yeah :/ [01:47] we can't ping it either [01:47] 100% packet loss [01:47] so it's probably down for some reason [01:48] if you go to portal.arpnetworks.com [01:48] and login [01:48] then click on vps [01:48] I have start it in portal [01:48] then it will tell you what your vnc server is [01:48] and what port [01:48] you don't usually need to start it, it should arleady be running [01:49] and if there is a problem it can be good to see what it thinks the problem is [01:49] how I can check it? [01:49] running or not [01:52] by using vnc [01:52] so you went to portal ? [01:52] and it says your vnc server and port? [01:53] ok. I need a vnc client. [01:54] ok [01:55] windows? mac? linux adoyee ? [01:55] I'm from China. u know, the firewall .. [01:55] mac [01:55] mac and linux [01:55] if you got a linux nearby [01:55] there should be a decent enough built in VNC client [01:56] yes, and irc clients [01:56] using irc for the first time [01:58] *** mike-burns has quit IRC (*.net *.split) [01:58] *** pcn has quit IRC (*.net *.split) [01:58] *** RandalSchwartz has quit IRC (*.net *.split) [01:58] *** meingtsla has quit IRC (*.net *.split) [01:58] *** kevr has quit IRC (*.net *.split) [01:58] *** KDE_Perry has quit IRC (*.net *.split) [01:58] *** mercutio has quit IRC (*.net *.split) [01:58] *** twobithacker has quit IRC (*.net *.split) [01:58] *** m0unds has quit IRC (*.net *.split) [01:58] *** adoyee has quit IRC (*.net *.split) [01:58] *** pjs has quit IRC (*.net *.split) [01:58] *** NiTeMaRe has quit IRC (*.net *.split) [01:59] *** adoyee has joined #arpnetworks [01:59] *** mike-burns has joined #arpnetworks [01:59] *** pjs has joined #arpnetworks [01:59] *** NiTeMaRe has joined #arpnetworks [01:59] *** pcn has joined #arpnetworks [01:59] *** RandalSchwartz has joined #arpnetworks [01:59] *** meingtsla has joined #arpnetworks [01:59] *** kevr has joined #arpnetworks [01:59] *** m0unds has joined #arpnetworks [01:59] *** twobithacker has joined #arpnetworks [01:59] *** mercutio has joined #arpnetworks [01:59] *** KDE_Perry has joined #arpnetworks [01:59] *** barjavel.freenode.net sets mode: +o mike-burns [02:01] *** BryceBot has quit IRC (*.net *.split) [02:01] *** up_the_irons has quit IRC (*.net *.split) [02:01] *** staticsafe_ has quit IRC (*.net *.split) [02:01] *** avj has quit IRC (*.net *.split) [02:01] *** carvite has quit IRC (*.net *.split) [02:01] *** m0unds_ has quit IRC (*.net *.split) [02:01] *** brycec has quit IRC (*.net *.split) [02:01] *** josephb has quit IRC (*.net *.split) [02:01] *** acf_ has quit IRC (*.net *.split) [02:01] *** jpalmer has quit IRC (*.net *.split) [02:01] *** eryc has quit IRC (*.net *.split) [02:01] *** raptelan has quit IRC (*.net *.split) [02:01] *** BryceBot has joined #arpnetworks [02:01] *** up_the_irons has joined #arpnetworks [02:01] *** staticsafe_ has joined #arpnetworks [02:01] *** avj has joined #arpnetworks [02:01] *** carvite has joined #arpnetworks [02:01] *** m0unds_ has joined #arpnetworks [02:01] *** brycec has joined #arpnetworks [02:01] *** josephb has joined #arpnetworks [02:01] *** acf_ has joined #arpnetworks [02:01] *** jpalmer has joined #arpnetworks [02:01] *** eryc has joined #arpnetworks [02:01] *** raptelan has joined #arpnetworks [02:01] *** barjavel.freenode.net sets mode: +o up_the_irons [02:02] *** adoyee has quit IRC (*.net *.split) [02:02] *** pjs has quit IRC (*.net *.split) [02:02] *** NiTeMaRe has quit IRC (*.net *.split) [02:04] *** adoyee has joined #arpnetworks [02:04] *** pjs has joined #arpnetworks [02:04] *** NiTeMaRe has joined #arpnetworks [02:06] oh it's back [02:07] hi~ [02:08] what happened, so many left [02:16] woo, freenode [02:16] net split [02:17] ha` [02:18] moments pls [03:04] netsplit [03:15] ~.`. [03:37] did you get sorted adoyee [03:39] hi, mercutio. I'm in office now. It's better check the vps in my private time. [03:39] ahh ok [03:54] *** adoyee has quit IRC (Remote host closed the connection) [03:55] *** adoyee has joined #arpnetworks [04:00] *** adoyee has quit IRC (Ping timeout: 265 seconds) [05:52] *** ziyourenxiang has joined #arpnetworks [06:18] *** ziyourenxiang has quit IRC (Quit: ziyourenxiang) [06:41] professional pingers, represent! [06:41] heh [06:41] i think NTT is just screwy - this only happened in the last few days [06:41] i can see it from work (centurylink) as well [06:41] uhh it was happening months ago [06:41] i'm talking about the thing with comcast and me [06:41] not the thing with verizon [06:41] oh [06:42] well, comcast and centurylink and me [06:42] well whenever we talked about it last time with some of us, verizon and comast were both a bit screwy [06:42] and ntt had had some los angeles routing oddities on top of thagt [06:42] where ntt via san jose wasn't as bad as ntt via los angeles [06:42] or osmething ilke that [06:42] yeah, but i wasn't seeing 20+% packet loss and 200% higher latency [06:42] oh [06:42] but there was 5% loss or something [06:42] based on time of day [06:42] yeah [06:42] reoccuring [06:43] hadn't seen that in months via comcast here though [06:43] oh ok [06:43] there was 20 msec higher latency before or something i think? [06:43] with the exception of that weird thing late last week, when kvr26 was experiencing tons of pkt loss [06:43] i don't actually even remember what month this was :) [06:43] yeh that wasn't ntt's fault [06:44] well, it's weird - one of my addresses ordinarily pings ~30ms and is routed via LAX directly off comcast to arp [06:44] but other ones are routed to SJC via denver and ping at 50ms or so [06:44] have you tried iperf? [06:44] nah [06:44] throughput is fine - i tunnel ipv6 traffic through this vm [06:45] ahh ok [06:45] you can figure out direction of loss [06:45] i'm reasonably sure it's NTT, because my CL route is NTT bidirectionally [06:46] but comcast is only NTT outbound from the vm [06:46] and the CL one has the same hop both directions [06:46] yeah i think it's outbound via ntt that is the issue too [06:46] ae-3.r05.lsanca03.us.bb.gin.ntt.net <- that one [06:47] bnut that's from last time rather than testing [06:47] right [06:47] bah i'll see if i have any affected hosts :/ [06:47] i think i didn't [06:47] oh [06:47] it's too late at night now [06:47] isn't it [06:48] yeah [06:48] http://i.imgur.com/zPKFrvm.png [06:48] that's on one of the low-latency IP addresses via LAX [06:48] so what 4 hours too late? [06:48] 7 [06:48] oh [06:49] stops at around midnight gmt -0700 [06:49] it's nearly 2 am here [06:49] i should be in bed heh [06:49] but yeah means it finishes about 7 pm here [06:49] which is about when it's easy to do testing hah [06:49] haha [06:50] oh it's only 5% loss [06:50] it was way worse than that what i saw? [06:50] yeah, i think the vz one is much worse [06:50] ahh [06:50] speed congestion and latency are what i'm seeing here [06:51] just coincides with the vz thing, just less severe [06:51] yeah it's complicated to determine impact [06:51] usually the graph is nice and flat 24/7 [06:51] i normalyl reckon 0.5% to 5% loss is degraded [06:51] and over 5% is severely degraded [06:51] 29ms +/- .3ms right now [06:51] and 20%+ loss is unusable. [06:51] but i thikn loss should be below 0.05% [06:52] and ordinarily it is, there's just some wonky stuff somewhere [06:52] for transit links etc [06:52] well i imagine comcast may be resolvable [06:52] ie they may actually take notice if you email them [06:53] whereas i think verizon is long standing dispute territory? [06:53] yeah, they're terrible [06:53] i've got a buddy who's an engineer w/comcast, but he's out of town. i figured if it's persisting when he gets back, i'll drop him a line [06:56] i doubt much will happen in a timely manner [06:56] but i've been wrong in the past [06:57] oh hey i'm seeing latency as well [06:57] http://202.49.71.24:24/smokeping/smokeping.fcgi?displaymode=n;start=2014-06-06%2019:53;end=now;target=other.comcastnet [06:58] from nz, via verizon to comcast [06:58] it's probably ntt return path [06:58] it originates from 202.49.71.24 [06:59] can you traceroute from comcast back to that ip? [07:00] uhh it looks like level3 [07:00] oh it's ntt forward path [07:02] ok it verizon forward path again [07:02] will see what it liketommorow [08:16] lol professional pingers.... [08:32] yeah, we're pro-pingers [09:18] up_the_irons: I realized yesterday that my card on file, had expired and I was two months behind. Thanks for not killing my account! I've updated the card info, and you can run it whenever you'd like. [09:29] Wouldn't up_the_irons have emailed on payment failure? [09:29] I agree - thanks for not killing the account - but I think there should've been a warning/notification [09:31] brycec: I get a ton of emails, I probably just overlooked one, if it came in. [09:36] Ah [13:01] ok, just sent NTT an email about Comcast [13:14] Alan, [13:14] We are currently seeing no issues on the NTT network. Please contact [13:14] your network provider to have the issue investigated further. [13:14] lol [13:15] haha, i guess give it 2 hours [13:20] yeah, if you look at my Verizon email [13:20] it went the same way [13:20] I have a modified version of the reply: http://paste.unixcube.org/k/eec77a [13:20] with s/Verizon/Comcast/ [13:21] hahaha, nice [13:46] *** toddf has quit IRC (Ping timeout: 245 seconds) [15:42] *** pseudorandom has joined #arpnetworks [15:47] *** pseudodoge has joined #arpnetworks [15:50] *** pseudorandom has quit IRC (Ping timeout: 240 seconds) [15:55] *** pseudodoge has quit IRC (Quit: Leaving) [16:33] haha, here it comes [16:40] Love the arpnetworksssss!!!!! [16:41] Running psybnc on arpnetworks box, idle time is almost 800 minutes now! :D [16:41] Very stable [16:41] Thanks arpnetworks! [18:22] anis: that's not very high :/ [18:23] heh [18:23] acf: i wonder where people are meant to cpomlain about these things [18:24] there's that outages mailing list but they probably won't notice [18:24] there's nanong :/ [18:24] but its' not really relevant [18:24] i added that particularly flaky hop to my monitoring [18:24] could email nanong where to discuss such things? [18:24] i'd love to see some kind of proper reporting of interconnects [18:24] the only one that's around doesn't actually seem very good [18:25] stupid wind. ugh. [18:25] http://internetpulse.net/ [18:25] @wx [18:25] !wx [18:26] ? [18:26] m0unds: You looking for @weather ? [18:26] wasn't sure what the trigger was, figured it'd be one of those two [18:26] haha [18:27] wx is kinda uncommon... weather makes way more sense :p [18:27] moar chars [18:27] @weather kaeg [18:27] There are 3 weather alerts in effect for your area! There is a Tornado Watch. There is a Areal Flood Advisory. There is a Wind Advisory. [18:27] Double Eagle Ii, NM: Partly Cloudy ☁ 79°F (26°C), Humidity: 19%, Wind: From the West at 14 MPH Gusting to 12 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=, or re-request this with: @weather -v kaeg [18:28] tornado watch o_o [18:28] yep, north of me [18:28] @weather 87113 [18:28] There are 3 weather alerts in effect for your area! There is a Tornado Watch. There is a Areal Flood Advisory. There is a Wind Advisory. [18:28] Albuquerque, NM: Partly Cloudy ☁ 86°F (30°C), Humidity: 23%, Wind: From the SE at 9.4 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=35.169689,-106.567902 or re-request this with: @weather -v 87113 [18:28] Nice weather m0unds [18:28] is that a unicode cloud? ☁ [18:28] wind seems off, it's shaking light poles and making the building creak [18:28] erk [18:28] that doesn't sound good :) [18:28] Yes acf_ [18:29] does that include gust speeds for wind? [18:29] yeah, tornado thing is north of here - they just implicated the county albuquerque is in for whatever reason [18:29] tonado watch doesn't mean there's tornado [18:29] nope, but conditions are favorable [18:29] winds do move :) [18:29] they had golfball sized hail up in santa fe a bit ago (where the tornado watch is) [18:30] ouch [18:30] lots of broken windows etc? [18:30] no idea, someone on failbuque just said there was a lot of hail [18:36] @weather auckland [18:36] Auckland, New Zealand: Mostly Cloudy ☁ 67°F (19°C), Humidity: 91%, Wind: From the NE at 4.5 MPH Gusting to 12.1 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=-36.973896,174.878021 or re-request this with: @weather -v auckland [18:38] 67 sounds nice [18:38] @wx 99019 [18:38] Liberty Lake, WA: Scattered Clouds 74°F (23°C), Humidity: 27%, Wind: Calm -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=47.666508,-117.100792 or re-request this with: @wx -v 99019 [18:38] bah [18:39] what is wind calm [18:39] haha [18:39] does that mean below 5mph or something? [18:40] so how does one get oen of these personal meter thingys [18:40] @wiki Beaufort scale [18:40] Beaufort scale :: The Beaufort scale /ˈboʊfərt/ is an empirical measure that relates wind speed to observed conditions at sea or on land. Its full name is the Beaufort wind force scale, although it is a measure of wind speed and not of force in the scientific sense. History The scale was devised in 1805 by Francis Beaufort (later Rear Admiral Sir Francis Beaufort), an Irish... http://en.wikipedia.org/wiki/Beaufort%20scale [18:40] mercutio: ^ [18:41] weather underground had a thing outlining hardware requirements for PWS data feeding [18:41] with some suggestions for vendors in the us and whatnot [18:41] oh it's less than 1km/h [18:41] there's one like 500ft from my house [18:41] eerie still [18:42] m0unds: there's quite a few around it seems [18:42] so i imagine it's not extermely expensive [18:42] http://www.wunderground.com/personal-weather-station/dashboard?ID=KNMALBUQ167#history [18:42] http://www.wunderground.com/weatherstation/about.asp [18:42] that's the one right near my house [18:42] yea, my tunnel is shit slow right now so i didn't dare to try and dig around the site [18:42] haha [18:43] you need to use windows? [18:43] No? [18:43] WeatherLink suonded windowsy [18:43] It's self-contained, just plug in Ethernet [18:44] "The RainWise RapidFire(tm) enabled weather station doesn't need a PC to upload to us." [18:44] "WeatherBridge is an IP WiFi appliance that connects to weather stations and updates to Weather Underground real-time, providing up to the second data without the use of a PC." [18:44] etc [18:44] all non-PC [18:44] $1271 [18:45] that's expesnive :/ [18:45] Starting at $250 tho [18:45] you also need to wire it up etc [18:45] some of them http://www.ambientweather.com/amws1000wifi.html [18:45] the $250 are the windows ones? [18:45] No? [18:46] http://rainwise.com/wunderground/ $139 [18:46] oh so they are all self contained except the usb ones? [18:46] (And still no PC required) [18:46] What USB ones? [18:46] http://www.davisnet.com/weather/products/weather_product.asp?pnum=06510USB [18:46] Oh you clicked on the top of the line one :p [18:47] i googled? [18:47] you save $139 [18:47] it's $1350 :/ [18:47] i can do too many fun things with $1350 to tie it up in weather stations [18:48] unless i had interesting weather or something [18:48] but it's sunny something like 300 days a year in ABQ [18:48] exactly [18:48] haha [18:48] but it's windy [18:48] wind just means it's either dusty or cold usually [18:48] (right now it's dusty) [18:48] http://www.ambientweather.com/amws1000wifi.html [18:48] so this one is good? [18:48] m0unds: If you ran an airport, for instance, you'd care about weather details such as the $$ will get you [18:49] airports have weather stations :/ [18:49] that upload to these weather thingys [18:49] mercutio: Yes, they buy them from somewhere... [18:50] if i ran an airport, it'd matter [18:50] (small airports, air strips, etc) [18:50] i live near a small civil aviation airport [18:50] DOUBLE EAGLE Ii (the second i is lowercase for some reason) [18:50] Yeah m0unds but the idea seemed to go over mercutio's head [18:51] lol [18:51] And mercutio I have no idea if it's "good" - I only know that it was linked from the wunderground URL I provided. [18:51] it didn't go over my head [18:52] i just saw there were like 20 personal stations in my city [18:52] and figured that lots of them wouldn't be spending lots of money [18:52] there are two airport official ones in the city too [18:53] Airport weather rarely matches the weather a few miles/km away [18:53] oh? [18:53] well lots of sample sets give better ideas anyway [18:53] this city doesnt' have the same waether all over :/ [18:53] Either on account of the airport's distance away from the city, or the microclimate caused by jet engines. [18:54] (The airport here is atop a tall hill, 30 minutes away) [18:54] (for example) [18:54] kind of annoying when driving, because you have to turn wipers on/off/on/off [18:54] * brycec has automatic wipers :D [18:54] that sounds more annoying :) [18:54] I don't see how [18:55] i find automatic cars annoying too :/ [18:55] it'sk ind of disturbing when somethnng else is decding when to do stuff [18:55] Starts raining, wipers start. Rains harder, wipers speed up to match. [18:55] my truck does that and the lights turn on when it gets dark too [18:56] MAGIC [18:56] (me too) [18:56] magic! [18:56] lol [18:59] heh [18:59] i gotta go into work super early on wednesday [19:00] have to leave at 0300. bleck. [19:00] And honestly, the magic wipers have come in handy in some very inclement weather; I could keep my hands on the wheel at all times (necessary) and not worry about tweaking the speed. [19:02] SHAZAAM [19:08] ouch, m0unds [19:08] What time are you going to go to bed? [19:08] probably 2100 or so [19:08] get to be half asleep while i take down our recording and operator environment so a vendor can fix UPS' [19:08] and you'll wake up, i imagine, at 0200? [19:09] yeah, something like that probably [19:09] haha [19:09] >_< [19:09] I haven't done anything like that since college [19:09] pair of 80kVA eaton powerware units that our facility engineer decided didn't need maintenance bypasses :) [19:10] oh dear [19:19] oh cool. the air pressure differential caused the roof covering over the convention center at work to balloon up, tear, then flop down like a deflated balloon [19:20] m0unds: you guys probably saved a couple hundred bucks, you should say congrats to your power engineer :P [19:20] gizmoguy: ikr [19:20] it was a $381 option for the two UPS' i specced as replacements late last year (they won't give me the money) [19:21] EVERYTHING MUST STAY UP. wait. you want money? whoa, whoa whoa. watch it. [19:22] why are you guys even buying UPSs [19:22] they are pretty expensive [19:22] hahaha [19:22] and the maintainence of replacing batteries etc [19:22] business continuity, brah [19:23] is the rest of the casino on generators? [19:23] that's the funny part [19:23] i think i told you about that whole lightning/flooding mess we had in july, right? [19:23] yeah [19:23] we have N+1 gensets for the facility, total generating capacity is like 7MW [19:24] the ones (two of them) that backed our gear had no maintenance records, dead batteries, dead battery chargers, so they failed to start when we lost utility power due to the transformer exploding [19:24] rofl [19:24] so they decided that the way to fix it was to throw more money at new UPS subsystems to keep us up longer if the gens fail to fire up [19:24] see, way better off if you didn't have them :) [19:25] why don't you buy laptops for servers [19:25] lol [19:25] YUS [19:25] I hear they have built in UPSs [19:25] so, the old men with the big checkbooks wanted us to get new, better UPS' (which i would like, because the ones we have are 13 years old) [19:25] they have battery maintenance and stuff, but they're old and don't have maintenance bypasses >:| [19:26] so they asked me for a ballpark price range for what we wanted, so i specced 4 80kVA 9390s w/vycon flywheel frontends as an option to reduce battery wear and extended battery cabinets rated for 15 mins at full load (because that's the magic number they had in their minds) [19:28] it was gonna be something like $385k installed, with transfer switches and everything [19:28] but that was too much [19:29] but they just approved $280k for us to wire up a temporary parking lot for camera coverage and stuff while a parking garage is built :) [19:29] buy second hand [19:29] ebay probably has a lot of UPSs [19:30] i'd rather just buy a conex container full of 1000VA rackmount ups' and dump them in the fountain [19:30] the flywheels are neat though [19:31] http://vyconenergy.com/pq/pages_pq/pqprod.htm [19:50] *** toddf has joined #arpnetworks [19:50] *** ChanServ sets mode: +o toddf [20:08] well, the packet loss + latency thing is back in full force again. seeing a 30ms differential between ae10-50g.cr1.lax2.us.nlayer.net and ge0-arpnet.cust.lax07.mzima.net [20:08] gizmo [20:08] and 10-15% pkt loss [20:08] haha [20:08] ok sending mail [20:08] you know that some people do actually advocate buying laptops for servers [20:08] worst idea ever :/ [20:09] mercutio: you're too late, i already bought 60 at his advice [20:09] they'll be here friday [20:09] s/at/on [20:09] mercutio: you're too lone, i already bought 60 on his advice [20:09] hahahah [20:09] lone [20:09] * m0unds gives up [20:09] oh i tried smokeping last night [20:09] i should check it [20:10] yep :/ [20:10] http://kremvax.acfsys.net/smokeping.cgi?target=Local.unixcube [20:10] oh that was to comcast [20:10] yeah [20:10] and yeah via verizon to comcast no loss [20:10] like /no/ loss [20:10] today, and loss yesterday [20:10] acf_: i've got a 30ms differential between ntt and comcast, and a 30ms differential between arp and nlayer [20:10] ntt vs verizon forward route [20:10] level3 return route [20:10] so yeah it's ntt's forward path that sucks most likely [20:11] arp and nlayer could be about return path too [20:11] just hard to diagnose [20:11] haha [20:12] if you look at the mtr for ARP -> Comcast [20:12] i'm doing both now [20:12] the packet loss/latency clearly starts right between NTT and Comcast [20:12] yeah, there's a hop w/no rdns [20:12] wow that's some sucky packet loss [20:12] the whois says Comcast [20:12] yup [20:13] well ntt aren't going to help [20:13] it soundedl ike :/ [20:13] I know. I just want to see what they say [20:13] if they'll tell me if it's peering again this time [20:13] need lots of people cmoplaining really [20:13] complaining [20:13] not even sure if that will help [20:14] yeah might not [20:14] I'm sure they *want* to fix it [20:14] it probably helps if you pay them money [20:14] just Comcast, etc... is being unreasonable / they're being unreasonable [20:14] mercutio: :D [20:14] did you try complaining to comcast as well? [20:14] no. I just figure I'll get the Verizon-type response [20:14] I like to walk the line of ironic [20:15] well anyway i'm vonvinced that if the outbound path gets fixed it'll be a lot better [20:15] based on changing outbound path on my nz host: http://202.49.71.24:24/smokeping/smokeping.fcgi?target=other.comcastnet [20:15] http://pastebin.com/xrywYeLq [20:15] and it going to 0 loss [20:15] when it didn't go via ntt [20:16] it had like 4 bad days [20:16] and some in between stuff before that [20:16] http://paste.unixcube.org/k/91504f [20:16] with visible loss [20:16] that nlayer to mzima hop is return path most likely [20:16] nlayer and mzima are the same company [20:17] one side is arp's, if i was to bet i'd say lax07 = arp switch 07 [20:17] s7.lax.arpnetworks.com [20:17] that's the new server i think [20:17] and s1 is the old [20:17] s/server/router/ [20:17] that's the new router i think [20:18] I would think that lax07 would be controlled by nlayer/mzima [20:18] just guessing that particular hop is probably the mzima port [20:18] oh ok [20:18] could be wrong [20:18] yeah that's almost definitely s7.lax.arpnetworks.com [20:19] it would definitely explain the packet loss [20:19] if the return path was via ntt [20:21] and just because it's so odd, http://pastebin.com/xABATfn9 [20:21] that is on the same VM, just a different IP addr [20:21] but the path to it is completely different [20:21] haha [20:21] abq -> denver -> sjc -> lax -> arp [20:21] vs abq -> lax -> arp [20:21] hmm strange [20:22] Comcast's paths seem to change a lot [20:22] at one point I was going through Chicago [20:22] well basically up_the_irons can complain to ntt himself, or can maek exception to not wanting to route around issues. [20:22] but I'm in CA [20:22] I don't think NTT can do anything even if up_the_irons complains [20:22] yeah, it seems like that particular path bw comcast and ntt is hot [20:23] just oversubscribed or something [20:23] it's been hot in the past too [20:23] it looked like i started monitoring it in march, so i imagine it's march when we discussed last [20:23] at least it's not tata [20:23] tata is comcast's cesspool for traffic [20:23] arp traffic used to go via NTT <-> TATA <-> Comcast [20:24] I think [20:24] it used to be nlayer both directions for me [20:24] i wonder if routes are being advertised to level3 yet [20:25] level3 looking glass traceroute says no [20:25] i found a comcast looking glass last night [20:25] but it was in canada [20:25] oh cool [20:26] at least it's on their asn [20:26] i didn't even know that comcast was in canada [20:26] I didn't either [20:26] Verizon hasn't responded to my request for looking glass :( [20:28] http://www.comcast.com/peering/ [20:28] Dear Verizon Enterprise Solutions Customer, [20:28] We have received your correspondence and forwarded it to the appropriate personnel for processing. [20:29] that's cool [20:31] oh, here you go [20:31] telnet://route-server.newyork.ny.ibone.comcast.net [20:31] ^ looking glass [20:31] haha [20:31] oh boo [20:31] authenticated [20:31] haha [20:33] actually, rviews is the username, no pw [20:36] http://pastebin.com/H0K9vMDw [20:37] only route is via NTT? [20:38] I don't know how to read those... [20:41] 4436 is nlayer [20:42] oh ok I didn't look up the AS [20:43] so, only route out is via nlayer [20:43] does that mean NTT peers only one way? [20:43] that doesn't make sense [20:43] well, from comcast's perspective, nlayer is the preferred route to arp [20:44] which we can see when we traceroute from comcast to an arp address [20:44] ok, right [20:46] haha, nice, ntt is using juniper gear [20:47] yeah, they use some fun load balancers too [20:47] ping ntt.net and see if you notice anything strange [20:47] just saw the junos syntax in their lg when i had it show me bgp [20:48] haha, funny [20:48] dupe responses [20:48] yup [20:49] i get why they'd only return terse output, but i think that also makes them jerks [20:50] so, Verizon and Comcast are doing the paid peering thing [20:50] and NTT is refusing to cooperate? [20:52] comcast makes lots of stipulations for settlement-free peering - you have to have a big network with reasonable route announcements, you have to receive as much as you send, etc [21:18] ntt are probably in that situation [21:18] ntt is tier 1 i think [21:18] they're not meant to pay [21:18] yeah [21:19] think is cogent is huge [21:19] and is tier 2 [21:19] level3 payed up I think [21:19] it gets messy [21:19] that cogent HE ipv6 peering thing is still a problem [21:19] it doesn't erally matter if it's paid or free peering [21:19] it's whether links are congested or not [21:20] it seems to them it matters :) [21:20] from an operational rather than political pov [21:20] yeah [21:20] i am actually pro paid peering [21:20] i just think it should be at reasonable cost [21:20] as whenever things are free people don't take them as seriously [21:21] it seems like the idea of settlement-free peering requires that the traffic ratios be roughly equal [21:21] because both sides would be paying each other the same amount [21:22] well it depends [21:22] i mean for big transit providers they should [21:22] normally sender pays [21:23] right [21:23] so if you transmit 200 megabit/sec and receive 100 megabit/sec [21:23] how much shuold you pay? [21:23] 50% of your transmit volume? [21:23] and say you want to be able to send 2000 megabit/sec [21:24] and so you want to have aggregated ethernet or 10 gigabit [21:24] how much shoudl you pay [21:25] i imagine paid peering for 2 gigabit on 10 gigabit is around $1000/month or more [21:25] but i could be wrong [21:25] that doesn't seem unreasonable [21:25] for NTT who has big bucks to spend on that sort of thing [21:25] but if it is that kind of cost [21:25] than people shoudl be able to justify it [21:25] unless people want to cost it out at ilke $10,000/month [21:25] but [21:26] if you're only transmitting 200 megabit normally it's hard to justify [21:26] so say instead it's 200 megabit average, but 800 megabit peak [21:26] during the evenings [21:26] on a gigabit link [21:26] as soon as any link fails, that traffic would instead go over that link [21:26] you're screwwed [21:27] but the cost goes up generally going above 1gigabit [21:27] leaving either upgrading to 10 gigabit, or doubling up gigabit link [21:27] s [21:27] or rerouting traffic [21:28] but if the other link was only peaking at 300 megabit in evings [21:28] it might have been culled [21:28] because it wasn't worthwhile [21:28] but of course that puts more pressure on the first link [21:28] of course don't know all the details, but it's a complicated problem [21:28] just it seems like network quality would be more important than a few dollars to NTT and the like [21:28] and the simplest solution is to way overprovision [21:29] so if your peak traffic is over 500 megabit [21:29] and most effective I would imagine [21:29] then you should have another link [21:29] if you have two gigabit links, that peak at over 1 gigabit combined, you shoudl have a third link etc [21:31] but basically the way internet changes are moving towards is make it cheaper to go for 10 gigabit [21:31] s/changes/exchanges/ [21:31] but basically the way internet exchanges are moving towards is make it cheaper to go for 10 gigabit [21:31] do the Tier 1 providers still use gigabit links at all? [21:31] because that also reduces impact of ddos attacks. [21:31] for interconnect [21:32] if it says ge in a traceroute it's one gigabit [21:32] if it says xe it's 10 gigabit [21:32] like is NTT <-> Verizon/Comcast really gigabit? [21:32] in juniper speaker [21:32] it's 10 gigabit i think [21:32] te is cisco 10 gigabit [21:32] my mtrs all say ae [21:32] which is aggregated [21:32] ae is hard to know [21:32] it's juniper aggregated ethernet [21:32] yeh [21:32] it could be either [21:33] more likely gigabit though? [21:33] well it means it's at least 2 gigabit or at least 20 gigabit [21:33] hard to know [21:33] would they have more than 10 gigabit? [21:33] could do [21:33] 40 gigabit is common now [21:33] wow [21:34] what is he [21:34] comcast are using he and te [21:34] i imagine he is moer than 10 gigabit [21:34] I think it's old 10 gigabit interfaces [21:34] http://t1rex.blogspot.co.nz/2013/01/comcasts-100-gbps-core-network-supports.html [21:35] so comcast is shifting from 40 to 100 gigabit [21:35] with multiple links probably [21:35] what [21:35] he- 100 Gigabit Ethernet (Cisco) [21:35] hmm i think denver can go la->phoenix>denver [21:36] or la->sj>denver [21:36] yeah anyway, ntt talk to comcast at 10 gigabit [21:36] or come into their netowrk at 10 gigbit [21:36] it's hard to know [21:36] so Verizon still likely uses SONET/OC-192 for their backbone? [21:36] arp talks to ntt at gigabit [21:37] yeah, I see that in the mtr [21:37] verizon do internationally i'm pretty sure [21:37] i imagine they have lots of legacy gear [21:37] and it doesn't make sense to update it [21:37] why would it make sense for Comcast and not Verizon? [21:38] because comcast have national gear not international? [21:38] their network is newer/less established [21:38] I guess [21:38] and their users mostly talk to netflix etc :/ [21:38] verizon also have slas [21:38] with regards to latency, packet loss etc. [21:38] which sonet makes it easier to maange [21:38] but can they really stay with 10 gigabit forever? [21:39] 10 gigabit is heaps [21:39] I suppose [21:39] but as you said, other networks are going 40/100 gigabit [21:39] just becasue it's 10 gigabit doesn't mean it's only one 10 gigabit [21:40] and 10 gigabit interconnects between cities are probably normally fine [21:40] comcast is the biggest provider int eh US afaik [21:40] but even in NZ there's one provider doing 100 gigabit [21:40] because they don't suck nearly as bad as Verizon [21:40] the thing is how much is 100 gigabit [21:40] is it the same there? [21:40] like comcast seem to have 100 gigabit between san jose and denver [21:40] the dominate provider is dominate because their service doesnt suck? [21:41] i imagine most cities are 10 gigabit though [21:41] we have this regulated deregulation system [21:41] uhh [21:42] the country sold the telephone network [21:42] then they imposed limits of them [21:42] then the telephone company split in to two companies [21:42] and one of them is wholesale and one is consumer [21:42] and the wholesale seels to the consumer [21:42] within the two split companies [21:43] and then the government is paying money to different companies to build their own fibre networks [21:43] which impact the only cable company here [21:43] the cable company bought out the third biggest provider or something [21:43] can't copy / paste because im on a tablet, but comcast's own ibone backbone is still relatively new, so the comparison of vz w legacy gear vs comcast's new network is an apt one [21:43] so now there are two major companies with cellphone/broadband. [21:44] they used to rely on at&t for coast to coast stuff [21:44] but there are lots of smaller companies too [21:44] yaeh i imagine verizon has a lot of legacy cruft [21:44] didn't uunet go bust? [21:45] and was bailed out by government? [21:45] they are verizon [21:45] now [21:45] then acquired by verizon? [21:45] yep [21:45] so-2-1-0-0.DFW03-CORE-RTR1.veriz [21:45] mtr verizon.net [21:45] companies that go bust often have more legavy stuff prior :) [21:45] as there's less desire to invest in modernising [21:45] and companies that are bought [21:45] well people don't understand :) [21:45] lots of staff leave etc [21:45] management dilemas etc [21:46] it's really common to have billnig issues when companies are bought out [21:46] and billing systems always take ages to integrated [21:46] -d [21:46] verizon seems like a giant stew of bought up telecoms [21:46] that has no desire to integrate its components [21:46] haha [21:46] joy [21:46] or do anything other than mainain the status quo [21:46] is cox good? [21:47] and get out of the landline and dsl market [21:47] I've heard it is [21:47] it's just not anywhere that comcast is? [21:47] it doesn't tend to have the peering congestion we see on Comcast and Verizon [21:47] right [21:47] it's a regional monopoly thing [21:47] because it's like a drug cartel, and they can't compete. [21:48] did i show you guys the youtube video [21:48] don't think so [21:48] https://t.co/wD3MiiXOSM [21:48] https://t.co/wD3MiiXOSM -> https://www.youtube.com/watch?v=fpbOEoRrHyU [21:48] i thought it was humourous [21:48] @youtube fpbOEoRrHyU [21:48] YouTube Entertainment: "Last Week Tonight with John Oliver (HBO): Net Neutrality" by LastWeekTonight (13m 18s), 2,856,851 views, 44,185 likes and 428 dislikes. Uploaded 2014-06-02T06:30:01.000Z. [21:49] but it did make it sound more interesting than lots of things [21:50] had to drop my ipv6 tunnel. congested enough that netflix was only hitting 1.5mbit/sec, haha [21:50] crap thats really not good [21:50] yea, typically i can max out my downstream (50mbps). meh. [21:51] via the tunnel, not w netflix [21:51] nutflix lol [21:52] right [21:52] heh [21:52] he does make it sound more interesting than most people [21:52] netflix would do whatever their usual max bitrate is. 8-15mbit or so [21:53] there's an la weekly editorial that tears apart his whole argument [21:53] oh well. bbl. [21:53] m0unds: where? [21:58] acf: what do you think of it? [21:58] almost done. but it's awesome [21:58] yeah [22:09] it's not that deep but it's accessible [22:11] mercutio: Not very high but it is good enough for me :) [22:15] anisfarhana: ahh you watched it too [22:20] i cant find the obnoxious one, but i think i had the sites mixed up. there was one at latimes.com that provided a little extra detail [22:20] . [22:20] http://www.latimes.com/opinion/opinion-la/la-ol-john-oliver-gets-net-neutrality-wrong-20140604-story.html [22:21] this was the milder one. i saw the other one at work, but cant remember where [22:21] mercutio: You don't want me to watch it? :( [22:44] nah you can watch it [22:44] i just thought acf was the only one who did :) [23:13] *** Mifta has joined #arpnetworks [23:18] *** Mifta has left [23:31] uhh newegg is considering bitcoin? [23:43] hmm, i notice t.co is on level3 but it seems to be on ntt too, and i noticed arp seems to have really high pings to it for some reason [23:57] whats at 1.1.1.1 or supposed to live there?