| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | adoyee has joined #arpnetworks | [01:22] |
| adoyee | hi,all | [01:25] |
| .... (idle for 15mn) | ||
| mercutio | hi | [01:40] |
| adoyee | I can't login my VPS :( | [01:41] |
| mercutio | it got hacked?
have you tried using vnc? | [01:42] |
| adoyee | I can't ping it. can you help me to ping it? | [01:43] |
| gizmoguy | we have some professional pingers in the channel
but yeah, does the VNC console let you in? | [01:45] |
| adoyee | oh, really? I think my VPN was blocked by the fireworks | [01:46] |
| mercutio | what? | [01:46] |
| gizmoguy | firewall ? | [01:47] |
| mercutio | ok what is your ip
for your vps he's in china so if he was vpn'ing via it, then it may have randomly got blocked | [01:47] |
| adoyee | 174.136.103.90 | [01:47] |
| gizmoguy | ah. yeah :/ | [01:47] |
| mercutio | we can't ping it either | [01:47] |
| gizmoguy | 100% packet loss | [01:47] |
| mercutio | so it's probably down for some reason
if you go to portal.arpnetworks.com and login then click on vps | [01:47] |
| adoyee | I have start it in portal | [01:48] |
| mercutio | then it will tell you what your vnc server is
and what port you don't usually need to start it, it should arleady be running and if there is a problem it can be good to see what it thinks the problem is | [01:48] |
| adoyee | how I can check it?
running or not | [01:49] |
| mercutio | by using vnc
so you went to portal ? and it says your vnc server and port? | [01:52] |
| adoyee | ok. I need a vnc client. | [01:53] |
| mercutio | ok | [01:54] |
| gizmoguy | windows? mac? linux adoyee ? | [01:55] |
| adoyee | I'm from China. u know, the firewall ..
mac mac and linux | [01:55] |
| gizmoguy | if you got a linux nearby
there should be a decent enough built in VNC client | [01:55] |
| adoyee | yes, and irc clients
using irc for the first time | [01:56] |
| *** | mike-burns has quit IRC (*.net *.split)
pcn has quit IRC (*.net *.split) RandalSchwartz has quit IRC (*.net *.split) meingtsla has quit IRC (*.net *.split) kevr has quit IRC (*.net *.split) KDE_Perry has quit IRC (*.net *.split) mercutio has quit IRC (*.net *.split) twobithacker has quit IRC (*.net *.split) m0unds has quit IRC (*.net *.split) adoyee has quit IRC (*.net *.split) pjs has quit IRC (*.net *.split) NiTeMaRe has quit IRC (*.net *.split) adoyee has joined #arpnetworks mike-burns has joined #arpnetworks pjs has joined #arpnetworks NiTeMaRe has joined #arpnetworks pcn has joined #arpnetworks RandalSchwartz has joined #arpnetworks meingtsla has joined #arpnetworks kevr has joined #arpnetworks m0unds has joined #arpnetworks twobithacker has joined #arpnetworks mercutio has joined #arpnetworks KDE_Perry has joined #arpnetworks barjavel.freenode.net sets mode: +o mike-burns BryceBot has quit IRC (*.net *.split) up_the_irons has quit IRC (*.net *.split) staticsafe_ has quit IRC (*.net *.split) avj has quit IRC (*.net *.split) carvite has quit IRC (*.net *.split) m0unds_ has quit IRC (*.net *.split) brycec has quit IRC (*.net *.split) josephb has quit IRC (*.net *.split) acf_ has quit IRC (*.net *.split) jpalmer has quit IRC (*.net *.split) eryc has quit IRC (*.net *.split) raptelan has quit IRC (*.net *.split) BryceBot has joined #arpnetworks up_the_irons has joined #arpnetworks staticsafe_ has joined #arpnetworks avj has joined #arpnetworks carvite has joined #arpnetworks m0unds_ has joined #arpnetworks brycec has joined #arpnetworks josephb has joined #arpnetworks acf_ has joined #arpnetworks jpalmer has joined #arpnetworks eryc has joined #arpnetworks raptelan has joined #arpnetworks barjavel.freenode.net sets mode: +o up_the_irons adoyee has quit IRC (*.net *.split) pjs has quit IRC (*.net *.split) NiTeMaRe has quit IRC (*.net *.split) adoyee has joined #arpnetworks pjs has joined #arpnetworks NiTeMaRe has joined #arpnetworks | [01:58] |
| mercutio | oh it's back | [02:06] |
| adoyee | hi~
what happened, so many left | [02:07] |
| gizmoguy | woo, freenode
net split | [02:16] |
| adoyee | ha`
moments pls | [02:17] |
| .......... (idle for 46mn) | ||
| mercutio | netsplit | [03:04] |
| gizmoguy | ~.`. | [03:15] |
| ..... (idle for 22mn) | ||
| mercutio | did you get sorted adoyee | [03:37] |
| adoyee | hi, mercutio. I'm in office now. It's better check the vps in my private time. | [03:39] |
| mercutio | ahh ok | [03:39] |
| .... (idle for 15mn) | ||
| *** | adoyee has quit IRC (Remote host closed the connection)
adoyee has joined #arpnetworks | [03:54] |
| adoyee has quit IRC (Ping timeout: 265 seconds) | [04:00] | |
| ....................... (idle for 1h52mn) | ||
| ziyourenxiang has joined #arpnetworks | [05:52] | |
| ...... (idle for 26mn) | ||
| ziyourenxiang has quit IRC (Quit: ziyourenxiang) | [06:18] | |
| ..... (idle for 23mn) | ||
| m0unds | professional pingers, represent! | [06:41] |
| mercutio | heh | [06:41] |
| m0unds | i think NTT is just screwy - this only happened in the last few days
i can see it from work (centurylink) as well | [06:41] |
| mercutio | uhh it was happening months ago | [06:41] |
| m0unds | i'm talking about the thing with comcast and me
not the thing with verizon | [06:41] |
| mercutio | oh | [06:41] |
| m0unds | well, comcast and centurylink and me | [06:42] |
| mercutio | well whenever we talked about it last time with some of us, verizon and comast were both a bit screwy
and ntt had had some los angeles routing oddities on top of thagt where ntt via san jose wasn't as bad as ntt via los angeles or osmething ilke that | [06:42] |
| m0unds | yeah, but i wasn't seeing 20+% packet loss and 200% higher latency | [06:42] |
| mercutio | oh
but there was 5% loss or something based on time of day | [06:42] |
| m0unds | yeah | [06:42] |
| mercutio | reoccuring | [06:42] |
| m0unds | hadn't seen that in months via comcast here though | [06:43] |
| mercutio | oh ok
there was 20 msec higher latency before or something i think? | [06:43] |
| m0unds | with the exception of that weird thing late last week, when kvr26 was experiencing tons of pkt loss | [06:43] |
| mercutio | i don't actually even remember what month this was :)
yeh that wasn't ntt's fault | [06:43] |
| m0unds | well, it's weird - one of my addresses ordinarily pings ~30ms and is routed via LAX directly off comcast to arp
but other ones are routed to SJC via denver and ping at 50ms or so | [06:44] |
| mercutio | have you tried iperf? | [06:44] |
| m0unds | nah
throughput is fine - i tunnel ipv6 traffic through this vm | [06:44] |
| mercutio | ahh ok
you can figure out direction of loss | [06:45] |
| m0unds | i'm reasonably sure it's NTT, because my CL route is NTT bidirectionally
but comcast is only NTT outbound from the vm and the CL one has the same hop both directions | [06:45] |
| mercutio | yeah i think it's outbound via ntt that is the issue too | [06:46] |
| m0unds | ae-3.r05.lsanca03.us.bb.gin.ntt.net <- that one | [06:46] |
| mercutio | bnut that's from last time rather than testing | [06:47] |
| m0unds | right | [06:47] |
| mercutio | bah i'll see if i have any affected hosts :/
i think i didn't oh it's too late at night now isn't it | [06:47] |
| m0unds | yeah
http://i.imgur.com/zPKFrvm.png that's on one of the low-latency IP addresses via LAX | [06:48] |
| mercutio | so what 4 hours too late? | [06:48] |
| m0unds | 7 | [06:48] |
| mercutio | oh | [06:48] |
| m0unds | stops at around midnight gmt -0700 | [06:49] |
| mercutio | it's nearly 2 am here
i should be in bed heh but yeah means it finishes about 7 pm here which is about when it's easy to do testing hah | [06:49] |
| m0unds | haha | [06:49] |
| mercutio | oh it's only 5% loss
it was way worse than that what i saw? | [06:50] |
| m0unds | yeah, i think the vz one is much worse | [06:50] |
| mercutio | ahh | [06:50] |
| m0unds | speed congestion and latency are what i'm seeing here
just coincides with the vz thing, just less severe | [06:50] |
| mercutio | yeah it's complicated to determine impact | [06:51] |
| m0unds | usually the graph is nice and flat 24/7 | [06:51] |
| mercutio | i normalyl reckon 0.5% to 5% loss is degraded
and over 5% is severely degraded | [06:51] |
| m0unds | 29ms +/- .3ms right now | [06:51] |
| mercutio | and 20%+ loss is unusable.
but i thikn loss should be below 0.05% | [06:51] |
| m0unds | and ordinarily it is, there's just some wonky stuff somewhere | [06:52] |
| mercutio | for transit links etc
well i imagine comcast may be resolvable ie they may actually take notice if you email them whereas i think verizon is long standing dispute territory? | [06:52] |
| m0unds | yeah, they're terrible
i've got a buddy who's an engineer w/comcast, but he's out of town. i figured if it's persisting when he gets back, i'll drop him a line | [06:53] |
| mercutio | i doubt much will happen in a timely manner
but i've been wrong in the past oh hey i'm seeing latency as well http://202.49.71.24:24/smokeping/smokeping.fcgi?displaymode=n;start=2014-06-06%2019:53;end=now;target=other.comcastnet from nz, via verizon to comcast it's probably ntt return path it originates from 202.49.71.24 can you traceroute from comcast back to that ip? uhh it looks like level3 oh it's ntt forward path ok it verizon forward path again will see what it liketommorow | [06:56] |
| ............... (idle for 1h14mn) | ||
| brycec | lol professional pingers.... | [08:16] |
| .... (idle for 16mn) | ||
| m0unds | yeah, we're pro-pingers | [08:32] |
| .......... (idle for 46mn) | ||
| jpalmer | up_the_irons: I realized yesterday that my card on file, had expired and I was two months behind. Thanks for not killing my account! I've updated the card info, and you can run it whenever you'd like. | [09:18] |
| brycec | Wouldn't up_the_irons have emailed on payment failure?
I agree - thanks for not killing the account - but I think there should've been a warning/notification | [09:29] |
| jpalmer | brycec: I get a ton of emails, I probably just overlooked one, if it came in. | [09:31] |
| brycec | Ah | [09:36] |
| .......................................... (idle for 3h25mn) | ||
| acf_ | ok, just sent NTT an email about Comcast | [13:01] |
| Alan,
We are currently seeing no issues on the NTT network. Please contact your network provider to have the issue investigated further. lol | [13:14] | |
| m0unds | haha, i guess give it 2 hours | [13:15] |
| acf_ | yeah, if you look at my Verizon email
it went the same way I have a modified version of the reply: http://paste.unixcube.org/k/eec77a with s/Verizon/Comcast/ | [13:20] |
| m0unds | hahaha, nice | [13:21] |
| ...... (idle for 25mn) | ||
| *** | toddf has quit IRC (Ping timeout: 245 seconds) | [13:46] |
| ........................ (idle for 1h56mn) | ||
| pseudorandom has joined #arpnetworks | [15:42] | |
| pseudodoge has joined #arpnetworks
pseudorandom has quit IRC (Ping timeout: 240 seconds) | [15:47] | |
| pseudodoge has quit IRC (Quit: Leaving) | [15:55] | |
| ........ (idle for 38mn) | ||
| m0unds | haha, here it comes | [16:33] |
| anisfarhana | Love the arpnetworksssss!!!!!
Running psybnc on arpnetworks box, idle time is almost 800 minutes now! :D Very stable Thanks arpnetworks! | [16:40] |
| ..................... (idle for 1h41mn) | ||
| mercutio | anis: that's not very high :/
heh acf: i wonder where people are meant to cpomlain about these things there's that outages mailing list but they probably won't notice there's nanong :/ but its' not really relevant | [18:22] |
| m0unds | i added that particularly flaky hop to my monitoring | [18:24] |
| mercutio | could email nanong where to discuss such things?
i'd love to see some kind of proper reporting of interconnects the only one that's around doesn't actually seem very good | [18:24] |
| m0unds | stupid wind. ugh. | [18:25] |
| mercutio | http://internetpulse.net/ | [18:25] |
| m0unds | @wx
!wx | [18:25] |
| brycec | ?
m0unds: You looking for @weather ? | [18:26] |
| m0unds | wasn't sure what the trigger was, figured it'd be one of those two
haha | [18:26] |
| brycec | wx is kinda uncommon... weather makes way more sense :p | [18:27] |
| m0unds | moar chars
@weather kaeg | [18:27] |
| BryceBot | There are 3 weather alerts in effect for your area! There is a Tornado Watch. There is a Areal Flood Advisory. There is a Wind Advisory.
Double Eagle Ii, NM: Partly Cloudy ☁ 79°F (26°C), Humidity: 19%, Wind: From the West at 14 MPH Gusting to 12 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=, or re-request this with: @weather -v kaeg | [18:27] |
| staticsafe | tornado watch o_o | [18:28] |
| m0unds | yep, north of me
@weather 87113 | [18:28] |
| BryceBot | There are 3 weather alerts in effect for your area! There is a Tornado Watch. There is a Areal Flood Advisory. There is a Wind Advisory.
Albuquerque, NM: Partly Cloudy ☁ 86°F (30°C), Humidity: 23%, Wind: From the SE at 9.4 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=35.169689,-106.567902 or re-request this with: @weather -v 87113 | [18:28] |
| brycec | Nice weather m0unds | [18:28] |
| acf_ | is that a unicode cloud? ☁ | [18:28] |
| m0unds | wind seems off, it's shaking light poles and making the building creak | [18:28] |
| mercutio | erk
that doesn't sound good :) | [18:28] |
| brycec | Yes acf_ | [18:28] |
| mercutio | does that include gust speeds for wind? | [18:29] |
| m0unds | yeah, tornado thing is north of here - they just implicated the county albuquerque is in for whatever reason | [18:29] |
| mercutio | tonado watch doesn't mean there's tornado | [18:29] |
| m0unds | nope, but conditions are favorable | [18:29] |
| mercutio | winds do move :) | [18:29] |
| m0unds | they had golfball sized hail up in santa fe a bit ago (where the tornado watch is) | [18:29] |
| mercutio | ouch
lots of broken windows etc? | [18:30] |
| m0unds | no idea, someone on failbuque just said there was a lot of hail | [18:30] |
| mercutio | @weather auckland | [18:36] |
| BryceBot | Auckland, New Zealand: Mostly Cloudy ☁ 67°F (19°C), Humidity: 91%, Wind: From the NE at 4.5 MPH Gusting to 12.1 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=-36.973896,174.878021 or re-request this with: @weather -v auckland | [18:36] |
| m0unds | 67 sounds nice | [18:38] |
| brycec | @wx 99019 | [18:38] |
| BryceBot | Liberty Lake, WA: Scattered Clouds 74°F (23°C), Humidity: 27%, Wind: Calm -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=47.666508,-117.100792 or re-request this with: @wx -v 99019 | [18:38] |
| m0unds | bah | [18:38] |
| mercutio | what is wind calm
haha does that mean below 5mph or something? so how does one get oen of these personal meter thingys | [18:39] |
| brycec | @wiki Beaufort scale | [18:40] |
| BryceBot | Beaufort scale :: The Beaufort scale /ˈboʊfərt/ is an empirical measure that relates wind speed to observed conditions at sea or on land. Its full name is the Beaufort wind force scale, although it is a measure of wind speed and not of force in the scientific sense. History The scale was devised in 1805 by Francis Beaufort (later Rear Admiral Sir Francis Beaufort), an Irish... http://en.wikipedia.org/wiki/Beaufort%20scale | [18:40] |
| brycec | mercutio: ^ | [18:40] |
| m0unds | weather underground had a thing outlining hardware requirements for PWS data feeding
with some suggestions for vendors in the us and whatnot | [18:41] |
| mercutio | oh it's less than 1km/h | [18:41] |
| m0unds | there's one like 500ft from my house | [18:41] |
| mercutio | eerie still
m0unds: there's quite a few around it seems so i imagine it's not extermely expensive | [18:41] |
| m0unds | http://www.wunderground.com/personal-weather-station/dashboard?ID=KNMALBUQ167#history | [18:42] |
| brycec | http://www.wunderground.com/weatherstation/about.asp | [18:42] |
| m0unds | that's the one right near my house
yea, my tunnel is shit slow right now so i didn't dare to try and dig around the site haha | [18:42] |
| mercutio | you need to use windows? | [18:43] |
| brycec | No? | [18:43] |
| mercutio | WeatherLink suonded windowsy | [18:43] |
| brycec | It's self-contained, just plug in Ethernet
"The RainWise RapidFire(tm) enabled weather station doesn't need a PC to upload to us." "WeatherBridge is an IP WiFi appliance that connects to weather stations and updates to Weather Underground real-time, providing up to the second data without the use of a PC." etc all non-PC | [18:43] |
| mercutio | $1271
that's expesnive :/ | [18:44] |
| brycec | Starting at $250 tho | [18:45] |
| mercutio | you also need to wire it up etc | [18:45] |
| brycec | some of them http://www.ambientweather.com/amws1000wifi.html | [18:45] |
| mercutio | the $250 are the windows ones? | [18:45] |
| brycec | No?
http://rainwise.com/wunderground/ $139 | [18:45] |
| mercutio | oh so they are all self contained except the usb ones? | [18:46] |
| brycec | (And still no PC required)
What USB ones? | [18:46] |
| mercutio | http://www.davisnet.com/weather/products/weather_product.asp?pnum=06510USB | [18:46] |
| brycec | Oh you clicked on the top of the line one :p | [18:46] |
| mercutio | i googled?
you save $139 it's $1350 :/ | [18:47] |
| m0unds | i can do too many fun things with $1350 to tie it up in weather stations
unless i had interesting weather or something but it's sunny something like 300 days a year in ABQ | [18:47] |
| mercutio | exactly
haha but it's windy | [18:48] |
| m0unds | wind just means it's either dusty or cold usually
(right now it's dusty) | [18:48] |
| mercutio | http://www.ambientweather.com/amws1000wifi.html
so this one is good? | [18:48] |
| brycec | m0unds: If you ran an airport, for instance, you'd care about weather details such as the $$ will get you | [18:48] |
| mercutio | airports have weather stations :/
that upload to these weather thingys | [18:49] |
| brycec | mercutio: Yes, they buy them from somewhere... | [18:49] |
| m0unds | if i ran an airport, it'd matter | [18:50] |
| brycec | (small airports, air strips, etc) | [18:50] |
| m0unds | i live near a small civil aviation airport
DOUBLE EAGLE Ii (the second i is lowercase for some reason) | [18:50] |
| brycec | Yeah m0unds but the idea seemed to go over mercutio's head | [18:50] |
| m0unds | lol | [18:51] |
| brycec | And mercutio I have no idea if it's "good" - I only know that it was linked from the wunderground URL I provided. | [18:51] |
| mercutio | it didn't go over my head
i just saw there were like 20 personal stations in my city and figured that lots of them wouldn't be spending lots of money there are two airport official ones in the city too | [18:51] |
| brycec | Airport weather rarely matches the weather a few miles/km away | [18:53] |
| mercutio | oh?
well lots of sample sets give better ideas anyway this city doesnt' have the same waether all over :/ | [18:53] |
| brycec | Either on account of the airport's distance away from the city, or the microclimate caused by jet engines.
(The airport here is atop a tall hill, 30 minutes away) (for example) | [18:53] |
| mercutio | kind of annoying when driving, because you have to turn wipers on/off/on/off | [18:54] |
| brycec | brycec has automatic wipers :D | [18:54] |
| mercutio | that sounds more annoying :) | [18:54] |
| brycec | I don't see how | [18:54] |
| mercutio | i find automatic cars annoying too :/
it'sk ind of disturbing when somethnng else is decding when to do stuff | [18:55] |
| brycec | Starts raining, wipers start. Rains harder, wipers speed up to match. | [18:55] |
| m0unds | my truck does that and the lights turn on when it gets dark too
MAGIC | [18:55] |
| brycec | (me too) | [18:56] |
| m0unds | magic! | [18:56] |
| brycec | lol | [18:56] |
| mercutio | heh | [18:59] |
| m0unds | i gotta go into work super early on wednesday
have to leave at 0300. bleck. | [18:59] |
| brycec | And honestly, the magic wipers have come in handy in some very inclement weather; I could keep my hands on the wheel at all times (necessary) and not worry about tweaking the speed. | [19:00] |
| m0unds | SHAZAAM | [19:02] |
| z310 | ouch, m0unds
What time are you going to go to bed? | [19:08] |
| m0unds | probably 2100 or so
get to be half asleep while i take down our recording and operator environment so a vendor can fix UPS' | [19:08] |
| z310 | and you'll wake up, i imagine, at 0200? | [19:08] |
| m0unds | yeah, something like that probably
haha | [19:09] |
| z310 | >_<
I haven't done anything like that since college | [19:09] |
| m0unds | pair of 80kVA eaton powerware units that our facility engineer decided didn't need maintenance bypasses :) | [19:09] |
| z310 | oh dear | [19:10] |
| m0unds | oh cool. the air pressure differential caused the roof covering over the convention center at work to balloon up, tear, then flop down like a deflated balloon | [19:19] |
| gizmoguy | m0unds: you guys probably saved a couple hundred bucks, you should say congrats to your power engineer :P | [19:20] |
| m0unds | gizmoguy: ikr
it was a $381 option for the two UPS' i specced as replacements late last year (they won't give me the money) EVERYTHING MUST STAY UP. wait. you want money? whoa, whoa whoa. watch it. | [19:20] |
| gizmoguy | why are you guys even buying UPSs
they are pretty expensive | [19:22] |
| m0unds | hahaha | [19:22] |
| gizmoguy | and the maintainence of replacing batteries etc | [19:22] |
| m0unds | business continuity, brah | [19:22] |
| gizmoguy | is the rest of the casino on generators? | [19:23] |
| m0unds | that's the funny part
i think i told you about that whole lightning/flooding mess we had in july, right? | [19:23] |
| gizmoguy | yeah | [19:23] |
| m0unds | we have N+1 gensets for the facility, total generating capacity is like 7MW
the ones (two of them) that backed our gear had no maintenance records, dead batteries, dead battery chargers, so they failed to start when we lost utility power due to the transformer exploding | [19:23] |
| gizmoguy | rofl | [19:24] |
| m0unds | so they decided that the way to fix it was to throw more money at new UPS subsystems to keep us up longer if the gens fail to fire up | [19:24] |
| gizmoguy | see, way better off if you didn't have them :)
why don't you buy laptops for servers | [19:24] |
| m0unds | lol
YUS | [19:25] |
| gizmoguy | I hear they have built in UPSs | [19:25] |
| m0unds | so, the old men with the big checkbooks wanted us to get new, better UPS' (which i would like, because the ones we have are 13 years old)
they have battery maintenance and stuff, but they're old and don't have maintenance bypasses >:| so they asked me for a ballpark price range for what we wanted, so i specced 4 80kVA 9390s w/vycon flywheel frontends as an option to reduce battery wear and extended battery cabinets rated for 15 mins at full load (because that's the magic number they had in their minds) it was gonna be something like $385k installed, with transfer switches and everything but that was too much but they just approved $280k for us to wire up a temporary parking lot for camera coverage and stuff while a parking garage is built :) | [19:25] |
| gizmoguy | buy second hand
ebay probably has a lot of UPSs | [19:29] |
| m0unds | i'd rather just buy a conex container full of 1000VA rackmount ups' and dump them in the fountain
the flywheels are neat though http://vyconenergy.com/pq/pages_pq/pqprod.htm | [19:30] |
| .... (idle for 19mn) | ||
| *** | toddf has joined #arpnetworks
ChanServ sets mode: +o toddf | [19:50] |
| .... (idle for 18mn) | ||
| m0unds | well, the packet loss + latency thing is back in full force again. seeing a 30ms differential between ae10-50g.cr1.lax2.us.nlayer.net and ge0-arpnet.cust.lax07.mzima.net | [20:08] |
| mercutio | gizmo | [20:08] |
| m0unds | and 10-15% pkt loss
haha | [20:08] |
| acf_ | ok sending mail | [20:08] |
| mercutio | you know that some people do actually advocate buying laptops for servers
worst idea ever :/ | [20:08] |
| m0unds | mercutio: you're too late, i already bought 60 at his advice
they'll be here friday s/at/on | [20:09] |
| BryceBot | <m0unds> mercutio: you're too lone, i already bought 60 on his advice | [20:09] |
| m0unds | hahahah
lone m0unds gives up | [20:09] |
| mercutio | oh i tried smokeping last night
i should check it | [20:09] |
| acf_ | yep :/
http://kremvax.acfsys.net/smokeping.cgi?target=Local.unixcube | [20:10] |
| mercutio | oh that was to comcast | [20:10] |
| acf_ | yeah | [20:10] |
| mercutio | and yeah via verizon to comcast no loss
like /no/ loss today, and loss yesterday | [20:10] |
| m0unds | acf_: i've got a 30ms differential between ntt and comcast, and a 30ms differential between arp and nlayer | [20:10] |
| mercutio | ntt vs verizon forward route
level3 return route so yeah it's ntt's forward path that sucks most likely arp and nlayer could be about return path too | [20:10] |
| m0unds | just hard to diagnose
haha | [20:11] |
| acf_ | if you look at the mtr for ARP -> Comcast | [20:12] |
| m0unds | i'm doing both now | [20:12] |
| acf_ | the packet loss/latency clearly starts right between NTT and Comcast | [20:12] |
| m0unds | yeah, there's a hop w/no rdns | [20:12] |
| acf_ | wow that's some sucky packet loss
the whois says Comcast | [20:12] |
| m0unds | yup | [20:12] |
| mercutio | well ntt aren't going to help
it soundedl ike :/ | [20:13] |
| acf_ | I know. I just want to see what they say
if they'll tell me if it's peering again this time | [20:13] |
| mercutio | need lots of people cmoplaining really
complaining | [20:13] |
| acf_ | not even sure if that will help | [20:13] |
| mercutio | yeah might not | [20:14] |
| acf_ | I'm sure they *want* to fix it | [20:14] |
| mercutio | it probably helps if you pay them money | [20:14] |
| acf_ | just Comcast, etc... is being unreasonable / they're being unreasonable | [20:14] |
| gizmoguy | mercutio: :D | [20:14] |
| mercutio | did you try complaining to comcast as well? | [20:14] |
| acf_ | no. I just figure I'll get the Verizon-type response | [20:14] |
| gizmoguy | I like to walk the line of ironic | [20:14] |
| mercutio | well anyway i'm vonvinced that if the outbound path gets fixed it'll be a lot better
based on changing outbound path on my nz host: http://202.49.71.24:24/smokeping/smokeping.fcgi?target=other.comcastnet | [20:15] |
| m0unds | http://pastebin.com/xrywYeLq | [20:15] |
| mercutio | and it going to 0 loss
when it didn't go via ntt it had like 4 bad days and some in between stuff before that | [20:15] |
| acf_ | http://paste.unixcube.org/k/91504f
with visible loss | [20:16] |
| mercutio | that nlayer to mzima hop is return path most likely
nlayer and mzima are the same company | [20:16] |
| m0unds | one side is arp's, if i was to bet i'd say lax07 = arp switch 07
s7.lax.arpnetworks.com | [20:17] |
| mercutio | that's the new server i think
and s1 is the old s/server/router/ | [20:17] |
| BryceBot | <mercutio> that's the new router i think | [20:17] |
| acf_ | I would think that lax07 would be controlled by nlayer/mzima | [20:18] |
| m0unds | just guessing that particular hop is probably the mzima port | [20:18] |
| acf_ | oh ok | [20:18] |
| m0unds | could be wrong | [20:18] |
| acf_ | yeah that's almost definitely s7.lax.arpnetworks.com
it would definitely explain the packet loss if the return path was via ntt | [20:18] |
| m0unds | and just because it's so odd, http://pastebin.com/xABATfn9
that is on the same VM, just a different IP addr but the path to it is completely different haha abq -> denver -> sjc -> lax -> arp vs abq -> lax -> arp | [20:21] |
| acf_ | hmm strange
Comcast's paths seem to change a lot at one point I was going through Chicago | [20:21] |
| mercutio | well basically up_the_irons can complain to ntt himself, or can maek exception to not wanting to route around issues. | [20:22] |
| acf_ | but I'm in CA
I don't think NTT can do anything even if up_the_irons complains | [20:22] |
| m0unds | yeah, it seems like that particular path bw comcast and ntt is hot
just oversubscribed or something | [20:22] |
| mercutio | it's been hot in the past too
it looked like i started monitoring it in march, so i imagine it's march when we discussed last | [20:23] |
| m0unds | at least it's not tata
tata is comcast's cesspool for traffic | [20:23] |
| acf_ | arp traffic used to go via NTT <-> TATA <-> Comcast
I think | [20:23] |
| m0unds | it used to be nlayer both directions for me | [20:24] |
| mercutio | i wonder if routes are being advertised to level3 yet | [20:24] |
| acf_ | level3 looking glass traceroute says no | [20:25] |
| mercutio | i found a comcast looking glass last night
but it was in canada | [20:25] |
| acf_ | oh cool
at least it's on their asn | [20:25] |
| mercutio | i didn't even know that comcast was in canada | [20:26] |
| acf_ | I didn't either
Verizon hasn't responded to my request for looking glass :( | [20:26] |
| m0unds | http://www.comcast.com/peering/ | [20:28] |
| acf_ | Dear Verizon Enterprise Solutions Customer,
We have received your correspondence and forwarded it to the appropriate personnel for processing. that's cool | [20:28] |
| m0unds | oh, here you go
telnet://route-server.newyork.ny.ibone.comcast.net ^ looking glass haha oh boo authenticated haha actually, rviews is the username, no pw http://pastebin.com/H0K9vMDw | [20:31] |
| acf_ | only route is via NTT?
I don't know how to read those... | [20:37] |
| m0unds | 4436 is nlayer | [20:41] |
| acf_ | oh ok I didn't look up the AS
so, only route out is via nlayer does that mean NTT peers only one way? that doesn't make sense | [20:42] |
| m0unds | well, from comcast's perspective, nlayer is the preferred route to arp
which we can see when we traceroute from comcast to an arp address | [20:43] |
| acf_ | ok, right | [20:44] |
| m0unds | haha, nice, ntt is using juniper gear | [20:46] |
| gizmoguy | yeah, they use some fun load balancers too
ping ntt.net and see if you notice anything strange | [20:47] |
| m0unds | just saw the junos syntax in their lg when i had it show me bgp
haha, funny dupe responses | [20:47] |
| gizmoguy | yup | [20:48] |
| m0unds | i get why they'd only return terse output, but i think that also makes them jerks | [20:49] |
| acf_ | so, Verizon and Comcast are doing the paid peering thing
and NTT is refusing to cooperate? | [20:50] |
| m0unds | comcast makes lots of stipulations for settlement-free peering - you have to have a big network with reasonable route announcements, you have to receive as much as you send, etc | [20:52] |
| ...... (idle for 26mn) | ||
| mercutio | ntt are probably in that situation
ntt is tier 1 i think they're not meant to pay | [21:18] |
| acf_ | yeah | [21:18] |
| mercutio | think is cogent is huge
and is tier 2 | [21:19] |
| acf_ | level3 payed up I think | [21:19] |
| mercutio | it gets messy | [21:19] |
| acf_ | that cogent HE ipv6 peering thing is still a problem | [21:19] |
| mercutio | it doesn't erally matter if it's paid or free peering
it's whether links are congested or not | [21:19] |
| acf_ | it seems to them it matters :) | [21:20] |
| mercutio | from an operational rather than political pov | [21:20] |
| acf_ | yeah | [21:20] |
| mercutio | i am actually pro paid peering
i just think it should be at reasonable cost as whenever things are free people don't take them as seriously | [21:20] |
| acf_ | it seems like the idea of settlement-free peering requires that the traffic ratios be roughly equal
because both sides would be paying each other the same amount | [21:21] |
| mercutio | well it depends
i mean for big transit providers they should normally sender pays | [21:22] |
| acf_ | right | [21:23] |
| mercutio | so if you transmit 200 megabit/sec and receive 100 megabit/sec
how much shuold you pay? | [21:23] |
| acf_ | 50% of your transmit volume? | [21:23] |
| mercutio | and say you want to be able to send 2000 megabit/sec
and so you want to have aggregated ethernet or 10 gigabit how much shoudl you pay i imagine paid peering for 2 gigabit on 10 gigabit is around $1000/month or more but i could be wrong | [21:23] |
| acf_ | that doesn't seem unreasonable
for NTT who has big bucks to spend on that sort of thing | [21:25] |
| mercutio | but if it is that kind of cost
than people shoudl be able to justify it unless people want to cost it out at ilke $10,000/month but if you're only transmitting 200 megabit normally it's hard to justify so say instead it's 200 megabit average, but 800 megabit peak during the evenings on a gigabit link as soon as any link fails, that traffic would instead go over that link you're screwwed but the cost goes up generally going above 1gigabit leaving either upgrading to 10 gigabit, or doubling up gigabit link s or rerouting traffic but if the other link was only peaking at 300 megabit in evings it might have been culled because it wasn't worthwhile but of course that puts more pressure on the first link of course don't know all the details, but it's a complicated problem | [21:25] |
| acf_ | just it seems like network quality would be more important than a few dollars to NTT and the like | [21:28] |
| mercutio | and the simplest solution is to way overprovision
so if your peak traffic is over 500 megabit | [21:28] |
| acf_ | and most effective I would imagine | [21:29] |
| mercutio | then you should have another link
if you have two gigabit links, that peak at over 1 gigabit combined, you shoudl have a third link etc but basically the way internet changes are moving towards is make it cheaper to go for 10 gigabit s/changes/exchanges/ | [21:29] |
| BryceBot | <mercutio> but basically the way internet exchanges are moving towards is make it cheaper to go for 10 gigabit | [21:31] |
| acf_ | do the Tier 1 providers still use gigabit links at all? | [21:31] |
| mercutio | because that also reduces impact of ddos attacks. | [21:31] |
| acf_ | for interconnect | [21:31] |
| mercutio | if it says ge in a traceroute it's one gigabit
if it says xe it's 10 gigabit | [21:32] |
| acf_ | like is NTT <-> Verizon/Comcast really gigabit? | [21:32] |
| mercutio | in juniper speaker
it's 10 gigabit i think te is cisco 10 gigabit | [21:32] |
| acf_ | my mtrs all say ae
which is aggregated | [21:32] |
| mercutio | ae is hard to know
it's juniper aggregated ethernet yeh it could be either | [21:32] |
| acf_ | more likely gigabit though? | [21:33] |
| mercutio | well it means it's at least 2 gigabit or at least 20 gigabit
hard to know | [21:33] |
| acf_ | would they have more than 10 gigabit? | [21:33] |
| mercutio | could do
40 gigabit is common now | [21:33] |
| acf_ | wow | [21:33] |
| mercutio | what is he
comcast are using he and te i imagine he is moer than 10 gigabit | [21:34] |
| acf_ | I think it's old 10 gigabit interfaces | [21:34] |
| mercutio | http://t1rex.blogspot.co.nz/2013/01/comcasts-100-gbps-core-network-supports.html
so comcast is shifting from 40 to 100 gigabit with multiple links probably | [21:34] |
| acf_ | what
he- 100 Gigabit Ethernet (Cisco) | [21:35] |
| mercutio | hmm i think denver can go la->phoenix>denver
or la->sj>denver yeah anyway, ntt talk to comcast at 10 gigabit or come into their netowrk at 10 gigbit it's hard to know | [21:35] |
| acf_ | so Verizon still likely uses SONET/OC-192 for their backbone? | [21:36] |
| mercutio | arp talks to ntt at gigabit | [21:36] |
| acf_ | yeah, I see that in the mtr | [21:37] |
| mercutio | verizon do internationally i'm pretty sure
i imagine they have lots of legacy gear and it doesn't make sense to update it | [21:37] |
| acf_ | why would it make sense for Comcast and not Verizon? | [21:37] |
| mercutio | because comcast have national gear not international?
their network is newer/less established | [21:38] |
| acf_ | I guess | [21:38] |
| mercutio | and their users mostly talk to netflix etc :/
verizon also have slas with regards to latency, packet loss etc. which sonet makes it easier to maange | [21:38] |
| acf_ | but can they really stay with 10 gigabit forever? | [21:38] |
| mercutio | 10 gigabit is heaps | [21:39] |
| acf_ | I suppose
but as you said, other networks are going 40/100 gigabit | [21:39] |
| mercutio | just becasue it's 10 gigabit doesn't mean it's only one 10 gigabit
and 10 gigabit interconnects between cities are probably normally fine comcast is the biggest provider int eh US afaik but even in NZ there's one provider doing 100 gigabit | [21:39] |
| acf_ | because they don't suck nearly as bad as Verizon | [21:40] |
| mercutio | the thing is how much is 100 gigabit | [21:40] |
| acf_ | is it the same there? | [21:40] |
| mercutio | like comcast seem to have 100 gigabit between san jose and denver | [21:40] |
| acf_ | the dominate provider is dominate because their service doesnt suck? | [21:40] |
| mercutio | i imagine most cities are 10 gigabit though
we have this regulated deregulation system uhh the country sold the telephone network then they imposed limits of them then the telephone company split in to two companies and one of them is wholesale and one is consumer and the wholesale seels to the consumer within the two split companies and then the government is paying money to different companies to build their own fibre networks which impact the only cable company here the cable company bought out the third biggest provider or something | [21:41] |
| m0unds | can't copy / paste because im on a tablet, but comcast's own ibone backbone is still relatively new, so the comparison of vz w legacy gear vs comcast's new network is an apt one | [21:43] |
| mercutio | so now there are two major companies with cellphone/broadband. | [21:43] |
| m0unds | they used to rely on at&t for coast to coast stuff | [21:44] |
| mercutio | but there are lots of smaller companies too
yaeh i imagine verizon has a lot of legacy cruft didn't uunet go bust? and was bailed out by government? | [21:44] |
| acf_ | they are verizon
now | [21:45] |
| mercutio | then acquired by verizon? | [21:45] |
| acf_ | yep
so-2-1-0-0.DFW03-CORE-RTR1.veriz mtr verizon.net | [21:45] |
| mercutio | companies that go bust often have more legavy stuff prior :)
as there's less desire to invest in modernising and companies that are bought well people don't understand :) lots of staff leave etc management dilemas etc it's really common to have billnig issues when companies are bought out and billing systems always take ages to integrated -d | [21:45] |
| acf_ | verizon seems like a giant stew of bought up telecoms
that has no desire to integrate its components | [21:46] |
| mercutio | haha
joy | [21:46] |
| acf_ | or do anything other than mainain the status quo | [21:46] |
| mercutio | is cox good? | [21:46] |
| acf_ | and get out of the landline and dsl market
I've heard it is | [21:47] |
| mercutio | it's just not anywhere that comcast is? | [21:47] |
| acf_ | it doesn't tend to have the peering congestion we see on Comcast and Verizon
right it's a regional monopoly thing | [21:47] |
| mercutio | because it's like a drug cartel, and they can't compete.
did i show you guys the youtube video | [21:47] |
| acf_ | don't think so | [21:48] |
| mercutio | https://t.co/wD3MiiXOSM | [21:48] |
| BryceBot | https://t.co/wD3MiiXOSM -> https://www.youtube.com/watch?v=fpbOEoRrHyU | [21:48] |
| mercutio | i thought it was humourous | [21:48] |
| brycec | @youtube fpbOEoRrHyU | [21:48] |
| BryceBot | <http://youtu.be/fpbOEoRrHyU> YouTube Entertainment: "Last Week Tonight with John Oliver (HBO): Net Neutrality" by LastWeekTonight (13m 18s), 2,856,851 views, 44,185 likes and 428 dislikes. Uploaded 2014-06-02T06:30:01.000Z. | [21:48] |
| mercutio | but it did make it sound more interesting than lots of things | [21:49] |
| m0unds | had to drop my ipv6 tunnel. congested enough that netflix was only hitting 1.5mbit/sec, haha | [21:50] |
| acf_ | crap thats really not good | [21:50] |
| m0unds | yea, typically i can max out my downstream (50mbps). meh.
via the tunnel, not w netflix | [21:50] |
| acf_ | nutflix lol
right | [21:51] |
| mercutio | heh
he does make it sound more interesting than most people | [21:52] |
| m0unds | netflix would do whatever their usual max bitrate is. 8-15mbit or so
there's an la weekly editorial that tears apart his whole argument oh well. bbl. | [21:52] |
| mercutio | m0unds: where? | [21:53] |
| acf: what do you think of it? | [21:58] | |
| acf_ | almost done. but it's awesome | [21:58] |
| mercutio | yeah | [21:58] |
| it's not that deep but it's accessible | [22:09] | |
| anisfarhana | mercutio: Not very high but it is good enough for me :) | [22:11] |
| mercutio | anisfarhana: ahh you watched it too | [22:15] |
| m0unds | i cant find the obnoxious one, but i think i had the sites mixed up. there was one at latimes.com that provided a little extra detail
. http://www.latimes.com/opinion/opinion-la/la-ol-john-oliver-gets-net-neutrality-wrong-20140604-story.html this was the milder one. i saw the other one at work, but cant remember where | [22:20] |
| anisfarhana | mercutio: You don't want me to watch it? :( | [22:21] |
| ..... (idle for 23mn) | ||
| mercutio | nah you can watch it
i just thought acf was the only one who did :) | [22:44] |
| ...... (idle for 29mn) | ||
| *** | Mifta has joined #arpnetworks | [23:13] |
| Mifta has left | [23:18] | |
| mercutio | uhh newegg is considering bitcoin? | [23:31] |
| hmm, i notice t.co is on level3 but it seems to be on ntt too, and i noticed arp seems to have really high pings to it for some reason | [23:43] | |
| mnathani | whats at 1.1.1.1 or supposed to live there? | [23:57] |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
