[00:08] <mercutio> is maint window now
[00:09] <mercutio> i assume so cos someone asked if it was over ipmi :)
[00:23] <brycec> https://twitter.com/arpnetworks/status/476982306163286017 was 22 minutes ago, so presumably the upgrade has bee going for the last 17
[00:23] <BryceBot> TWITTER: Maintenance window on host kvr15 will begin in about 5 minutes (Thu Jun 12 07:00:18 +0000 2014)
[00:27] <mnathani> Broadcast message from root (Thu Jun 12 03:05:31 2014):
[00:27] <mnathani> The system is going down for system halt NOW!
[00:30] <SpaceDump> :D
[00:30] <CaZe> Because it doesn't make business sense.
[00:31] <CaZe> They do have 24 hour drive throughs where it does make sense.
[00:32] <brycec> When I was at uni, there was a Starbucks open 'til 11pm
[00:33] <brycec> (the library was open 24hr, and the city scene were very active late into the night)
[00:46] *** jm|laptop has joined #arpnetworks
[00:46] <jm|laptop> did it pop?
[01:05] <jm|laptop> there it is!
[01:07] <plett> jm|laptop: Oh. Hello :)
[01:07] <jm|laptop> Oh. Hi. :)
[01:07] <jm|laptop> https://twitter.com/arpnetworks/status/476982306163286017   was it that?
[01:08] <BryceBot> TWITTER: Maintenance window on host kvr15 will begin in about 5 minutes (Thu Jun 12 07:00:18 +0000 2014)
[01:08] <jm|laptop> Thu 12 Jun 09:08:07 BST 2014
[01:08] <mercutio> oh it seems backup but going slow
[01:08] <mercutio> well taking ages to login
[01:09] <up_the_irons> cuz all VMs doing fsck
[01:09] <mercutio> hmm fresh login 2.89 load average
[01:09] <mercutio> even with acpi shutdown?
[01:09] <mercutio> oh uptime is only 5 minutes
[01:09] <mercutio> ok :)
[01:09] <mercutio> i thought it took about half an hour
[01:10] <mercutio> so yeah, the fsck hasn't been done in x days thing
[01:31] <SpaceDump> weee
[01:31] <SpaceDump> I had done something wrong on that vps. Everything came up again after startup. :D
[04:19] <mercutio> hmm i need to change to virtio now
[04:20] <mercutio> well don't need but should
[05:17] <jbergstroem> are there any known issues with network connectivity/routing? One of my CDN pops in Sydney doesn't seem able to reach my server
[05:36] <jbergstroem> Can anyone else try to do a route from arp to 23.235.41.22?
[05:46] *** z310 has quit IRC (*.net *.split)
[06:01] *** jcv has quit IRC (Quit: leaving)
[06:01] *** jcv has joined #arpnetworks
[06:18] <plett> jbergstroem: http://pastebin.com/fjx9GiWC
[06:21] <jbergstroem> plett: a while ago, seems to have fixed itself now: https://gist.github.com/jbergstroem/fd23d8ca83c9462ec807
[06:21] <BryceBot> Gist: "https://gist.github.com/fd23d8ca83c9462ec807"
[06:22] <plett> Cool. Glad it's working again then
[06:22] <jbergstroem> just after i rerouted shit at my cdn's place :/
[06:26] *** z310 has joined #arpnetworks
[09:59] <brycec> Came across this long-running mtr that spans the Level3 turn-up, thought it might be interesting to some https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-06-12_09-58-07.png
[10:51] *** mus1cb0x has joined #arpnetworks
[10:51] <mus1cb0x> since 14 was upgraded this week, i've been getting noticably increased 'broken pipe' to it
[10:51] <mus1cb0x> i know it isn't my inet link because i also have a ssh session to a different arp server and it remains up
[10:52] <mus1cb0x> before this, my connection to 14 was rock solid. anyone else on 14 notice degradation in QoS?
[10:52] <m0unds> mus1cb0x: which OS?
[10:52] <mus1cb0x> freebsd 9.2
[10:52] <m0unds> did you enable virtio?
[10:53] <mus1cb0x> where would that have been done?
[10:53] <mus1cb0x> in rc.conf or something?
[10:53] <mus1cb0x> i'm running stock -release kernel/base
[11:02] <m0unds> mus1cb0x: have you made any changes to configuration since the maintenance event?
[11:02] <mus1cb0x> none
[11:02] <m0unds> huh, weird
[11:02] <m0unds> see anything in logs?
[11:03] <mus1cb0x> up_the_irons: any other reports of this?
[11:03] <mus1cb0x> which logs would you suggest i look at?
[11:03] <m0unds> you could just try grep-ing the logs folder for your interface name
[11:03] <m0unds> maybe look for watchdog as another item
[11:06] <mus1cb0x> do you know how i could grep all files in the current dir for str ?
[11:07] <mus1cb0x> em0 is the if
[11:10] <brycec> grep -rn em0 .
[11:10] <brycec> (the key here is -r)
[11:13] <mus1cb0x> ty brycec
[11:14] <mus1cb0x> http://pastebin.com/85d7sCwe is everything that matches em0 in /var/log
[11:16] <mus1cb0x> i don't know what "52:54:00:27:22:11" is, but it appears before the maintenance window so i doubt it's an issue
[11:18] <mus1cb0x> seems like there's a lot of promiscuous mode being enabled then disabled, not sure if that could be cleaned up
[11:18] <mus1cb0x> but again, appeared before maint. window also
[11:22] <brycec> mus1cb0x: You mean, em0: Ethernet address: 52:54:00:27:22:11
[11:22] <brycec> It's the MAC
[11:22] <brycec> As for "promiscuous mode" you were most likely running tcpdump
[12:02] *** httpduck1 has quit IRC (*.net *.split)
[12:03] *** httpducks has joined #arpnetworks
[12:07] <acf_> brycec: that's really cool
[12:07] <acf_> the mtr
[12:07] <acf_> I see three paths?
[12:08] <acf_> ntt, level3, and any2
[12:12] <mus1cb0x> ah
[13:37] <mercutio> paste has been ermoved
[13:37] <mercutio> i see none :)
[13:42] <acf_> https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-06-12_09-58-07.png
[13:42] <acf_> mercutio ^
[13:46] *** tabthorpe has quit IRC (Ping timeout: 252 seconds)
[13:46] *** tabthorpe has joined #arpnetworks
[13:47] *** avj has quit IRC (Ping timeout: 264 seconds)
[13:47] *** raptelan has quit IRC (Ping timeout: 264 seconds)
[13:47] *** wallshot1 has joined #arpnetworks
[13:47] *** CaZe has quit IRC (Ping timeout: 252 seconds)
[13:47] *** mnathani has quit IRC (Ping timeout: 252 seconds)
[13:47] *** jcv has quit IRC (Ping timeout: 264 seconds)
[13:47] *** brycec has quit IRC (Ping timeout: 264 seconds)
[13:47] *** acf_ has quit IRC (Ping timeout: 264 seconds)
[13:47] *** eryc has quit IRC (Ping timeout: 264 seconds)
[13:47] <wallshot1> i'm guessing all those timeouts confirm what i was wondering
[13:47] *** jlgaddis has joined #arpnetworks
[13:48] <wallshot1> network or vps host outage perhaps?  i should try to vnc in and see what it says from the inside
[13:48] <jlgaddis> internet b0rked?
[13:48] <dangel> wallshot1: Agreed. Me too.
[13:48] <jlgaddis> yeah i had two vps' drop in the last three minutes
[13:48] *** raptelan has joined #arpnetworks
[13:49] *** acf_ has joined #arpnetworks
[13:49] *** jcv has joined #arpnetworks
[13:49] *** CaZe has joined #arpnetworks
[13:50] <KILLALLHUMANS01> I'm still connected to my VPS just fine
[13:50] <KILLALLHUMANS01> And some of those VPS->IRC connections are still good, but not VPS->Freenode(whatever server I'm on)
[13:50] <m0unds> still connected to mine too
[13:50] <dangel> I can't ssh into my node.
[13:50] <wallshot1> ssh timeouts turned into ssh slow-but-successful
[13:50] <KILLALLHUMANS01> I'm seeing a bunch of OpenVPN connections flap thoguh
[13:50] <m0unds> hm
[13:50] <acf_> hmm that was strange
[13:51] <dangel> Oh, there it goes.
[13:51] <KILLALLHUMANS01> So I'm thinking there's aroute having issues.
[13:51] <wallshot1> and pidgin is reconnected to my jabber server again, yay
[13:51] <jlgaddis> and everything's back up lol
[13:51] *** eryc has joined #arpnetworks
[13:51] *** eryc has quit IRC (Changing host)
[13:51] *** eryc has joined #arpnetworks
[13:51] <m0unds> weird. didn't notice anything here at all.
[13:52] <acf_> mtr -4 asimov.freenode.net
[13:52] <KILLALLHUMANS01> │13:51:52 *status | Disconnected from IRC (Connection reset by peer).  Reconnecting...
[13:52] *** mnathani has joined #arpnetworks
[13:52] <mercutio> oh it means the route changed
[13:52] <acf_> the two hops after s7.lax.arpnetworks.com are ???
[13:52] <mercutio> it may have b een withdrawn and readvertised
[13:52] <KILLALLHUMANS01> The route can change without destroying the connection... Unless the route changed to something unroutable.
[13:53] <mercutio> well it's strange for 10.10.10.6 to appear in it
[13:53] <acf_> what are you tracing to mercutio?
[13:53] <mercutio> acf: nothing, i'm lokoing at your mtr :)
[13:53] <acf_> oh I don't see 10.10.10.6...
[13:53] <mercutio> yes you do
[13:53] <acf_> it goes via Level3
[13:53] <mercutio> hop 3 second number
[13:54] <mercutio> err 2nd line
[13:54] <acf_> http://paste.unixcube.org/k/7b6ab9
[13:54] <mercutio> the screenshot
[13:54] <mercutio> that kinud of thing doesn't show up in short mtr's normally
[13:54] <acf_> oh that
[13:54] <mercutio> only when you leave mtr running in the backgruond
[13:54] <mercutio> and catch a route dropping
[13:54] <mercutio> or changing path
[13:54] <acf_> yes, that's brycec's trace from the Level3 switch over
[13:54] <acf_> it's cool
[13:55] <mercutio> but when hop 10 says los angeles, it suggests it probably dropped
[13:55] <mercutio> and looped a little
[13:55] <KILLALLHUMANS01> Yeah that's an old, long-running mtr
[13:55] *** brycec has joined #arpnetworks
[13:55] <KILLALLHUMANS01> Oh look at me, I'm back
[13:55] <mercutio> oh youre bryce
[13:55] <m0unds> yeah
[13:55] <m0unds> haha
[13:55] <KILLALLHUMANS01> am
[13:55] <brycec> am
[13:55] <mercutio> i was wondering who it was
[13:55] <acf_> same here lol
[13:55] <mercutio> i'm sure i'd seen the name before
[13:56] <KILLALLHUMANS01> This is a connection I leave up from another host, so... lurking
[13:56] <mercutio> heh
[13:56] <mercutio> freenode has mostly been stable for me recently
[13:56] <mercutio> [mercutio] idle: 00 hours 00 minutes 03 seconds, signon at: Sat, 03 May 2014 06:31:08
[13:57] <m0unds> 14:57:02 freenode  -- | [m0unds] idle: 00 hours 01 minute 25 seconds, signon at: Mon, 21 Apr 2014 17:26:55
[13:57] <mercutio> there's been a netsplit in that time
[13:57] <mercutio> it used to be pretty unstable
[13:58] *** mus1cb0x has left "WeeChat 0.4.2"
[13:58] <mercutio> the vast majority of people in this channel seem to idle
[13:58] <m0unds> yep
[14:00] * dangel pets mosh.
[14:27] <m0unds> i guess apd cornered an armed robbery suspect in our employee parking lot
[14:28] <m0unds> haha
[14:35] <RandalSchwartz> apd?
[14:36] <RandalSchwartz> [something] police department, I would guess
[14:36] <m0unds> yep
[14:36] <m0unds> albuquerque
[14:37] <RandalSchwartz> well - that explains it :)
[14:56] <mercutio> hmm i left a mtr to amazon running
[14:56] <mercutio> after it goes into amazon's network it hops around a lot
[14:59] <wallshot1> i'd expect that at any good sized org as clusters are put into and out of rotation for maintenance
[15:00] <wallshot1> with entire data centers going in and out of rotation at any given time
[15:00] <brycec> Even for simple load balancing
[15:04] <mercutio> wall: sounds complicated
[15:04] <mercutio> hop 14 has 8 differnet ip's
[15:04] <mercutio> but they all start with 205.251
[15:05] <mercutio> it's 21 hops
[15:05] <mercutio> and 11 hops to amazon
[15:05] <mercutio> so 10 hops within their network
[15:06] <mercutio> i probably should stop this mtr
[15:06] <mercutio> id on't know where it's too :/
[15:06] <wallshot1> that is indeed a few hops in their network before hitting the endpoint
[15:06] <mercutio> ec2-54-214-15-46.us-west-2.compute.amazonaws.com
[15:06] <mercutio> someone gave me that host to try us west?
[15:07] <mercutio> http://54.214.15.46/videos/Yelle%20-%20Comme%20Un%20Enfant%20%28Freaks%20Remix%29.webm
[15:07] <mercutio> what's that heh
[15:07] <acf_> watch it
[15:07] <acf_> the whole thing
[15:07] <mercutio> ahh so it you acf?
[15:07] <acf_> yep
[15:07] <mercutio> why are you on ec2
[15:08] <acf_> it has super good transit
[15:08] <mercutio> i don't know if i have a webm player
[15:08] <acf_> https://www.youtube.com/watch?v=hAiu04sUDMg
[15:08] <BryceBot> YouTube Music: "Yelle - Comme Un Enfant (Freaks Remix)" by Nathan Barnatt (5m 41s), 141,973 views, 4,515 likes and 29 dislikes. Uploaded 2014-01-03T21:48:54.000Z.
[15:08] <mercutio> i dunno i have 2.7% packet loss to it :/
[15:08] <mercutio> and 0.2% to paix amazon
[15:08] <mercutio> arp never has that high loss :/
[15:08] <acf_> hmm, I don't see that
[15:08] <m0unds> neither do i
[15:09] <acf_> what do you do when the mtr goes off the terminal?
[15:09] <acf_> too long
[15:09] <mercutio> http://pastebin.com/epc89FBq
[15:09] <mercutio> i resize the window
[15:09] <mercutio> i did alt enter at first
[15:10] <acf_> my window won't go any bigger lol
[15:10] <mercutio> then i realised it was way too wide to cut and paste
[15:10] <mercutio> i'm using 1440p
[15:10] <acf_> ah, ok
[15:10] <brycec> 50pkts no loss
[15:10] <mercutio> and it's over half my vertical screen size
[15:10] <mercutio> so at 720p i'm sure you'd struggle.
[15:10] <mercutio> i have 58,000 packets
[15:10] <mercutio> :)
[15:15] *** wallshot1 has quit IRC (Read error: No route to host)
[15:15] <acf_> No route to host? how does that happen?
[15:16] <m0unds> depends
[15:16] *** wallshot1 has joined #arpnetworks
[16:08] <acf_> Level3 seems kind of jittery since it dropped out earlier today
[17:21] <up_the_irons> jbergstroem: plett : yeah i got some alerts of peering trouble earlier today
[17:31] <up_the_irons> m0unds: whenever you mention apd it always reminds me of breaking bad, lol
[17:31] <m0unds> up_the_irons: bahaha
[17:33] <brycec> And it probably will for years to come
[17:33] <brycec> One of these days I'm going to finish watching BB
[17:33] <m0unds> yeah, it's either BB or USDOJ investigation
[17:33] <m0unds> alternates between those two things, since all people know about ABQ is that breaking bad was shot here and that APD is under investigation for use of force
[17:34] <brycec> I was unaware of the latter, actually
[17:34] <brycec> So congrats on that
[17:35] <m0unds> haha, i'm surprised. there was a shooting event where APD killed a batshit insane homeless guy and it made national news
[17:35] <m0unds> and stupid anonymous targeted cabq.gov and apd's stuff
[17:35] <up_the_irons> i was unaware of the latter too
[17:36] <m0unds> well geez, i'm disappointed now
[17:36] <m0unds> haha
[17:36] <m0unds> lots of shootings by apd in the last 10 yrs
[17:37] <brycec> I've heard of DOJ investigating similar charges for LAPD and SPD (Seattle), but ABQ slipped under my very, very high radar.
[17:37] <m0unds> haha, SPD was turned into a metro PD under recommendation by usdoj
[17:37] <brycec> o.o
[17:37] <m0unds> or am i thinking of LVPD
[17:37] <m0unds> or something
[17:37] <brycec> Ok. I have no idea what that "means
[17:37] <m0unds> federally administered police dept
[17:38] <m0unds> something like that
[17:38] <brycec> oh my
[17:38] <brycec> The town I live in has 6 cops :)
[17:38] <brycec> The PD is also administered (or governed, or something like that) by the county sheriff
[17:39] <m0unds> yeah, lots of small agencies work that way
[17:42] <m0unds> actually, seems like LV metro pd is that way too
[17:42] <m0unds> and they're huge (officer count-wise)
[17:42] <jbergstroem> up_the_irons: ok, thanks for letting me know
[17:43] <up_the_irons> it's the jurisdiction radius, basically.  counties are protected by sheriff's departments.  within a county, the cities can have their own municipal police department (and many do, e.g. LAPD, BPD (Burbank), GPD (Glendale), etc... around my area)
[17:43] <up_the_irons> cities that don't have their own PD get automatically covered by the sheriff
[17:43] <up_the_irons> and lastly, some cities simply "outsource" their PD to the sheriff (e.g. West Hollywood)
[17:44] <up_the_irons> those are called "contracted" cities
[17:44] <up_the_irons> and is why you see sheriff cars in an otherwise LAPD controlled area
[17:45] <up_the_irons> but i still don't know what "federally administered police dept" would be
[17:45] <up_the_irons> either
[17:45] <jbergstroem> up_the_irons: btw, next time i have similar issues should i just hit support@ ?
[17:45] <up_the_irons> jbergstroem: yup
[17:45] <jbergstroem> up_the_irons: done
[17:45] <m0unds> up_the_irons: usdoj handles administrative duty and training stuff for a police agency
[17:45] <up_the_irons> :)
[17:45] <up_the_irons> m0unds: roger
[17:48] *** siignal has joined #arpnetworks
[17:49] <m0unds> haha, add qwest/centurylink to the list of ISPs who run their NTT peering way too hot
[17:49] <m0unds> ugh
[17:50] *** siignal has left 
[17:52] <acf_> Verizon, AT&T, Comcast, CenturyLink/Qwest, TWC, Cox, Charter
[17:52] <m0unds> centurylink announces bogons
[17:52] <m0unds> yay
[17:54] <acf_> so does Level3
[17:54] <acf_> according to bgp.he.net
[17:54] <acf_> and Verizon
[17:55] <acf_> even the Verizon <-> Comcast peering is hot
[17:56] <m0unds> it's for the best - i don't want my house to burn down
[19:31] <mercutio> acf: i kind of wish there was more of an international monitoring/graphing/record of this stuff
[19:32] <mercutio> broken down as much as possible
[19:32] <acf_> yeah. it's kind of surprising that one doesn't exist actually
[19:32] <acf_> considering the magnitude of some of these problems
[19:32] <mercutio> probably because it's hard
[19:32] <mercutio> i mean you need 10,000+ peopel testing to really get good results.
[19:33] <acf_> I wonder if nanog ring could help any
[19:33] <acf_> you have access to test points from a bunch of different nsps/etc...
[19:33] <mercutio> still isn't it like 20 locations?
[19:33] <mercutio> you kind of want more ilke 50 locatiosn
[19:33] <mercutio> oh it' smore complicated than that
[19:34] <mercutio> like if you can select different providers in 50 locations..
[19:34] <gizmoguy> https://ring.nlnog.net/
[19:34] <m0unds> eggnog ring
[19:34] <acf_> https://ring.nlnog.net/participants/
[19:34] <mercutio> i got 50 from 7 countries and 6 places in each country + a bit
[19:34] <mercutio> i don't relaly know good numbers
[19:34] <mercutio> but i imagine some of them are near each other
[19:35] <gizmoguy> nlnog ring is probably the best combined network monitoring project outside of perfsonar
[19:35] <gizmoguy> our software used to be used to monitor the nlnog ring, not sure if it still is
[19:35] <mercutio> so like in the US, you want to test at least dallas, los angeles, san jose, virginia, new york, chicago, phoenix, denver, miami, seattle, ..
[19:35] <mercutio> i'm missing a few important network locations right?
[19:36] <m0unds> i still like ripe atlas, despite the black box nature of it
[19:36] <mercutio> and test the different providers in each of those regions
[19:36] <acf_> so nlnog is just like a big shell access exchange?
[19:37] <gizmoguy> https://stats.es.net/perfSONAR/directorySearch.html
[19:37] <mercutio> can you do throughput tesitng on nlonog?
[19:37] <mercutio> nlnog ring
[19:37] <gizmoguy> I suspect so
[19:37] <mercutio> i imagine it's mostly about network health
[19:37] <gizmoguy> what we did for them
[19:38] <acf_> oh cool
[19:38] <acf_> https://ring.nlnog.net/toolbox/
[19:38] <gizmoguy> is create a full mesh of all participants, and do regular traceroute/icmp
[19:38] <mercutio> some people have a few, but some are jst one
[19:38] <gizmoguy> in the mesh
[19:38] <mercutio> and record networks traversed and jitter and so on?
[19:39] <gizmoguy> naw they turned off our software
[19:39] <gizmoguy> https://ring.nlnog.net/news/2012/10/root-cause-analysis-using-amp/
[19:39] <mercutio> damn
[19:39] <gizmoguy> here's an old blog post
[19:40] <mercutio> is it open source?
[19:40] <gizmoguy> not at the moment
[19:40] <gizmoguy> http://wand.net.nz/amp/
[19:41] <gizmoguy> we're in the middle of a 4 year project to rewrite the whole thing
[19:41] <mercutio> i think the best way to do testing is to involve enough people that local issues on less well connected sitse don't matter
[19:41] <mercutio> rather than having "well connected" test sites.
[19:42] <mercutio> gizmoguy: do you ever look at geekzone forums?  they were talking about congesting happening on 2.4 gigabit ONT's that go to 24 houses or such.
[19:42] <mercutio> but a lot of people don't seem to really understand congestion well.
[19:42] <mercutio> and likely places rather than unlikely places for it to happen
[19:43] <gizmoguy> I try and stay well away from that place
[19:43] <mercutio> haha good idea
[19:43] <mercutio> i still think congestino is not widely understood
[19:43] <gizmoguy> do you have a link to that forum post?
[19:43] <gizmoguy> I have a few people I would like to share that with :)
[19:43] <mercutio> it's on the 1 gigabit fibre thread
[19:43] <acf_> like the 24 houses are using up 2.4 gigabit of bandwidth?
[19:44] <mercutio> http://www.geekzone.co.nz/forums.asp?forumid=49&topicid=147171&page_no=6#1064686
[19:44] <mercutio> somewhere in there
[19:44] <mercutio> acf: gpon is 2.4/1.2 gigabit
[19:44] <mercutio> 2.4 down 1.2 up
[19:44] <mercutio> adn gpon supports oversubscription
[19:44] <mercutio> so you can have 24 customers with gigabit connections.
[19:44] <mercutio> the question is, is it likely to congest?
[19:45] <mercutio> google fibre is the same diff
[19:45] <mercutio> shoudl be calling it "google fiber" because it's only available in the US?
[19:45] <mercutio> it's also probably less than 24 people on a node in practice
[19:45] <m0unds> the iprovo thing is such a boondoggle
[19:46] <mercutio> iprovo?
[19:46] <m0unds> paid for by taxpayer money and the city sold it to google for $1
[19:46] <mercutio> oh
[19:46] <mercutio> ir ead about it
[19:46] <m0unds> https://transmission.xmission.com/2013/04/18/the-1-fiber-optic-network
[19:46] <mercutio> but google are paying $30 million on construction loans
[19:46] <m0unds> which is small potatoes
[19:46] <mercutio> i think it sonuded reasonable enough to me
[19:47] <mercutio> did you see the thing about them paying $5/month in taxes wehterhet or not they take it up?
[19:47] <mercutio> i can't tyo.  s/wehterhet/whether/
[19:47] <m0unds> hahaha
[19:47] <mercutio> i can't type my mistypings either can it.
[19:47] <mercutio> -t
[19:47] <mercutio> argh
[19:47] <gizmoguy> urgh myfullflavour
[19:48] <m0unds> gah, i'm dropping keystrokes like crazy
[19:48] <mercutio> gizmoguy: they're not around here, so dont' know much about them.
[19:48] <gizmoguy> I designed and built their first network
[19:48] <gizmoguy> they didn't pay me
[19:48] <mercutio> i suppose you can see i posted a bit heh
[19:48] <mercutio> eek
[19:48] <gizmoguy> well they paid 1% of the bill or something
[19:49] <mercutio> well apparently they're upgrading their core network now
[19:49] <mercutio> lots ofp eople are upgrading core networks atm]
[19:49] <mercutio> but yeah, i was trying to determine if any sites could push anythhing close to gigabit
[19:50] <mercutio> the fastest speeds i've had are to you ai think gizmo :)
[19:50] <gizmoguy> :)
[19:51] <mercutio> but i imagine that server has less load than citylink's ftp mirror etc
[19:51] <mercutio> and it's rare to have much nz content of any good size
[19:51] <gizmoguy> http://ps02.reannz.co.nz/iso/
[19:51] <gizmoguy> see how fast you can pull that down at
[19:51] <mercutio> it's not loading
[19:51] <acf_> same
[19:52] <mercutio> it's reannz only isn't it?
[19:52] <acf_> so the question is
[19:52] <acf_> when everyone has google fibre
[19:52] <acf_> then what happens to our NTT congestion?
[19:52] <gizmoguy> mercutio: oh yes right. sorry those machines aren't on the internet vrf
[19:53] <mercutio> google use ntt i think?
[19:53] <acf_> hmm idk
[19:53] <acf_> its any2 from arp
[19:53] <mercutio> acf: it's ape from me (nz peering exchange)
[19:54] <mercutio> it's good they're well peered at least
[19:54] <acf_> they're peered directly to amazon too
[19:54] <acf_> and HE
[19:54] <mercutio> via telecom?
[19:54] <mercutio> err global gateway
[19:54] <acf_> ?
[19:54] <mercutio> what ip
[19:54] <mercutio> is it like 210.55 or something
[19:55] <acf_> http://paste.unixcube.org/k/2cad50
[19:55] <acf_> top HE, bottom amazon
[19:55] <acf_> not sure what you're looking for
[19:55] <mercutio> that's google
[19:55] <mercutio> isnt' it?
[19:56] <acf_> teah
[19:56] <acf_> yeah
[19:56] <mercutio> oh right
[19:56] <mercutio> i thought you meant gizmo's server for some reason
[19:56] <mercutio> sorry i'm slow
[19:56] <mercutio> but that's google's web site
[19:56] <mercutio> their fibre is different
[19:56] <mercutio> i tracked down an ip ages ago
[19:56] <acf_> right
[19:56] <mercutio> but i lost it
[19:56] <mercutio> they don't have peering in kansas
[19:56] <mercutio> at least not well peered
[19:57] <mercutio> and their traffic goes via dallas/chicago
[19:57] <acf_> hmm I can see why that would be bad
[19:57] <acf_> can you buy transit from google fiber?
[19:57] <mercutio> well kansas sucks for peering in general
[19:57] <mercutio> it's one of the annoying places :/
[19:57] <mercutio> comcast and verizon suck for peering there too.
[19:58] <acf_> yeah, I haven't much experience with kansas
[19:58] <acf_> http://bgp.he.net/AS16591
[19:58] <mercutio> so what it looked like google were doing, is doing their own backhaul to places with peering
[19:58] <mercutio> then buyign bandwidth adn peering there
[19:58] <acf_> that would make sense
[19:58] <mercutio> i've got a friend in wichita
[19:58] <mercutio> which is close enough
[19:58] <acf_> what kind of backhaul do they have?
[19:58] <mercutio> he can download at 3 megabytes/sec from nz at least.
[19:58] <acf_> if everybody has gigabit...
[19:59] <mercutio> on comcast i think that was
[19:59] <m0unds> they don't all have gbit
[19:59] <m0unds> they have svc levels like anything else
[19:59] <mercutio> acf: they've probably got 10 gigabit links
[19:59] <mercutio> which they can upgrade to 40 gigabit when they congest.
[19:59] <mercutio> you have to have a lot of users to congest dual 10 gigabit linsk.
[19:59] <acf_> I guess
[20:00] <acf_> so they have 2 plans
[20:00] <mercutio> and it's more prudent to get links to more places than to make the pipes fatter.
[20:00] <m0unds> do you have a file to grab from something in nz?
[20:00] <m0unds> i'm curious
[20:00] <mercutio> m0unds: there's heaps of various files around :/
[20:00] <acf_> mercutio: ok I can see that
[20:00] <mercutio> http://202.49.71.24:24/10m
[20:00] <mercutio> that's a 10m test file on my server
[20:00] <acf_> or potentially http://ftp.nz.debian.org/debian/
[20:00] <mercutio> my rationality is that once files get big you're not waiting for them to download
[20:01] <mercutio> so it's better to test with files that aren't huge
[20:01] <m0unds> my experience with even domestic linux mirrors is that they're under pretty regular load and can vary wildly
[20:01] <gizmoguy> 0.9s download time on that mercutio
[20:01] <m0unds> ~500KB/sec
[20:01] <mercutio> gizmoguy: that seems slow
[20:02] <mercutio> oh hangon 0.9 isn't too bad
[20:02] <mercutio> that's 10mb/sec
[20:02] <mercutio> are you connected at 100 megabit?
[20:02] <gizmoguy> 11.0M/s   in 0.9s
[20:02] <gizmoguy> yup 100mbit
[20:02] <mercutio> beacuse that's basically 100 megabit
[20:02] <m0unds> i can grab it faster from home than my arp vm
[20:02] <m0unds> hah
[20:02] <m0unds> a
[20:02] <mercutio> m0unds: haha
[20:03] <mercutio> it's ntt to arp
[20:03] <mercutio> i dunno if that says anything :/
[20:03] <m0unds> outbound is any2
[20:03] <m0unds> comcast it's level3
[20:03] <mercutio> yeah
[20:03] <mercutio> comcast forward path is verizon i think
[20:03] <m0unds> level3 for me
[20:03] <mercutio> yeah verizon
[20:03] <m0unds> or fwd from you?
[20:04] <mercutio> forward from me
[20:04] <m0unds> 195ms
[20:04] <mercutio> 500kb/ssec is actually painfulyl slow :/
[20:04] <mercutio> i wonder why it's going so slow
[20:04] <acf_> verizon/comcast peering seems a bit congested atm
[20:04] <acf_> through lax
[20:04] <m0unds> ^ could be that, because it routes me through lax
[20:05] <mercutio> yeah it will route via la or sj
[20:05] <mercutio> to nz
[20:05] <m0unds> the only time i see sjc for stuff out west is when things are screwy
[20:05] <acf_> http://kremvax.acfsys.net/smokeping.cgi?displaymode=n;start=2014-06-12%2017:04;end=now;target=Remote.verizonnet~comcastep
[20:05] <m0unds> because it typically does abq > den > sjc > elsewhere
[20:05] <m0unds> or sometimes, abq > den > dfw > sjc > elsewhere
[20:05] <mercutio> but yeah, nz is pretty good for doing 100 megabit speeds
[20:06] <mercutio> but i really doubt that much is connected at any faster than 1 gigabit
[20:06] <mercutio> asnd like i'm pretty sure gizmo's being slowed down by a bsaically arbitary 100 megabit limit
[20:06] <mercutio> and that if he cared enough he coudl get it fixed
[20:06] <mercutio> to gigabit
[20:06] <mercutio> like one router or switch or such
[20:11] <m0unds> haha, those debian mirrors are using a cdn
[20:12] <mercutio> nz.archive.ubuntu.com does which is what i use for testing nz speeds normally
[20:12] <mercutio> but can't give you :(
[20:12] <mercutio> cos it is unfair
[20:12] <mercutio> cdns do complicate things
[20:12] <m0unds> yeah, the node it's giving me for the debian one is in san jose
[20:12] <m0unds> that one is the same
[20:12] <m0unds> actually
[20:13] <m0unds> haha
[20:13] <m0unds> 16. ubuntu.citylink.co.nz                            0.0%    13   43.5  43.4  41.5  44.6   0.8
[20:13] <mercutio> ftp://wits.cs.waikato.ac.nz/ispdsl/2/20100106-030946-0.dsl.erf.gz
[20:13] <mercutio> i found something on gizmo's server to link :/
[20:13] <mercutio> but it's rated limited at 10 megabit on ipv4, and needs ipv6
[20:13] <mercutio> to go faster
[20:13] <m0unds> and i can't do v6 because my v6 tunnel is to arp, which is congested atm
[20:14] <mercutio> the pseed isn't great from arp for that too
[20:14] <m0unds> 58.5KB/s  eta 2h 50m
[20:14] <mercutio> 551k/sec average at 9.6mb
[20:14] <m0unds> ^ that's via the tunnel
[20:14] <mercutio> i tried to ^C around 10mb
[20:14] <mercutio> hahaha wow
[20:14] <m0unds> 604KB/s  eta 11m 55s that was direct
[20:14] <m0unds> via v4
[20:15] <mercutio> weird
[20:15] <gizmoguy> i can give you a big file on non-ratelimited connection if you want
[20:15] <mercutio> gizmo: how about a 10mb file? :)
[20:15] <m0unds> for science
[20:15] <gizmoguy> http://wand.net.nz/boot/mirror/isos/CentOS-6.4-x86_64-LiveDVD.iso
[20:15] <gizmoguy> oh
[20:15] <gizmoguy> 10 M
[20:15] <gizmoguy> sure
[20:15] <mercutio> you can just download my 10mb file
[20:15] <mercutio> it's /dev/urandom though
[20:16] <mercutio> dd if=/dev/urandom of=10m bs=1024k count=10
[20:16] <m0unds>    ] 16,915,140  3.70MB/s
[20:16] <m0unds> it's bouncing around, but highest speed was 5.7MB/sec
[20:16] <mercutio> there were some dodgy vps providers serving /dev/zero content over https
[20:16] <gizmoguy> http://wand.net.nz/boot/mirror/isos/10m
[20:16] <mercutio> https compresses such files and makes it faster
[20:16] <mercutio> 100 10.0M  100 10.0M    0     0  71.4M      0 --:--:-- --:--:-- --:--:-- 71.9M
[20:17] <gizmoguy> waikato netwokring is very broken at the moment
[20:17] <gizmoguy> unfortunately
[20:17] <mercutio> oh it was hitting proxy oosp
[20:17] <mercutio> 100 10.0M  100 10.0M    0     0  14.9M      0 --:--:-- --:--:-- --:--:-- 14.9M
[20:17] <m0unds> i hit 2.77MB/sec on that one
[20:18] <mercutio> curl -v http://wand.net.nz/boot/mirror/isos/10m > /dev/null  0.01s user 0.03s system 5% cpu 0.748 total
[20:18] <mercutio> hmm so it's not really a lot faster than your 100 megabit connectino
[20:19] <m0unds> hahaha
[20:19] <m0unds> i'm 7081mi from hamilton
[20:19] <mercutio> bah from home it hit proxy all the time :/
[20:19] <m0unds> or almost 11400km
[20:20] <mercutio> this is why i use port 24 :)
[20:20] <mercutio> m0unds: that's pretty close
[20:20] <m0unds> that's if there was a cable draped from NM straight to nz
[20:20] <m0unds> haha
[20:22] <mercutio> so yeah, even if i had gigabit compared to 100 megabit, it'd only be about 20% faster to a location 5 msec away
[20:22] <mercutio> oh it's 2.5 msec
[20:23] <mercutio> actually i think gizmo is right about something being broken
[20:33] *** pseudorandom has joined #arpnetworks
[21:19] *** pseudorandom has quit IRC (Quit: Leaving)
[21:51] <anisfarhana> Might be off topic, but does gmail have problem yesterday or a day before?
[21:54] <mnathani> brycec: what season/episode are you on?
[21:54] <mnathani> brycec: of breaking bad
[22:05] <mercutio> anis: gmail has lots ofp roblems affecting some users but not others and some data centres but not others
[22:05] <mercutio> so it can easily be fine for other people and broken for you
[22:06] <anisfarhana> http://www.wired.com/2014/06/gmail-bug-could-have-exposed-every-users-address/
[22:06] <BryceBot> Wired: "Gmail Bug Could Have Exposed Every User’s Address | Threat Level | WIRED"
[22:07] <anisfarhana> Many people complaining about this since yesterday.
[22:07] <anisfarhana> But today seem ok.
[22:07] <anisfarhana> Yesterday and a day before.
[22:07] <anisfarhana> they dont received any emails.
[22:07] <anisfarhana> No errors
[22:08] <wallshot1> my gmail spam folder has made me aware for years of just how many countless assholes have my email address
[22:09] <mercutio> weird
[22:09] <anisfarhana> And here too
[22:09] <anisfarhana> http://www.dailymail.co.uk/sciencetech/article-2655616/Massive-flaw-revealed-revealed-Gmail-address-EVERY-user-Googles-mail-service.html?ITO=1490&ns_mchannel=rss&ns_campaign=1490
[22:09] <mercutio> my gmail spam mail box only gets false positives
[22:10] <mercutio> it happens quite frequently
[22:10] <anisfarhana> No wonder many friends yesterday dont receive the emails!!
[22:10] <mercutio> oh wow
[22:11] <mercutio> mail just came through for me quickly
[22:11] <mercutio> i just emadiled myself :0
[22:18] <brycec> Haven't noticed any gmail issues...
[22:18] <brycec> mnathani: Don't remember exactly (but Netflix ought to), somewhere around season 3 or 4
[22:19] <brycec> (I think I last watched it back in December or thereabouts. But then life got busy, and...)
[23:55] *** wallshot1 has quit IRC (Ping timeout: 252 seconds)