[04:46] *** [FBI] starts logging #arpnetworks at Sun Feb 09 04:46:57 2014 [04:46] *** [FBI] has joined #arpnetworks [04:48] *** mike-bur1 is now known as mike-burns [06:07] *** Grumptober has quit IRC (Quit: FreeBSD Power) [06:47] *** tehfink has joined #arpnetworks [08:54] *** tehfink has quit IRC (Ping timeout: 252 seconds) [08:58] *** tehfink has joined #arpnetworks [09:16] <.< [09:16] >.> [09:17] No drop at ~6am, yay [09:18] hah [09:18] yeh [09:20] so it wsa just any2ix creating issues/ [09:20] i'd say no [09:22] he mentioned memory corruption, so i'd guess that it's hw or less likely, an ios bug [09:23] up_the_irons: TAC does do case-by-case TAC stuff, but it seems like not all of their support people have any idea how to even set it up. we had a series of HW issues in july, and since we don't keep TAC contracts (ime, they're worthless about 85% of the time), we had to escalate to a manager to get a one-time case opened [09:30] it sucks when random traffic can crash a router [09:46] It's a terrible router that can be crashed by random traffic :p [10:00] yeah, and it's terrible when "support engineers" are paid to search through old support cases and KB entries hoping they can find somethign similar [10:00] and that during an environment down emergency they'll just tell you "sorry, we don't have a product engineer available because it's saturday; can you wait til monday for a callback?" [10:00] mad awesome [10:03] wow [10:20] *** FreeSpencer has joined #arpnetworks [10:20] Was there ever an east coast location with ARPNetworks? [10:20] not yet [10:20] i think it's due to come this year [10:20] Has it always been LA? [10:20] yeh [10:21] Hmm nvm then! [10:21] i told you FreeSpencer [10:21] *** FreeSpencer has left "Textual IRC Client: www.textualapp.com" [10:21] lol [10:21] what [10:36] hahaha [10:36] LA is..east of hawaii? [10:37] Yup. [10:42] north east [11:12] i was meaning in the context of an "east" location, hahaha [11:12] i need a new office chair [11:44] oh right [11:57] I do too... Seemed like a fine chair when I bought it, but less than a year later the cylinder won't hold pressure (10 minutes is all it takes), and the right armrest doesn't attach to the back correctly anymore (leaning back is exciting). [12:03] it's a pita shpopoing for new office chairs [12:03] most aren't very good [12:04] and how are you meant to know if it'll fail in a year's time withotu researching [12:04] my old chair lost gas acutally [12:04] but it was also really worn by then [12:04] adn it was about 7 years old [12:04] i dunno how long they're meant to last [12:15] *** staticsafe has quit IRC (Remote host closed the connection) [12:15] *** staticsafe has joined #arpnetworks [12:24] *** staticsafe has quit IRC (Quit: WeeChat 0.4.3) [12:25] *** staticsafe has joined #arpnetworks [12:56] anybody notice v6 issues? [12:56] I can ping6 google, but not facebook, and not my end of a he.net tunnel [12:58] pinging fine for me facebook v6 [12:58] from your arpnetworks vps? [13:01] yup [13:01] http://pastebin.com/cCTnCzXG [13:02] could be due to your gigabit connectivity? [13:03] I'd expect google would not work then [13:03] can you ping6 0.v.freedaemon.com ? [13:04] oh google doesn't work via v6 [13:05] yeah likely something was not saved or something, the odd thing though is I see the v6 router [13:05] http://sprunge.us/DXTG [13:07] 5 packets transmitted, 0 received, 100% packet loss, time 4101ms ( 0.v.freedaemon.com ) [13:19] sent a support request. [13:55] Are you experiencing the same on all your vpsen? [13:56] I guess they are all in the same VLAN [14:28] mercutio: yea, i had my previous one for 9 years..it was a cheapie, but it was uncomfortable by about year 4. i just couldn't find any i liked, so i gave up on it [14:28] my chair is annyoing me [14:28] but it's liek $700 for a good chair [14:29] i just bought some commercial grade one like i have at work [14:29] Build your own tractor seat chair. [14:32] haha [14:35] officemax had a $30 off coupon thing, so this one ended up being almost $150 off w/their office furniture promotion [14:35] http://www.wired.com/reviews/2013/06/office-chair-gallery/ [14:35] Wired: "You're Gonna Want to Sit Down for This: 6 High-End Desk Chairs Reviewed | Product Reviews | Wired.com" [14:36] brycec: does BryceBot have URL shortening capabilities? [14:39] haha, nice chairs [14:39] cheapest one is only roughly 175% of the cost of this one, hahahaha [14:39] see they're all expensive [14:39] yea [14:39] http://www.officemax.com/catalog/2013/Customs/WorkPro/cst-workpro-chair4.png [14:39] i thought i should splash out on a chair sometime [14:39] I'm perfectly comfortable withot armrests. [14:39] but i'd want to do enoqugh research [14:40] i can go either way - arm rests get in the way of my desk [14:40] also, my cat likes armrests [14:40] that looks like my chair sort of [14:40] he likes to stand up and paw at my arm to get my attention [14:40] only sort of [14:40] the back looks diff [14:40] i wasn't a fan of mesh backs at first, but i grew to like how well it breathes [14:40] i have mesh with a soft headrest type thing [14:40] especially when it's warm [14:41] yeah - that's how my work one is, has a mesh headrest [14:41] i'm too tall for that though, so i just removed it [14:41] it's that where you sit down is to thin that seems to my biggest issue [14:41] http://www.target.com/p/tractor-chair-black/-/A-13704390?ref=tgt_adv_XSG10001&AFID=Google_PLA_df&LNM=%7C13704390&CPNG=Furniture&kpid=13704390&LID=PA&ci_src=17588969&ci_sku=13704390&gclid=CKDU5r6OwLwCFUcV7AodRS4ALw [14:41] and tends to thin in the middle after owning longer [14:41] I used to have that. [14:41] now that i think about it it's probably fixable [14:42] i suppose you could probably open it up and replace whatever cushion type thing is in there [14:42] I was perfectly comfortable in it, but the plastic eventually cracked in the seat. [14:42] I bought a metal tractor seat off of ebay with the intention of replacing it. [14:42] Never got around to it though. [14:42] arctulaly it looks like it's not that easy [14:43] and would involve unscrewing the bottom [14:43] yea, or the cushion might be glued in place and screwed in place [14:43] it doesn't matter if it glued [14:43] like, wood board with cushion and upholstery glued to it, then the whole thing screwed into the base [14:43] you just pack another one on in middle [14:44] TWSS [14:44] Okay! twss! 'you just pack another one on in middle' [14:44] the thing is i don't even weigh a lot [14:44] the upholstery would probably fail and it'd look like you beat the stuffing out of the seat, hahaha [14:44] i must sit a lot ;/ [14:44] hahahaha [14:44] get one of those kneeling seats [14:45] http://vvro.us/1f7JlPB they look goofy [14:45] it's like a bicycle seat and part of a massage chair or something [14:45] oh i tried one of those once [14:46] Can't wank in those. [14:46] never got used to it [14:46] you could, but you'd probably break your kneck [14:46] neck* [14:46] kneck? wtf. [14:46] i dno't think the one i used had wheels [14:46] * m0unds sighs [14:46] yeah, i'd think the wheels would make it more dangerous [14:46] i like the idea of a standing desk [14:46] have you guys heard of that? [14:46] yeah, i did that for a few weeks [14:47] how'd it go? [14:47] raised my workstation up to chest level [14:47] pretty well, but i found that it was hard to get my displays quite right [14:47] i have them on arms, but they didn't have enough tilt to get them to where i could look at them naturally [14:47] hmm [14:47] now that you mention it i'd probably have that issue too [14:47] my legs felt great though [14:47] haha [14:47] i want to try it [14:48] but my desk can't raise that high [14:48] heh [14:48] put another desk on top of your desk [14:48] did your normal desk support it? [14:48] that soudns dangerous [14:48] only if you don't anchor that sucker [14:48] earthquakes don't happen here, but wheer i used to live i had to move from because of earthquakes [14:48] so i'm paranoid now [14:49] yeah, my desk at work is adjustable for height [14:49] like i'm always thinking where things might fall :) [14:49] http://www.katu.com/news/local/Snow-plow-rolls-blocks-Highway-105--244634991.html yeesh [14:49] is the US still screwed up by weather? [14:50] it was like < 0c highs when i last looked [14:50] it's 65F here today; should be 30-40 [14:50] i forgot the command [14:50] @wa 65F in C [14:50] i think [14:50] 65f is 18.3c [14:50] convert 65 °F (degrees Fahrenheit) to degrees Celsius;18.33 °C (degrees Celsius);291.48 K (kelvins);524.67 °R (degrees Rankine);14.667 °Ré (degrees Réaumur);17.125 °Rø (degrees Rømer);1.667 °C below temperature at STP (standard temperature and pressure), using the National Institute of Standards and Technology convention (20 °C);2.778 °C above temperature at STP (standard temperature and pressure), using the US customary convent [14:50] so slightly cold [14:50] @wa 30F in C [14:50] ^ that's what it should be [14:50] convert 30 °F (degrees Fahrenheit) to degrees Celsius;-1.111 °C (degrees Celsius);272.04 K (kelvins);489.67 °R (degrees Rankine);-0.88889 °Ré (degrees Réaumur);6.9167 °Rø (degrees Rømer);1.111 °C below melting point of water ice (at standard pressure) (0 °C);1.111 °C below temperature at STP (standard temperature and pressure), using the International Union of Pure and Applied Chemistry convention (0 °C);1.11 °C below ice poin [14:50] @wa 0c in f [14:50] 32 [14:50] convert 0 °C (degrees Celsius) to degrees Fahrenheit;32 °F (degrees Fahrenheit);273.15 K (kelvins);491.67 °R (degrees Rankine);0 °Ré (degrees Réaumur);7.5 °Rø (degrees Rømer);= melting point of water ice (at standard pressure) (0 °C);= temperature at STP (standard temperature and pressure), using the International Union of Pure and Applied Chemistry convention (0 °C);~~ ice point of water (273.15 K);temperature;[temperature];Ther [14:50] i know that offhand [14:50] haha [14:51] oh [14:51] but yea, should be around that as a high where i live, but it's been in the 60s on and off for two weeks [14:51] nice [14:51] driving me nuts [14:51] i want snow [14:51] so do i [14:51] it's only 23c here atm [14:51] but it might get hotter :( [14:51] i wish it could abbbbbbbereviate [14:51] 23c=73.4f [14:52] heh my repeat value a bit high [14:53] do you ever get stuck at work? [14:53] by snow [15:17] *** josephb has quit IRC (Ping timeout: 252 seconds) [15:18] umm, i've gotten stuck there twice [15:19] once for ice and once for snow [15:19] biggest snow i can remember here was ~60cm or so [15:19] *** josephb has joined #arpnetworks [16:02] Haha, it does, but they're not enabled (last I checked). BryceBot only has URL de-shortening :D 14:35:21 >> m0unds<< brycec: does BryceBot have URL shortening capabilities? [16:04] (Because brycec hates shortened URLs) [16:12] haha debian is falling apart [16:39] *** tehfink has quit IRC (Quit: tehfink) [16:53] mercutio: init debacle you mean? [17:00] up_the_irons: yes [17:00] people are asking Ian to resign. [17:00] https://lists.debian.org/debian-ctte/2014/02/msg00363.html [17:00] it's like a university. [17:01] on a positive note i figured out how to get notrack to work. [17:01] mo drammuh [17:01] i want to see systemd go ahead. [17:02] i just wanna see something modern and well engineered [17:02] but OMGZ NAHT INIT NAHT GUN UZE IT [17:02] well i find systemd a little confusing [17:02] but it has grown on me over time [17:03] and nearly everyone except ubuntu is using it [17:03] and ubuntu seems to just want to do things differently for the sake of it [17:03] i also want to see waylong go ahead rather than mir [17:03] yea, i felt the same way about systemd, but it makes a lot of sense in a lot of ways [17:04] and i like how you can specify dependencies for startup and stuff [17:04] i still haven't found a linux i really like [17:05] i've started using arch for personal systems [17:05] but i don't think it's really suitable for critical systems [17:06] not really a fan of ubuntu, centos, or debian. debian and centos have outdated packages. [17:06] ubuntu at least has 2 yearly long term versions, but has a few annoying things [17:07] so in the end i end up using ubuntu anyway. but it doesn't mean it's ideal. [17:08] i kind of think debian has to look at two year release cycle or such [17:09] actually it looks like it does have two year release cycle now [17:09] i thin it was woody -> sarge when i moved to ubuntu [17:10] mercutio: outdated packages == system stability. keep in mind, they do backport security fixes [17:10] yeah i had problems with php 4 and apache 1.3 ? [17:11] debian was quite far behind with php [17:11] that said i shifted to ubuntu 6 monthly rather than two yearly [17:11] php is a bit of a nightmare though [17:11] * jpalmer uses CentOS at work. [17:11] do you like centos? [17:11] yeah, I personally avoid PHP like the devil. [17:11] if your hw is supported by debian or centos, it works great [17:12] but modern raid cards and stuff can be a huge hassle [17:12] i kind of would like to see things like tmux stay current [17:12] or be included [17:12] depending on mfgr [17:12] whereas core thingss can be a bit more stable [17:12] i've been using custom kernels for a while now [17:12] m0unds: that advice was valid 15 years ago [17:12] mercutio: At this point, all of the unixen are very similar to me. I use puppet, so.. I just define what I think the system should look like.. and puppet "makes it happen" [17:12] is it still the way it is? :) [17:12] it's still valid. [17:12] haha [17:13] puppet is great [17:13] the annoying thing i foundwith centos is it had lots of braindead config by default which required tweaking [17:13] i like openbsd having bare config, but still keeping important things there [17:13] lots of LSI cards have issues with 2.6.x kernels, as much as that shouldn't happen [17:13] interetsing [17:14] it does and it's annoying [17:14] lsi 9211 is fine isn't it? [17:14] mercutio: I don't really consider that a valid concern/issue. as an admin, I very rarely ever install anything, and leave a default config. tweaking and such are a fact of life [17:14] true i just feel like it's nice when systems depart from defaults in known ways. . [17:14] i think it was things like ntp from memory [17:14] no idea, but a client of mine managed to find three separate lsi card models made in the last 2 years that would crash and die pretty regularly [17:15] m0unds: ouch. [17:15] i thought it was the disks at first, but then noticed that happened with every disk we tried [17:15] i saw a problem with lsi and western digital disks [17:15] i was wondering what was happening :/ [17:15] but in the end it magically worked with seagate disks [17:16] yeah, that sort of behavior is total crap in 2014 [17:16] hahaha [17:16] yeah this was older hardware [17:16] but still [17:16] i tried emailing lsi and western digital [17:16] western digital don't have any alternative firmwares or firmware updates [17:16] yeah, they only make firmware updates available if they screw something up during production [17:16] at least that's been my exp with them [17:16] lsi won't support cards that use their chipset [17:16] * jpalmer ponders spinning up a VM to play with katello [17:17] i think it was intel lsi card [17:17] * m0unds has no idea what katello is [17:17] oh, neat [17:17] it's kind of bad there's only wd and seagate making disks now imo [17:17] sounds like fun [17:18] i never used to try emailing vendors [17:18] but i've tried a bit recently [17:18] asrock have been the best so far [17:18] i've heard their support is really good [17:19] m0unds: foreman, pulp, candlepin, puppet [17:20] an interesting point on the debian thing is that people probably see it [debian] as different things. arguments on cross-os is probably aligned towards package management and set of rules while others just want "one" operating system. [17:20] jpalmer: yeah, was just checking out the project's site - sounds neat [17:21] i changed my knobset for my garage entry door and keep locking myself out of the house [17:22] whoops, wrong channel [17:22] haha [19:12] the packet loss on NTT has gotten really bad [19:12] http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcast6net [19:12] http://kremvax.acfsys.net/smokeping.cgi?target=Remote.verizon-snloca [19:19] *** tabthorpe has quit IRC (Read error: Connection reset by peer) [19:19] *** tabthorpe has joined #arpnetworks [19:21] *** KDE_Perry has quit IRC (Ping timeout: 260 seconds) [19:22] *** KDE_Perry has joined #arpnetworks [19:27] wow [19:27] are you sure it's not some icmp limit? [19:28] you're not pinging a comcast router are you [19:29] no, that's comcast.com [19:29] it still might be deprioritisng [19:29] i'm doing mtr to that now [19:29] see how the hop before doesn't have packet loss [19:29] oh now it does [19:29] haha [19:29] ok i think it is prob an issue [19:29] yeah, my mtrs can confirm that [19:29] oh that's strange my route is via akamai from home [19:30] ntt is a lot more jittery for me than nlayer, haha [19:30] oh you need to leave off www [19:30] yeah they been havving issues recently it seems [19:30] it seems to come and go a bit [19:30] this is really bad though [19:30] hmm comcast is fine from home [19:30] but it's not going via ntt [19:31] ikr [19:31] i hadn't seen issues on arp [19:31] i think it's random like some ip's may be hit worse than others [19:31] that's what it seems like [19:31] so yeh 0%packet loss with 100 pings from home [19:31] 7.1% with 111 packets from arp [19:32] I think it's the return path [19:32] it's hard to check that [19:32] but i can check a different forward path [19:32] well, the return path when pinging comcast->arp [19:32] return to arp via comcast? [19:32] lemme do that (comcast in albuquerque here) [19:32] ok got a ntt forward path [19:32] http://paste.unixcube.org/k/ccdaba [19:33] it's being problematic sntraight away [19:33] my outbound path is via nlayer [19:33] same here [19:33] restarting it in case it was "getting used to" new traffic [19:33] it got weird after that last reboot [19:33] ntt is random [19:33] return path from arp to comcast used to be symmetrical [19:33] it seems to randomly have issues on/off [19:33] so, nlayer out, nlayer back [19:33] yeah i'm definitely htiting it with forward path via ntt [19:33] but not via verizon [19:34] well on the positive side it's easier to route around [19:34] if up_the_irons will do so [19:34] is up_the_irons planning on making level3 the default outbound path? [19:35] mind you there's that annaying 100k route limit [19:35] i imagine comcast has some big subnets [19:35] there's about 560 comcast routes [19:36] oh less than that cos double ups etc [19:36] about 180 [19:36] arp networks has level3? [19:36] they're getting it [19:36] :o [19:37] contract signed, but not in yet [19:37] i assume it shouldn't take that long [19:37] but you never know [19:37] yay for more variety [19:37] http://pastebin.com/8CDG5nAg [19:38] i thin kit's actually as6453 creating the isssue btw [19:38] i've seen issues with as6453 before [19:38] but that was to europe [19:38] of course that's a hunch rather than antyhing proven [19:38] yea, tata sucks [19:38] comcast has a funny thing w/tata [19:39] they're where comcast traffic goes when you don't want to peer directly with comcast [19:39] comcast has a funny thing with cogent too [19:39] i would say that comcast suck and no-one should care [19:39] but i realise comcast is all most people have available [19:39] other than at&t dsl :/ [19:39] you don't know too much about us telecom :P [19:39] i know a bit [19:39] there's lots of options, comcast is the best in most areas [19:39] there's verizon fios and cox in some areas [19:39] but for instance san jose has just got comcast and at&t [19:40] cox sucks, verizon isn't great and their installers like setting stuff on fire [19:40] but yeah i don't know that much [19:40] cox use cogent [19:40] verizon use vdsl? [19:40] my friend in Charlotte, NC has either AT&T DSL or TWC [19:40] verizon is adsl2 [19:40] verizon vdsl2 or fios [19:40] err, adsl2 [19:40] i know that at&t suck [19:40] yea, twc is bad [19:40] staticsafe: is twc any good? [19:41] they can't afford to upgrade systems [19:41] they're way behind [19:41] no idea, I'm not in the US either :p [19:41] they almost were part of a hostile takeover [19:41] heh [19:41] because their customers hate them [19:41] nice [19:41] but they're now being forced to actually start investing in infrastructure [19:41] not much better over in Ontario, Canada though :( [19:42] http://stopthecap.com/2014/01/30/time-warner-cable-plans-to-triple-broadband-speeds-if-they-survive-a-hostile-takeover/ [19:42] err [19:42] wrong link [19:43] http://www.theverge.com/2014/1/27/5351082/charter-reportedly-selling-comcast-twc-assets-if-buyout-succeeds that was the one i wanted to link [19:43] stupid chrome [19:43] The Verge: "Comcast reportedly wants to nab millions of customers from Time Warner Cable" [19:44] well at least i don't have any usage caps [19:44] haha, neither do i [19:45] i like usage caps [19:45] i'd rather have fast net than congested net [19:45] Jan 8/14 - Feb 7/14 826GB [19:46] that said i don't have cap [19:46] but i like other people to have caps [19:46] so my net keeps going fast [19:46] lol. [19:46] show interfaces terse [19:46] err, wrong terminal [19:46] dd if=/dev/urandom of=/dev/sd.... [19:47] i love trying to explain to people how to extract tarballs [19:47] and then they ask what every bit does :/ [19:48] xz -dc blah.tar.xz | tar -vxf - [19:48] etc [19:48] i don't even know if tar has an option for xz yet [19:48] tar -xvZf ? [19:48] does that do xz [19:48] i kind of got appehensive when it was I and j for bzip2 [19:49] and then you found a system that had neither [19:49] haha, 248.94GB down, 23.66 up [19:49] hmm [19:49] 4.7 gig yesterday, 8 gig the ady before [19:50] yeh i think i'm a light user [19:50] and roughly 13.8% of my traffic was ipv6 [19:50] how do you work that out? [19:50] it's tunneled, so i just graph the ipip if on my srx [19:50] som\e days i only do 1.3gig [19:51] oh i see [19:51] so it's an approximation, but i'd say it's reasonable [19:51] most of that is probably netflix, oddly enough [19:51] yah [19:51] i reckon 500gb caps all around isn't a bad idea [19:51] and pay a little more for 2tb cap or something [19:51] or even 250 gig for light [19:51] i pay a little more for "business internet", and it has no cap [19:51] but i am against the idea of 1 gig 5 gig etc caps [19:52] comcast's 3mbit-16mbit usage tiers are 250gb cap [19:52] 50mbit is 500gb and i think 100mbit is 1tb [19:52] or something [19:52] seems reasonable [19:52] can everyone get 50 megabit [19:52] major metros can, yep [19:53] so no [19:53] i think most markets support up to 16mbit [19:53] so can you pay extra on a 3 to 16 megabit plan to get more than 250gb? [19:53] doubt it [19:54] well [19:54] actually [19:54] if you pay an extra like $15/mo, you can get biz svc and have no cap [19:54] and static ip? [19:54] $20 for no cap and static IP [19:54] ahh ok [19:55] i'd probably just do that [19:55] if i had to use comcast [19:55] my frieend is on comcast it didn't seem that good though [19:55] 50 msec pings from sj to sj [19:55] high jitter [19:55] throughput to arp was ok though [19:55] sounds like routing is silly [19:55] i think 20 msec to arp [19:55] yeah [19:55] i ping 9ms to denver [19:55] it's screwed to cogent :/ [19:56] that's ~400mi from me [19:56] but seems to have high affinity for cogent or something [19:56] and i ping 30ms to los angeles, and that's twice as far as denver [19:57] oh shit, hahah [19:57] i switched over to hbo and didn't realize for almost 5 minutes that it was hbo latino [20:05] m0unds: wow that's good [20:05] my friend in kansas had terrible routing [20:05] evernyything went via chicago or texas or both [20:05] and neither had very low pings [20:07] 80ms for me via v4 to ARP [20:11] which is why i thought it was curious google were doing fibreeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee [20:11] erk [20:11] my computer crashed ;/ [20:11] i was trying to undervolt to try and get it to run cooler [20:11] i think i need another fan though [20:12] it's like 40c system [20:12] google's doing screwy stuff to deploy fiber [20:12] i dont' think kansas has very good routing [20:12] it's kind of the middle of nowhere, haha [20:12] i didn't have lots of destinations to try, but there's joes data centre for instance [20:12] which would go via texas or something [20:13] https://transmission.xmission.com/2013/04/18/the-1-fiber-optic-network [20:13] he wasn't in kansas city [20:13] wichita [20:13] which doesnt' maen a lot to me [20:13] i looked on google maps, and it looked desolate [20:13] yeah [20:13] haha [20:13] us is kinda big [20:14] he had a few internet options that sucked differently [20:14] i think it's 140 msec to arp for me [20:14] and like 200 to 220 msec to him [20:15] but it's like 200 to 220 to east coast too [20:15] and can be 160 to texas, 180 to chicago etc [20:15] ouch [20:16] texas seems to be one of the best places for routing i've found [20:16] less outages, less congestion etc [20:16] yea, well connected, especially DFW [20:16] but that's from external pov [20:16] atlanta is pretty good too, depending on the facility [20:16] but kansas, atlanta, , florida etc are bad [20:17] i found routing often got indirect to atlanta [20:17] virginia is better [20:17] i'm 60ms to atlanta, or 65ms to VA [20:17] haha [20:17] and like tampa would often go via miami [20:17] everything US basically goes through Chicago for me [20:18] atlanta to tampa would go like via virgina and miami if you're lucky [20:18] otherwise it'd go via chicago or new york [20:18] you're in canada? [20:18] yea [20:18] hmm [20:18] toronto? [20:18] yes [20:18] i thought most canada stuff went through seattle or new york [20:18] seattle if west coast [20:19] but chicago is probably better [20:19] seattle routing from here isn't wonderful [20:19] luckily hardly anything is in seattle :) [20:19] other than valve [20:19] when the east coast had flooding europe connectivity was really broken from here though [20:20] cos like europe goes via west coast us east coast us [20:20] and lots of stuff went down [20:20] i think there's a path to europe via japan? [20:21] but the vast majority relies on new yorkish [20:21] and somewhere south of there [20:21] which is also exposed [20:21] but it's a bit crazy when things 10s of thousands of km away can effect you [20:22] maybe they will find a way to put fibre thorugh the core of the earth one day [20:22] *** anisfarhana has quit IRC (Ping timeout: 252 seconds) [21:06] acf_: we run full routing tables, so there really isn't a strict default outbound path [21:07] *** qwerty_ has joined #arpnetworks [21:08] *** qwerty_ has left [21:09] mercutio: m0unds : yeah but it crashed within 24 hours of having the any2ix circuit spanned to it. all three times. i've since removed it, and no crash. [21:10] m0unds: noted on single TAC case, thanks [21:11] up_the_irons: good luck - that's so weird though w/the any2ix circuit [21:16] m0unds: mercutio : i also had probs with LSI cards [21:16] jpalmer: pulp and candlepin? i gotta google those... [21:21] m0unds: might not be so weird if it is an IOS bug. just need to get a newer one and try it. [21:21] m0unds: may have not liked some traffic on that shared network. lots of crap tends to come through IX's ;) [21:22] haha, true [21:23] $ sudo ./birdcl sh p | grep Est [21:23] AS45177 BGP master up 04:51:03 Established [21:23] AS40528 BGP master up 04:51:02 Established [21:23] AS4739 BGP master up 04:51:52 Established [21:23] AS15133 BGP master up 04:51:03 Established [21:23] AS6939 BGP master up 04:51:05 Established [21:23] AS11798 BGP master up 04:51:03 Established [21:23] AS20144 BGP master up 04:51:02 Established [21:23] AS46489 BGP master up 04:51:04 Established [21:23] AS20161 BGP master up 14:20:24 Established [21:23] $ [21:23] First few Any2 peers on BIRD :) [21:26] nice [21:28] grepidemic: taking this public [21:28] grepidemic: BUT, for logging, the complexity might be too much [21:28] (we're talking about journald) [21:29] grepidemic: that part is kinda neat, yes [21:29] it is pretty intuitive though. just have to learn the arguments for journalctl [21:29] the arch wiki has a really simplistic rundown of it [21:30] https://wiki.archlinux.org/index.php/systemd#Journal [21:32] grepidemic: neat [21:32] i'm dont know everything about systemd yet, but just from looking at journald, i can get used to it. [21:32] this part sounds brittle, "If you or some program delete that directory, systemd will not recreate it automatically; however, it will be recreated during the next update of the systemd package. Until then, logs will be written to /run/systemd/journal, and logs will be lost on reboot" [21:33] the probability that debian testing will be using systemd prompted me to read more into it. [21:33] I kinda dislike systemd removing choice of syslogger from hte equation [21:33] wow it uses btrfs [21:33] but that's systemd philosophy in general [21:34] the large amount of changes makes me uncomfortable [21:34] I'm (fortunately) on a os where I can choose to avoid systemd if I have to. Will probably revisit down the road [21:35] it is a big change... but it is looking like systemd will be the default for most linus distros in the near future. [21:35] all the big ones at least. [21:37] i think syslog-ng can be used with systemd/journald [21:37] grepidemic: they'll probably tune down the changes though, or introduce them more slowly [21:38] there is still a long list of bugs in systemd. [21:38] much longer than sysvinit [21:38] this made the rounds today as well: http://ewontfix.com/14/ [21:41] up_the_irons: it seems that NTT is having a lot of issues with Comcast and Verizon atm [21:41] http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcastnet [21:42] http://kremvax.acfsys.net/smokeping.cgi?target=Remote.verizon-snloca [21:42] acf_: can you provide forward and reverse path? if possible [21:42] of course [21:45] regarding having to reboot after an update, it looks like systemd can be restarted with "systemctl daemon-reload" [21:48] yea [21:49] when packages make changes to systemd startup configs, you have to reload the daemon too [21:50] or if you add a new service, say you install nginx or something - you'll reload the daemon and it'll become aware of nginx and allow you to control it [21:50] comcast: http://paste.unixcube.org/k/bf88a3 [21:50] verizon: http://paste.unixcube.org/k/78f880 [21:50] but it detects that something's different and prompts you to reload it (at least in the case of arch) [21:53] acf_: thanks [21:56] acf_: some of those routers might rate limit ICMP (we do also) [21:57] my stance on systemd is still neutral, but i do really like journald. i would use it with sysvinit if that is possible (i'm gonna go look that up). [21:57] up_the_irons: for stuff passing through? [21:57] up_the_irons: i adjusted trafic to send out via ntt and hit the same issue [21:57] with reverse path intact [21:57] and wasn't seeing packet loss normally [21:58] it could be rate limiting sending icmp through, it does happen, but i don't think it's likely [21:58] and i've seen more than one weird ntt packet loss issue recently [21:58] mercutio: well, traceroutes never really "pass thru"; what you see is the response to the TTL dying, and some of those ICMPs will be dropped by CPU rate limiting [21:58] and i know it slows ssh down and that's when i notice teh packet loss in mtr [21:58] up_the_irons: yehn but it's to every hop until the destination [21:58] including the destination [21:58] mercutio: ok, if ssh slows, then that is more of a concern [21:58] usually if it's rate limiting you'll see like 20% 20% 0% [21:58] or such [21:59] or 20% 15% 0% 35% [21:59] etc [21:59] grepidemic: guessing no [21:59] like often one hop doesn't rate limiting icmp as much [21:59] a normal 1 second interval ping shows packet loss here [21:59] err rate likmit [21:59] you may want to contact their NOC (noc@us.ntt.net) [21:59] i know it's psuedo science [21:59] up_the_irons: it was too diff destination i saw ssh packet loss [21:59] err ssh lag [21:59] i can't ssh to comcast [21:59] and it was very on/off [21:59] starbucks kicking me out (closing), bbiab [22:00] nlayer rate limit a lot of icmp btw [22:00] it's annoying :/ [22:00] fwiw, NTT noc says " [22:00] We are not seeing any packet loss within our network. To investigate this issue further please contact your upstream provider." [22:03] haha [22:03] shit [22:03] i don't think it's necessarily ntt's fault in this instance [22:03] my guess is overloaded interconnects [22:03] we determined it could be tata [22:04] it's relaly hard to know [22:04] how about NTT <-> Verizon [22:04] cos of what up_the_irons said about icmp rate limiting [22:04] got a test destination? [22:04] yeah [22:04] ok tell me i'll test it [22:04] but i'll make another coffee first :) [22:05] 72.55.8.69 for comcast [22:05] 108.40.173.223 for verizon (beware super slow dsl) [22:05] but no packet loss to google [22:06] i repeated the comcast issue [22:06] I agree it's probably not in the NTT backbone [22:06] ok verizon normal path is via alter.net [22:07] (verizon) [22:07] which a little jitter [22:07] is AS2914 ntt [22:07] yip cool [22:08] verizon looks fine too [22:09] via ntt [22:09] which means it's prob reverse path if chornically bad [22:09] oh no some packet loss [22:09] it seems like the loss is only at the lax verizon <-> NTT exchange btw [22:09] it's not nearly as bad as that comcast one was [22:10] I get http://paste.unixcube.org/k/e57747 [22:10] the jitter is worse via ntt too [22:10] this is via lax [22:10] i screwed my up my other route i don't think i can send via sj [22:10] oh i can send via sj [22:11] interesting [22:11] via san jose only seeing packet loss on last hop and in betwen hops [22:11] err and some early ones [22:11] which always depriortise [22:11] but via los angeles seing it on last 3 hops [22:11] err 4 hops [22:11] but i changed source ip gah [22:11] yep. that's what I've been seeing [22:12] last 4 hops? [22:12] yeah [22:12] why am i only seeing last hop [22:12] bah i'll fowrad it through oso i can keep to same source address [22:13] the jitter is def worse via ntt [22:13] ok last two hops now [22:13] and it's either wose right now [22:13] or sj is worse than la for forward path with reverse path combined too [22:13] i'm seeing 17% packet loss [22:14] http://paste.unixcube.org/k/78f880 [22:14] the bottom one basically describes what I've been seeing for the last few days [22:14] yeah that's what i was seeing orig [22:14] hangon i'll pastebin [22:15] actually what's your paste thing :) [22:15] http://paste.unixcube.org/ [22:15] runs from arp [22:15] heh [22:15] cool [22:15] http://paste.unixcube.org/k/e57f7a [22:15] it went down a bit [22:15] so sampling bias :) [22:15] it seems like it gets better and worse [22:16] yeah, that's been happening a lot for me [22:16] so ok, it doesn't matter if it's sj or la [22:16] and it's better over alter.net [22:16] i'll shift it back to alter.net [22:17] and wait a minute or two [22:17] oh only being last two hops i think is cos shorter route? [22:18] i having problems cutting grr [22:18] i hate linux cutg and pate [22:18] paste [22:19] http://paste.unixcube.org/k/7dfe16 [22:19] weird going from top to bottom fixed it [22:20] that one looks nice [22:20] oh i got a packet dropped [22:20] that's like what was happeing with comcast too :/ [22:20] maybe my crappy dsl [22:20] http://paste.unixcube.org/k/8ad869 [22:20] well the first one had depriorisation on some hops [22:20] packet loss stops when return path changes (NTT <-> Verizon in Dallas instead of LAX) [22:21] but still packet loss to destination is WAY down [22:21] and the source ip remained the same [22:21] so basically it doesn't matter if it goes via sj or la from ntt [22:21] they both suck [22:21] but alter.net is fine [22:22] well when i talk about fowrard path you're talking about return path right [22:22] forward path for me is my traffic going to your dsl [22:22] and return path for you is traffic coming to you [22:22] yep, I'm switching them [22:22] ok so dallas is better [22:23] yeah [22:23] los angeles and san jose may share some hidden path [22:23] like i don't know what city you in [22:23] but i'm sure it's not in california [22:23] lsanca == San Luis Obispo, CA [22:23] so there's probably an in between point hidden [22:23] you are in california? [22:23] everything from here goes though lax [22:23] yep [22:23] verizon dsl sucks [22:23] dude your internet sucks [22:24] i have better pings to new york :/ [22:24] sometimes it's not as bad lol [22:24] *lsanca snloca [22:24] yeah i'm in new zealand which you may have been able to tell from those traces ;) [22:24] and that's from dsl [22:25] why woudl it e going via dallas [22:25] ntt.net is in tx [22:25] oh hmm [22:25] ntt.net is japanese [22:25] us.ntt.net goes the same place afaik [22:26] hmm i found somewhere with nlayer to trace from [22:26] it's still jittery but not lossy [22:26] it's nlayer -> alter.net -> verizon [22:26] the arp path does that once in a while [22:27] ahh ok [22:27] yeh it looks fine [22:27] yesterday I think it did that and the packet loss stopped [22:27] i don't think up_the_irons wants to get involved in screwing with routes lots [22:27] yeah that sounds messy [22:27] heh i did it a bit before [22:27] but i just did sweeping generalisations [22:27] like "prepend anything that says cogent" [22:28] so that if a non cogent route is available it tcan take that [22:28] if it's not much longer [22:28] and some shift for at&t traffic [22:28] if it's just verizon and comcast which both have heaps of users maybe he'd make adjustments? [22:29] 1% packet loss via nlayer now heh [22:29] maybe. it seems like he has ntt prioritized heavily? [22:29] it hink your dsl does drop packets a bit too [22:29] likely [22:29] it's default route [22:29] cos the old router has 100k table limit [22:29] comcast has 180 routes [22:29] i dunno how many verizon have [22:29] plenty I'm sure [22:30] but i reckon we should try and get them to be part of it :) [22:30] not necessarily [22:30] bigger providers often have less prefixes [22:30] as they just get huge chunks [22:30] i can check [22:31] yeh [22:31] verizon have heaps [22:31] 198.181.111.0/24, 198.181.112.0/24 [22:31] etc [22:31] nearly 1500 [22:31] oh that can't be aggregated [22:32] cos that would be 110 and 111 [22:32] 198.105.10.0/24 and 198.105.11.0/24 [22:32] that can be aggregated though [22:32] and 198.105.8.0/24 and 198.105.9.0/24 [22:33] "AS701 announces bogons" [22:33] why would they do that? [22:33] do they [22:34] http://bgp.he.net/AS701#_asinfo [22:34] 192.203.48.0/20 looks dodgy [22:34] nah it looks fine actually [22:35] what bogons [22:35] http://bgp.he.net/AS701#_bogons [22:35] oh wow 192.168.0.0/16 [22:36] oh 198.168.0.0/16 [22:36] wow [22:36] 192 is mostly legacy space [22:36] up_the_irons: we talked a lot while you were away [22:36] i saw the scrollback [22:37] i dunno how bogus these ips really are [22:37] it's more just that no-one has notes on what they are [22:37] traffic issues on other networks is a PITA. You fix one thing, another thing breaks. It never ends. [22:38] up_the_irons: yes [22:39] which is why it only makes sense if it's severe [22:39] I would think NTT and Verizon would have decent networks and be fairly responsive to issues of this severity [22:39] oh well [22:39] well ntt have been having issues only recently right? [22:40] i reckon comcast is more worth fixing than verizon [22:40] they have way less routes too [22:41] ho hmm [22:42] ok got a comcast dsl ip [22:43] weird i'm not seeing issues, only jhitter at last hop [22:43] things are looking better [22:43] on comcast [22:43] http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcastnet [22:43] oh [22:43] wow so on/off [22:43] is comcast ip static normally? [22:44] I don't think so [22:44] will get him to setup smokepnig maybe :) [22:48] comcast tends to assign "sticky" IPs - so long as your cpe doesnt change and isnt powered off for longer than it takes to release the dhcp lease, youll likely have the same addr for a long time. [22:48] yeh i'll get him to curl some file randomly [22:48] when i had residential svc, my dhcp addr stuck with me for over a year [22:48] he's running into some depency issues [22:48] oh interesting. [22:48] usually when people say sticky ip it's sticky until you want to access it [22:49] it only changed when they brought new cmts gear up [22:49] at least in my case [22:49] oh [22:49] maybe they'll do that to him [22:49] is 4 msec jitter normal? [22:50] yea [22:50] it's 0.1 jitter two hops prior [22:50] (and worse the hop immediately before [22:50] oh i thought that may mean he was legacy node [22:51] i dont tend to worry til it hits +/- 15-20ms continually [22:51] well both traces have go ne over 500 seconds / 500 pings with no loss [22:51] hm [22:51] i worry about 2 or 3 msec jitter normally [22:51] but maye i shouldn't [22:51] for idle connectinos [22:51] it tends to suggest there's some congestion somewehre [22:52] but it depends on bandwidth etc [22:52] like 2 to 3 msec jitter on gigabit means it's pretty packed [22:52] also, not sure if this is still the case, but when looking at comcast rdns for intermediate gear, he = 100gbit, te = 10gbit [22:53] since they use lotsa j gear [22:54] lots of fun hw to play with [22:55] this is te [23:53] thinking about it more, latency between verizon and ntt in lax has always doubled from ~30 to ~60 for me [23:55] ie, at last verizon hop is 30ms and first ntt hop is 60ms