***: [FBI] has joined #arpnetworks
mike-bur1 is now known as mike-burns
Grumptober has quit IRC (Quit: FreeBSD Power)
tehfink has joined #arpnetworks
tehfink has quit IRC (Ping timeout: 252 seconds)
tehfink has joined #arpnetworks brycec: <.<
>.>
No drop at ~6am, yay mercutio: hah
yeh
so it wsa just any2ix creating issues/ m0unds: i'd say no
he mentioned memory corruption, so i'd guess that it's hw or less likely, an ios bug
up_the_irons: TAC does do case-by-case TAC stuff, but it seems like not all of their support people have any idea how to even set it up. we had a series of HW issues in july, and since we don't keep TAC contracts (ime, they're worthless about 85% of the time), we had to escalate to a manager to get a one-time case opened mercutio: it sucks when random traffic can crash a router brycec: It's a terrible router that can be crashed by random traffic :p m0unds: yeah, and it's terrible when "support engineers" are paid to search through old support cases and KB entries hoping they can find somethign similar
and that during an environment down emergency they'll just tell you "sorry, we don't have a product engineer available because it's saturday; can you wait til monday for a callback?"
mad awesome mercutio: wow ***: FreeSpencer has joined #arpnetworks FreeSpencer: Was there ever an east coast location with ARPNetworks? mercutio: not yet
i think it's due to come this year FreeSpencer: Has it always been LA? mercutio: yeh FreeSpencer: Hmm nvm then! staticsafe: i told you FreeSpencer ***: FreeSpencer has left "Textual IRC Client: www.textualapp.com" staticsafe: lol mercutio: what m0unds: hahaha
LA is..east of hawaii? mike-burns: Yup. mercutio: north east m0unds: i was meaning in the context of an "east" location, hahaha
i need a new office chair mercutio: oh right brycec: I do too... Seemed like a fine chair when I bought it, but less than a year later the cylinder won't hold pressure (10 minutes is all it takes), and the right armrest doesn't attach to the back correctly anymore (leaning back is exciting). mercutio: it's a pita shpopoing for new office chairs
most aren't very good
and how are you meant to know if it'll fail in a year's time withotu researching
my old chair lost gas acutally
but it was also really worn by then
adn it was about 7 years old
i dunno how long they're meant to last ***: staticsafe has quit IRC (Remote host closed the connection)
staticsafe has joined #arpnetworks
staticsafe has quit IRC (Quit: WeeChat 0.4.3)
staticsafe has joined #arpnetworks toddf: anybody notice v6 issues?
I can ping6 google, but not facebook, and not my end of a he.net tunnel mnathani: pinging fine for me facebook v6 toddf: from your arpnetworks vps? mnathani: yup
http://pastebin.com/cCTnCzXG
could be due to your gigabit connectivity? toddf: I'd expect google would not work then
can you ping6 0.v.freedaemon.com ?
oh google doesn't work via v6
yeah likely something was not saved or something, the odd thing though is I see the v6 router
http://sprunge.us/DXTG mnathani: 5 packets transmitted, 0 received, 100% packet loss, time 4101ms ( 0.v.freedaemon.com ) toddf: sent a support request. mnathani: Are you experiencing the same on all your vpsen?
I guess they are all in the same VLAN m0unds: mercutio: yea, i had my previous one for 9 years..it was a cheapie, but it was uncomfortable by about year 4. i just couldn't find any i liked, so i gave up on it mercutio: my chair is annyoing me
but it's liek $700 for a good chair m0unds: i just bought some commercial grade one like i have at work CaZe: Build your own tractor seat chair. m0unds: haha
officemax had a $30 off coupon thing, so this one ended up being almost $150 off w/their office furniture promotion mercutio: http://www.wired.com/reviews/2013/06/office-chair-gallery/ BryceBot: Wired: "You're Gonna Want to Sit Down for This: 6 High-End Desk Chairs Reviewed | Product Reviews | Wired.com" m0unds: brycec: does BryceBot have URL shortening capabilities?
haha, nice chairs
cheapest one is only roughly 175% of the cost of this one, hahahaha mercutio: see they're all expensive m0unds: yea
http://www.officemax.com/catalog/2013/Customs/WorkPro/cst-workpro-chair4.png mercutio: i thought i should splash out on a chair sometime CaZe: I'm perfectly comfortable withot armrests. mercutio: but i'd want to do enoqugh research m0unds: i can go either way - arm rests get in the way of my desk
also, my cat likes armrests mercutio: that looks like my chair sort of m0unds: he likes to stand up and paw at my arm to get my attention mercutio: only sort of
the back looks diff m0unds: i wasn't a fan of mesh backs at first, but i grew to like how well it breathes mercutio: i have mesh with a soft headrest type thing m0unds: especially when it's warm
yeah - that's how my work one is, has a mesh headrest
i'm too tall for that though, so i just removed it mercutio: it's that where you sit down is to thin that seems to my biggest issue CaZe: http://www.target.com/p/tractor-chair-black/-/A-13704390?ref=tgt_adv_XSG10001&AFID=Google_PLA_df&LNM=%7C13704390&CPNG=Furniture&kpid=13704390&LID=PA&ci_src=17588969&ci_sku=13704390&gclid=CKDU5r6OwLwCFUcV7AodRS4ALw mercutio: and tends to thin in the middle after owning longer CaZe: I used to have that. mercutio: now that i think about it it's probably fixable
i suppose you could probably open it up and replace whatever cushion type thing is in there CaZe: I was perfectly comfortable in it, but the plastic eventually cracked in the seat.
I bought a metal tractor seat off of ebay with the intention of replacing it.
Never got around to it though. mercutio: arctulaly it looks like it's not that easy
and would involve unscrewing the bottom m0unds: yea, or the cushion might be glued in place and screwed in place mercutio: it doesn't matter if it glued m0unds: like, wood board with cushion and upholstery glued to it, then the whole thing screwed into the base mercutio: you just pack another one on in middle CaZe: TWSS BryceBot: Okay! twss! 'you just pack another one on in middle' mercutio: the thing is i don't even weigh a lot m0unds: the upholstery would probably fail and it'd look like you beat the stuffing out of the seat, hahaha mercutio: i must sit a lot ;/ m0unds: hahahaha
get one of those kneeling seats
http://vvro.us/1f7JlPB they look goofy
it's like a bicycle seat and part of a massage chair or something mercutio: oh i tried one of those once CaZe: Can't wank in those. mercutio: never got used to it m0unds: you could, but you'd probably break your kneck
neck*
kneck? wtf. mercutio: i dno't think the one i used had wheels -: m0unds sighs m0unds: yeah, i'd think the wheels would make it more dangerous mercutio: i like the idea of a standing desk
have you guys heard of that? m0unds: yeah, i did that for a few weeks mercutio: how'd it go? m0unds: raised my workstation up to chest level
pretty well, but i found that it was hard to get my displays quite right
i have them on arms, but they didn't have enough tilt to get them to where i could look at them naturally mercutio: hmm
now that you mention it i'd probably have that issue too m0unds: my legs felt great though
haha mercutio: i want to try it
but my desk can't raise that high
heh m0unds: put another desk on top of your desk mercutio: did your normal desk support it?
that soudns dangerous m0unds: only if you don't anchor that sucker mercutio: earthquakes don't happen here, but wheer i used to live i had to move from because of earthquakes
so i'm paranoid now m0unds: yeah, my desk at work is adjustable for height mercutio: like i'm always thinking where things might fall :) m0unds: http://www.katu.com/news/local/Snow-plow-rolls-blocks-Highway-105--244634991.html yeesh mercutio: is the US still screwed up by weather?
it was like < 0c highs when i last looked m0unds: it's 65F here today; should be 30-40 mercutio: i forgot the command m0unds: @wa 65F in C
i think mercutio: 65f is 18.3c BryceBot: convert 65 °F (degrees Fahrenheit) to degrees Celsius;18.33 °C (degrees Celsius);291.48 K (kelvins);524.67 °R (degrees Rankine);14.667 °Ré (degrees Réaumur);17.125 °Rø (degrees Rømer);1.667 °C below temperature at STP (standard temperature and pressure), using the National Institute of Standards and Technology convention (20 °C);2.778 °C above temperature at STP (standard temperature and pressure), using the US customary convent mercutio: so slightly cold m0unds: @wa 30F in C
^ that's what it should be BryceBot: convert 30 °F (degrees Fahrenheit) to degrees Celsius;-1.111 °C (degrees Celsius);272.04 K (kelvins);489.67 °R (degrees Rankine);-0.88889 °Ré (degrees Réaumur);6.9167 °Rø (degrees Rømer);1.111 °C below melting point of water ice (at standard pressure) (0 °C);1.111 °C below temperature at STP (standard temperature and pressure), using the International Union of Pure and Applied Chemistry convention (0 °C);1.11 °C below ice poin mercutio: @wa 0c in f m0unds: 32 BryceBot: convert 0 °C (degrees Celsius) to degrees Fahrenheit;32 °F (degrees Fahrenheit);273.15 K (kelvins);491.67 °R (degrees Rankine);0 °Ré (degrees Réaumur);7.5 °Rø (degrees Rømer);= melting point of water ice (at standard pressure) (0 °C);= temperature at STP (standard temperature and pressure), using the International Union of Pure and Applied Chemistry convention (0 °C);~~ ice point of water (273.15 K);temperature;[temperature];Ther m0unds: i know that offhand
haha mercutio: oh m0unds: but yea, should be around that as a high where i live, but it's been in the 60s on and off for two weeks mercutio: nice m0unds: driving me nuts
i want snow mercutio: so do i
it's only 23c here atm
but it might get hotter :(
i wish it could abbbbbbbereviate
23c=73.4f
heh my repeat value a bit high
do you ever get stuck at work?
by snow ***: josephb has quit IRC (Ping timeout: 252 seconds) m0unds: umm, i've gotten stuck there twice
once for ice and once for snow
biggest snow i can remember here was ~60cm or so ***: josephb has joined #arpnetworks brycec: Haha, it does, but they're not enabled (last I checked). BryceBot only has URL de-shortening :D 14:35:21 >> m0unds<< brycec: does BryceBot have URL shortening capabilities?
(Because brycec hates shortened URLs) mercutio: haha debian is falling apart ***: tehfink has quit IRC (Quit: tehfink) jbergstroem: mercutio: init debacle you mean? mercutio: up_the_irons: yes
people are asking Ian to resign.
https://lists.debian.org/debian-ctte/2014/02/msg00363.html
it's like a university.
on a positive note i figured out how to get notrack to work. m0unds: mo drammuh mercutio: i want to see systemd go ahead. m0unds: i just wanna see something modern and well engineered
but OMGZ NAHT INIT NAHT GUN UZE IT mercutio: well i find systemd a little confusing
but it has grown on me over time
and nearly everyone except ubuntu is using it
and ubuntu seems to just want to do things differently for the sake of it
i also want to see waylong go ahead rather than mir m0unds: yea, i felt the same way about systemd, but it makes a lot of sense in a lot of ways
and i like how you can specify dependencies for startup and stuff mercutio: i still haven't found a linux i really like
i've started using arch for personal systems
but i don't think it's really suitable for critical systems
not really a fan of ubuntu, centos, or debian. debian and centos have outdated packages.
ubuntu at least has 2 yearly long term versions, but has a few annoying things
so in the end i end up using ubuntu anyway. but it doesn't mean it's ideal.
i kind of think debian has to look at two year release cycle or such
actually it looks like it does have two year release cycle now
i thin it was woody -> sarge when i moved to ubuntu jpalmer: mercutio: outdated packages == system stability. keep in mind, they do backport security fixes mercutio: yeah i had problems with php 4 and apache 1.3 ?
debian was quite far behind with php
that said i shifted to ubuntu 6 monthly rather than two yearly
php is a bit of a nightmare though -: jpalmer uses CentOS at work. mercutio: do you like centos? jpalmer: yeah, I personally avoid PHP like the devil. m0unds: if your hw is supported by debian or centos, it works great
but modern raid cards and stuff can be a huge hassle mercutio: i kind of would like to see things like tmux stay current
or be included m0unds: depending on mfgr mercutio: whereas core thingss can be a bit more stable
i've been using custom kernels for a while now
m0unds: that advice was valid 15 years ago jpalmer: mercutio: At this point, all of the unixen are very similar to me. I use puppet, so.. I just define what I think the system should look like.. and puppet "makes it happen" mercutio: is it still the way it is? :) m0unds: it's still valid.
haha
puppet is great mercutio: the annoying thing i foundwith centos is it had lots of braindead config by default which required tweaking
i like openbsd having bare config, but still keeping important things there m0unds: lots of LSI cards have issues with 2.6.x kernels, as much as that shouldn't happen mercutio: interetsing m0unds: it does and it's annoying mercutio: lsi 9211 is fine isn't it? jpalmer: mercutio: I don't really consider that a valid concern/issue. as an admin, I very rarely ever install anything, and leave a default config. tweaking and such are a fact of life mercutio: true i just feel like it's nice when systems depart from defaults in known ways. .
i think it was things like ntp from memory m0unds: no idea, but a client of mine managed to find three separate lsi card models made in the last 2 years that would crash and die pretty regularly mercutio: m0unds: ouch. m0unds: i thought it was the disks at first, but then noticed that happened with every disk we tried mercutio: i saw a problem with lsi and western digital disks
i was wondering what was happening :/
but in the end it magically worked with seagate disks m0unds: yeah, that sort of behavior is total crap in 2014
hahaha mercutio: yeah this was older hardware
but still
i tried emailing lsi and western digital
western digital don't have any alternative firmwares or firmware updates m0unds: yeah, they only make firmware updates available if they screw something up during production
at least that's been my exp with them mercutio: lsi won't support cards that use their chipset -: jpalmer ponders spinning up a VM to play with katello mercutio: i think it was intel lsi card -: m0unds has no idea what katello is m0unds: oh, neat mercutio: it's kind of bad there's only wd and seagate making disks now imo m0unds: sounds like fun mercutio: i never used to try emailing vendors
but i've tried a bit recently
asrock have been the best so far m0unds: i've heard their support is really good jpalmer: m0unds: foreman, pulp, candlepin, puppet jbergstroem: an interesting point on the debian thing is that people probably see it [debian] as different things. arguments on cross-os is probably aligned towards package management and set of rules while others just want "one" operating system. m0unds: jpalmer: yeah, was just checking out the project's site - sounds neat
i changed my knobset for my garage entry door and keep locking myself out of the house
whoops, wrong channel
haha acf_: the packet loss on NTT has gotten really bad
http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcast6net
http://kremvax.acfsys.net/smokeping.cgi?target=Remote.verizon-snloca ***: tabthorpe has quit IRC (Read error: Connection reset by peer)
tabthorpe has joined #arpnetworks
KDE_Perry has quit IRC (Ping timeout: 260 seconds)
KDE_Perry has joined #arpnetworks mercutio: wow
are you sure it's not some icmp limit?
you're not pinging a comcast router are you acf_: no, that's comcast.com mercutio: it still might be deprioritisng
i'm doing mtr to that now
see how the hop before doesn't have packet loss
oh now it does
haha
ok i think it is prob an issue acf_: yeah, my mtrs can confirm that mercutio: oh that's strange my route is via akamai from home m0unds: ntt is a lot more jittery for me than nlayer, haha mercutio: oh you need to leave off www
yeah they been havving issues recently it seems
it seems to come and go a bit acf_: this is really bad though mercutio: hmm comcast is fine from home
but it's not going via ntt
ikr
i hadn't seen issues on arp
i think it's random like some ip's may be hit worse than others acf_: that's what it seems like mercutio: so yeh 0%packet loss with 100 pings from home
7.1% with 111 packets from arp acf_: I think it's the return path mercutio: it's hard to check that
but i can check a different forward path acf_: well, the return path when pinging comcast->arp m0unds: return to arp via comcast?
lemme do that (comcast in albuquerque here) mercutio: ok got a ntt forward path acf_: http://paste.unixcube.org/k/ccdaba mercutio: it's being problematic sntraight away m0unds: my outbound path is via nlayer acf_: same here mercutio: restarting it in case it was "getting used to" new traffic m0unds: it got weird after that last reboot mercutio: ntt is random m0unds: return path from arp to comcast used to be symmetrical mercutio: it seems to randomly have issues on/off m0unds: so, nlayer out, nlayer back mercutio: yeah i'm definitely htiting it with forward path via ntt
but not via verizon
well on the positive side it's easier to route around
if up_the_irons will do so acf_: is up_the_irons planning on making level3 the default outbound path? mercutio: mind you there's that annaying 100k route limit
i imagine comcast has some big subnets
there's about 560 comcast routes
oh less than that cos double ups etc
about 180 staticsafe: arp networks has level3? mercutio: they're getting it staticsafe: :o mercutio: contract signed, but not in yet
i assume it shouldn't take that long
but you never know staticsafe: yay for more variety m0unds: http://pastebin.com/8CDG5nAg mercutio: i thin kit's actually as6453 creating the isssue btw
i've seen issues with as6453 before
but that was to europe
of course that's a hunch rather than antyhing proven m0unds: yea, tata sucks
comcast has a funny thing w/tata
they're where comcast traffic goes when you don't want to peer directly with comcast mercutio: comcast has a funny thing with cogent too
i would say that comcast suck and no-one should care
but i realise comcast is all most people have available
other than at&t dsl :/ m0unds: you don't know too much about us telecom :P mercutio: i know a bit m0unds: there's lots of options, comcast is the best in most areas mercutio: there's verizon fios and cox in some areas
but for instance san jose has just got comcast and at&t m0unds: cox sucks, verizon isn't great and their installers like setting stuff on fire mercutio: but yeah i don't know that much
cox use cogent
verizon use vdsl? staticsafe: my friend in Charlotte, NC has either AT&T DSL or TWC acf_: verizon is adsl2 m0unds: verizon vdsl2 or fios
err, adsl2 mercutio: i know that at&t suck m0unds: yea, twc is bad mercutio: staticsafe: is twc any good? m0unds: they can't afford to upgrade systems
they're way behind staticsafe: no idea, I'm not in the US either :p m0unds: they almost were part of a hostile takeover mercutio: heh m0unds: because their customers hate them mercutio: nice m0unds: but they're now being forced to actually start investing in infrastructure staticsafe: not much better over in Ontario, Canada though :( m0unds: http://stopthecap.com/2014/01/30/time-warner-cable-plans-to-triple-broadband-speeds-if-they-survive-a-hostile-takeover/
err
wrong link
http://www.theverge.com/2014/1/27/5351082/charter-reportedly-selling-comcast-twc-assets-if-buyout-succeeds that was the one i wanted to link
stupid chrome BryceBot: The Verge: "Comcast reportedly wants to nab millions of customers from Time Warner Cable" staticsafe: well at least i don't have any usage caps m0unds: haha, neither do i mercutio: i like usage caps
i'd rather have fast net than congested net staticsafe: Jan 8/14 - Feb 7/14 826GB mercutio: that said i don't have cap
but i like other people to have caps
so my net keeps going fast staticsafe: lol. m0unds: show interfaces terse
err, wrong terminal mercutio: dd if=/dev/urandom of=/dev/sd....
i love trying to explain to people how to extract tarballs
and then they ask what every bit does :/
xz -dc blah.tar.xz | tar -vxf -
etc
i don't even know if tar has an option for xz yet acf_: tar -xvZf ? mercutio: does that do xz
i kind of got appehensive when it was I and j for bzip2
and then you found a system that had neither m0unds: haha, 248.94GB down, 23.66 up mercutio: hmm
4.7 gig yesterday, 8 gig the ady before
yeh i think i'm a light user m0unds: and roughly 13.8% of my traffic was ipv6 mercutio: how do you work that out? m0unds: it's tunneled, so i just graph the ipip if on my srx mercutio: some days i only do 1.3gig
oh i see m0unds: so it's an approximation, but i'd say it's reasonable
most of that is probably netflix, oddly enough mercutio: yah
i reckon 500gb caps all around isn't a bad idea
and pay a little more for 2tb cap or something
or even 250 gig for light m0unds: i pay a little more for "business internet", and it has no cap mercutio: but i am against the idea of 1 gig 5 gig etc caps m0unds: comcast's 3mbit-16mbit usage tiers are 250gb cap
50mbit is 500gb and i think 100mbit is 1tb
or something mercutio: seems reasonable
can everyone get 50 megabit m0unds: major metros can, yep mercutio: so no m0unds: i think most markets support up to 16mbit mercutio: so can you pay extra on a 3 to 16 megabit plan to get more than 250gb? m0unds: doubt it
well
actually
if you pay an extra like $15/mo, you can get biz svc and have no cap mercutio: and static ip? m0unds: $20 for no cap and static IP mercutio: ahh ok
i'd probably just do that
if i had to use comcast
my frieend is on comcast it didn't seem that good though
50 msec pings from sj to sj
high jitter
throughput to arp was ok though m0unds: sounds like routing is silly mercutio: i think 20 msec to arp
yeah m0unds: i ping 9ms to denver mercutio: it's screwed to cogent :/ m0unds: that's ~400mi from me mercutio: but seems to have high affinity for cogent or something m0unds: and i ping 30ms to los angeles, and that's twice as far as denver
oh shit, hahah
i switched over to hbo and didn't realize for almost 5 minutes that it was hbo latino mercutio: m0unds: wow that's good
my friend in kansas had terrible routing
evernyything went via chicago or texas or both
and neither had very low pings staticsafe: 80ms for me via v4 to ARP mercutio: which is why i thought it was curious google were doing fibreeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
erk
my computer crashed ;/
i was trying to undervolt to try and get it to run cooler
i think i need another fan though
it's like 40c system m0unds: google's doing screwy stuff to deploy fiber mercutio: i dont' think kansas has very good routing m0unds: it's kind of the middle of nowhere, haha mercutio: i didn't have lots of destinations to try, but there's joes data centre for instance
which would go via texas or something m0unds: https://transmission.xmission.com/2013/04/18/the-1-fiber-optic-network mercutio: he wasn't in kansas city
wichita
which doesnt' maen a lot to me
i looked on google maps, and it looked desolate m0unds: yeah
haha
us is kinda big mercutio: he had a few internet options that sucked differently
i think it's 140 msec to arp for me
and like 200 to 220 msec to him
but it's like 200 to 220 to east coast too
and can be 160 to texas, 180 to chicago etc staticsafe: ouch mercutio: texas seems to be one of the best places for routing i've found
less outages, less congestion etc m0unds: yea, well connected, especially DFW mercutio: but that's from external pov m0unds: atlanta is pretty good too, depending on the facility mercutio: but kansas, atlanta, , florida etc are bad
i found routing often got indirect to atlanta
virginia is better m0unds: i'm 60ms to atlanta, or 65ms to VA
haha mercutio: and like tampa would often go via miami staticsafe: everything US basically goes through Chicago for me mercutio: atlanta to tampa would go like via virgina and miami if you're lucky
otherwise it'd go via chicago or new york
you're in canada? staticsafe: yea mercutio: hmm m0unds: toronto? staticsafe: yes mercutio: i thought most canada stuff went through seattle or new york staticsafe: seattle if west coast mercutio: but chicago is probably better
seattle routing from here isn't wonderful
luckily hardly anything is in seattle :)
other than valve
when the east coast had flooding europe connectivity was really broken from here though
cos like europe goes via west coast us east coast us
and lots of stuff went down
i think there's a path to europe via japan?
but the vast majority relies on new yorkish
and somewhere south of there
which is also exposed
but it's a bit crazy when things 10s of thousands of km away can effect you
maybe they will find a way to put fibre thorugh the core of the earth one day ***: anisfarhana has quit IRC (Ping timeout: 252 seconds) up_the_irons: acf_: we run full routing tables, so there really isn't a strict default outbound path ***: qwerty_ has joined #arpnetworks
qwerty_ has left up_the_irons: mercutio: m0unds : yeah but it crashed within 24 hours of having the any2ix circuit spanned to it. all three times. i've since removed it, and no crash.
m0unds: noted on single TAC case, thanks m0unds: up_the_irons: good luck - that's so weird though w/the any2ix circuit up_the_irons: m0unds: mercutio : i also had probs with LSI cards
jpalmer: pulp and candlepin? i gotta google those...
m0unds: might not be so weird if it is an IOS bug. just need to get a newer one and try it.
m0unds: may have not liked some traffic on that shared network. lots of crap tends to come through IX's ;) m0unds: haha, true up_the_irons: $ sudo ./birdcl sh p | grep Est
AS45177 BGP master up 04:51:03 Established
AS40528 BGP master up 04:51:02 Established
AS4739 BGP master up 04:51:52 Established
AS15133 BGP master up 04:51:03 Established
AS6939 BGP master up 04:51:05 Established
AS11798 BGP master up 04:51:03 Established
AS20144 BGP master up 04:51:02 Established
AS46489 BGP master up 04:51:04 Established
AS20161 BGP master up 14:20:24 Established
$
First few Any2 peers on BIRD :) m0unds: nice up_the_irons: grepidemic: taking this public
grepidemic: BUT, for logging, the complexity might be too much
(we're talking about journald)
grepidemic: that part is kinda neat, yes grepidemic: it is pretty intuitive though. just have to learn the arguments for journalctl
the arch wiki has a really simplistic rundown of it
https://wiki.archlinux.org/index.php/systemd#Journal up_the_irons: grepidemic: neat grepidemic: i'm dont know everything about systemd yet, but just from looking at journald, i can get used to it. up_the_irons: this part sounds brittle, "If you or some program delete that directory, systemd will not recreate it automatically; however, it will be recreated during the next update of the systemd package. Until then, logs will be written to /run/systemd/journal, and logs will be lost on reboot" grepidemic: the probability that debian testing will be using systemd prompted me to read more into it. jbergstroem: I kinda dislike systemd removing choice of syslogger from hte equation up_the_irons: wow it uses btrfs jbergstroem: but that's systemd philosophy in general up_the_irons: the large amount of changes makes me uncomfortable jbergstroem: I'm (fortunately) on a os where I can choose to avoid systemd if I have to. Will probably revisit down the road grepidemic: it is a big change... but it is looking like systemd will be the default for most linus distros in the near future.
all the big ones at least.
i think syslog-ng can be used with systemd/journald up_the_irons: grepidemic: they'll probably tune down the changes though, or introduce them more slowly grepidemic: there is still a long list of bugs in systemd.
much longer than sysvinit jbergstroem: this made the rounds today as well: http://ewontfix.com/14/ acf_: up_the_irons: it seems that NTT is having a lot of issues with Comcast and Verizon atm
http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcastnet
http://kremvax.acfsys.net/smokeping.cgi?target=Remote.verizon-snloca up_the_irons: acf_: can you provide forward and reverse path? if possible acf_: of course grepidemic: regarding having to reboot after an update, it looks like systemd can be restarted with "systemctl daemon-reload" m0unds: yea
when packages make changes to systemd startup configs, you have to reload the daemon too
or if you add a new service, say you install nginx or something - you'll reload the daemon and it'll become aware of nginx and allow you to control it acf_: comcast: http://paste.unixcube.org/k/bf88a3
verizon: http://paste.unixcube.org/k/78f880 m0unds: but it detects that something's different and prompts you to reload it (at least in the case of arch) up_the_irons: acf_: thanks
acf_: some of those routers might rate limit ICMP (we do also) grepidemic: my stance on systemd is still neutral, but i do really like journald. i would use it with sysvinit if that is possible (i'm gonna go look that up). mercutio: up_the_irons: for stuff passing through?
up_the_irons: i adjusted trafic to send out via ntt and hit the same issue
with reverse path intact
and wasn't seeing packet loss normally
it could be rate limiting sending icmp through, it does happen, but i don't think it's likely
and i've seen more than one weird ntt packet loss issue recently up_the_irons: mercutio: well, traceroutes never really "pass thru"; what you see is the response to the TTL dying, and some of those ICMPs will be dropped by CPU rate limiting mercutio: and i know it slows ssh down and that's when i notice teh packet loss in mtr
up_the_irons: yehn but it's to every hop until the destination
including the destination up_the_irons: mercutio: ok, if ssh slows, then that is more of a concern mercutio: usually if it's rate limiting you'll see like 20% 20% 0%
or such
or 20% 15% 0% 35%
etc jbergstroem: grepidemic: guessing no mercutio: like often one hop doesn't rate limiting icmp as much acf_: a normal 1 second interval ping shows packet loss here mercutio: err rate likmit up_the_irons: you may want to contact their NOC (noc@us.ntt.net) mercutio: i know it's psuedo science
up_the_irons: it was too diff destination i saw ssh packet loss
err ssh lag
i can't ssh to comcast
and it was very on/off up_the_irons: starbucks kicking me out (closing), bbiab mercutio: nlayer rate limit a lot of icmp btw
it's annoying :/ acf_: fwiw, NTT noc says "
We are not seeing any packet loss within our network. To investigate this issue further please contact your upstream provider." mercutio: haha
shit
i don't think it's necessarily ntt's fault in this instance acf_: my guess is overloaded interconnects mercutio: we determined it could be tata
it's relaly hard to know acf_: how about NTT <-> Verizon mercutio: cos of what up_the_irons said about icmp rate limiting
got a test destination? acf_: yeah mercutio: ok tell me i'll test it
but i'll make another coffee first :) acf_: 72.55.8.69 for comcast
108.40.173.223 for verizon (beware super slow dsl)
but no packet loss to google mercutio: i repeated the comcast issue acf_: I agree it's probably not in the NTT backbone mercutio: ok verizon normal path is via alter.net
(verizon)
which a little jitter
is AS2914 ntt
yip cool
verizon looks fine too
via ntt
which means it's prob reverse path if chornically bad
oh no some packet loss acf_: it seems like the loss is only at the lax verizon <-> NTT exchange btw mercutio: it's not nearly as bad as that comcast one was acf_: I get http://paste.unixcube.org/k/e57747 mercutio: the jitter is worse via ntt too
this is via lax
i screwed my up my other route i don't think i can send via sj
oh i can send via sj
interesting
via san jose only seeing packet loss on last hop and in betwen hops
err and some early ones
which always depriortise
but via los angeles seing it on last 3 hops
err 4 hops
but i changed source ip gah acf_: yep. that's what I've been seeing mercutio: last 4 hops? acf_: yeah mercutio: why am i only seeing last hop
bah i'll fowrad it through oso i can keep to same source address
the jitter is def worse via ntt
ok last two hops now
and it's either wose right now
or sj is worse than la for forward path with reverse path combined too
i'm seeing 17% packet loss acf_: http://paste.unixcube.org/k/78f880
the bottom one basically describes what I've been seeing for the last few days mercutio: yeah that's what i was seeing orig
hangon i'll pastebin
actually what's your paste thing :) acf_: http://paste.unixcube.org/
runs from arp mercutio: heh
cool
http://paste.unixcube.org/k/e57f7a
it went down a bit
so sampling bias :)
it seems like it gets better and worse acf_: yeah, that's been happening a lot for me mercutio: so ok, it doesn't matter if it's sj or la
and it's better over alter.net
i'll shift it back to alter.net
and wait a minute or two
oh only being last two hops i think is cos shorter route?
i having problems cutting grr
i hate linux cutg and pate
paste
http://paste.unixcube.org/k/7dfe16
weird going from top to bottom fixed it acf_: that one looks nice mercutio: oh i got a packet dropped
that's like what was happeing with comcast too :/ acf_: maybe my crappy dsl
http://paste.unixcube.org/k/8ad869 mercutio: well the first one had depriorisation on some hops acf_: packet loss stops when return path changes (NTT <-> Verizon in Dallas instead of LAX) mercutio: but still packet loss to destination is WAY down
and the source ip remained the same
so basically it doesn't matter if it goes via sj or la from ntt
they both suck
but alter.net is fine
well when i talk about fowrard path you're talking about return path right
forward path for me is my traffic going to your dsl
and return path for you is traffic coming to you acf_: yep, I'm switching them mercutio: ok so dallas is better acf_: yeah mercutio: los angeles and san jose may share some hidden path
like i don't know what city you in
but i'm sure it's not in california acf_: lsanca == San Luis Obispo, CA mercutio: so there's probably an in between point hidden
you are in california? acf_: everything from here goes though lax
yep
verizon dsl sucks mercutio: dude your internet sucks
i have better pings to new york :/ acf_: sometimes it's not as bad lol
*lsanca snloca mercutio: yeah i'm in new zealand which you may have been able to tell from those traces ;)
and that's from dsl
why woudl it e going via dallas acf_: ntt.net is in tx mercutio: oh hmm
ntt.net is japanese acf_: us.ntt.net goes the same place afaik mercutio: hmm i found somewhere with nlayer to trace from
it's still jittery but not lossy
it's nlayer -> alter.net -> verizon acf_: the arp path does that once in a while mercutio: ahh ok
yeh it looks fine acf_: yesterday I think it did that and the packet loss stopped mercutio: i don't think up_the_irons wants to get involved in screwing with routes lots acf_: yeah that sounds messy mercutio: heh i did it a bit before
but i just did sweeping generalisations
like "prepend anything that says cogent"
so that if a non cogent route is available it tcan take that
if it's not much longer
and some shift for at&t traffic
if it's just verizon and comcast which both have heaps of users maybe he'd make adjustments?
1% packet loss via nlayer now heh acf_: maybe. it seems like he has ntt prioritized heavily? mercutio: it hink your dsl does drop packets a bit too acf_: likely mercutio: it's default route
cos the old router has 100k table limit
comcast has 180 routes
i dunno how many verizon have acf_: plenty I'm sure mercutio: but i reckon we should try and get them to be part of it :)
not necessarily
bigger providers often have less prefixes
as they just get huge chunks
i can check
yeh
verizon have heaps
198.181.111.0/24, 198.181.112.0/24
etc
nearly 1500
oh that can't be aggregated
cos that would be 110 and 111
198.105.10.0/24 and 198.105.11.0/24
that can be aggregated though
and 198.105.8.0/24 and 198.105.9.0/24 acf_: "AS701 announces bogons"
why would they do that? mercutio: do they acf_: http://bgp.he.net/AS701#_asinfo mercutio: 192.203.48.0/20 looks dodgy
nah it looks fine actually
what bogons acf_: http://bgp.he.net/AS701#_bogons
oh wow 192.168.0.0/16
oh 198.168.0.0/16 up_the_irons: wow mercutio: 192 is mostly legacy space
up_the_irons: we talked a lot while you were away up_the_irons: i saw the scrollback mercutio: i dunno how bogus these ips really are
it's more just that no-one has notes on what they are up_the_irons: traffic issues on other networks is a PITA. You fix one thing, another thing breaks. It never ends. mercutio: up_the_irons: yes
which is why it only makes sense if it's severe acf_: I would think NTT and Verizon would have decent networks and be fairly responsive to issues of this severity
oh well mercutio: well ntt have been having issues only recently right?
i reckon comcast is more worth fixing than verizon
they have way less routes too
ho hmm
ok got a comcast dsl ip
weird i'm not seeing issues, only jhitter at last hop acf_: things are looking better
on comcast
http://kremvax.acfsys.net/smokeping.cgi?target=Remote.comcastnet mercutio: oh
wow so on/off
is comcast ip static normally? acf_: I don't think so mercutio: will get him to setup smokepnig maybe :) m0unds_: comcast tends to assign "sticky" IPs - so long as your cpe doesnt change and isnt powered off for longer than it takes to release the dhcp lease, youll likely have the same addr for a long time. mercutio: yeh i'll get him to curl some file randomly m0unds_: when i had residential svc, my dhcp addr stuck with me for over a year mercutio: he's running into some depency issues
oh interesting.
usually when people say sticky ip it's sticky until you want to access it m0unds_: it only changed when they brought new cmts gear up
at least in my case mercutio: oh
maybe they'll do that to him
is 4 msec jitter normal? m0unds_: yea mercutio: it's 0.1 jitter two hops prior
(and worse the hop immediately before
oh i thought that may mean he was legacy node m0unds_: i dont tend to worry til it hits +/- 15-20ms continually mercutio: well both traces have go ne over 500 seconds / 500 pings with no loss m0unds_: hm mercutio: i worry about 2 or 3 msec jitter normally
but maye i shouldn't
for idle connectinos
it tends to suggest there's some congestion somewehre
but it depends on bandwidth etc
like 2 to 3 msec jitter on gigabit means it's pretty packed m0unds_: also, not sure if this is still the case, but when looking at comcast rdns for intermediate gear, he = 100gbit, te = 10gbit
since they use lotsa j gear
lots of fun hw to play with mercutio: this is te acf_: thinking about it more, latency between verizon and ntt in lax has always doubled from ~30 to ~60 for me
ie, at last verizon hop is 30ms and first ntt hop is 60ms