#arpnetworks 2014-02-14,Fri

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***phlux has quit IRC (Ping timeout: 264 seconds) [03:21]
phlux has joined #arpnetworks [03:28]
........................................... (idle for 3h31mn)
mhoran1 is now known as mhoran
avj has joined #arpnetworks
[06:59]
....... (idle for 31mn)
m0undsoooook
oh, didn't realize the routing to arp via comcast went back to normal on the 12th
[07:34]
................... (idle for 1h30mn)
brycecYes, about the same PPS as when I had 200+ targets with 500B pings. (86pps or so)
^ 21:35:19 < mercutio> iyou're probably still doing a lot of pps
I don't recall when I added packetsize=500 but it wasn't yesterday. Might have been two days ago when I added a bunch more hosts, might have been earlier. 21:35:26 < mercutio> were you at defaults prior?
Today (now yesterday) went from 500B to 32B pings 21:36:03 < mercutio> whther you'd shifted to 500 byte packets today
And yes, I'll have to figure out the DNS woes. Someday.
But for now, we're settling for using better DNS servers. Hopefully we'll see improvement.
[09:05]
.................. (idle for 1h29mn)
m0undshaha
i just use IPs for smokeping stuff and an external monitor that uses dns to resolve the hostname
[10:37]
brycecI would too. However, some of the hosts I monitor are known to change their IPs
mercutio: One possible solution for pseudo-caching would be a script that [daily, hourly, whatever] did the DNS lookups and dumped the results in /etc/hosts
[10:39]
m0undsah, gotcha [10:40]
brycecAt the very least, it wouldn't hold up FPing
m0unds: Things like Tunnelr nodes, HE endpoints, my gateways and DNS servers are all easily static. But mirror sites and nlnog ring nodes may change
Or at least, there's no amount of guarantee they won't change.
:( better DNS servers did not fix Chunkhost's gaps
[10:40]
Oh cool! A useful log message! FPing: WARNING: smokeping took 85 seconds to complete 1 round of polling. It should complete polling in 60 seconds. You may have unresponsive devices in your setup.
It doesn't tell me WHAT or anything :/
But it's a start
I think I'll stop monitoring mirrors.
They were just there to provide more geographical data, which nlnogring is doing too
[10:49]
m0undsnlnogring?
oh, that
[10:54]
brycecheh [10:55]
m0undshttps://atlas.ripe.net/ [10:57]
brycec^ eh, up_the_irons? [10:58]
m0undsthat project seems less skeevy to me than a shared shell
but i'm paranoid
haha
not to say that anyone /would/ abuse it, but meh
[10:58]
brycecI like the transparency of NLNOG. I don't know *what* I'm hooking up to my network with RIPE
And if the RIPE device/network is compromised in some fashion??
At lest with the shell, I have eyes on it, I can monitor it etc
And I suspect it's easier to kill a VM than to shutoff a switch port in some remote DC
[10:59]
m0undswell [11:00]
brycecSpeaking as someone that admins a shared shell, with the right configuration, you've nothing to worry about. [11:01]
m0undsconf t >> int gi1/0/1 >> shut
assuming you have access to the switch in question
which, if you're a network op, i'd assume you do :)
[11:01]
brycecOf course you also have to connect to the switch :p [11:01]
m0undsthat's a given [11:01]
brycecOr if you're up_the_irons, I'm sure he has some management portal up and ready to click the right name.
(Switch: you also have to figure out the right port)
(and login)
But I digress. Hooking up a little black box to my network makes me nervous.
Also nlnog++ for just requiring a VM, and not requiring you to rack hardware
[11:01]
m0undsrequest remote hands to cut the cable, CUT IT
DO IT NAO
[11:03]
the old gen ripe dongle is 2x3x1"
http://probev2.ripe.net/
[11:10]
bryceclolol
"If you find this device and are unaware of its purpose"
[11:11]
m0undshttp://probev3.ripe.net/ [11:12]
brycecshit if I came across that in a closet, I'd be *pissed* (at least until I knew what it was [11:12]
m0undsyeah - NSA probably loves those little embedded linux server dinguses
http://www.lantronix.com/device-networking/embedded-device-servers/xport-pro.html
[11:12]
brycecThat is insane... The same fping takes 23s on ARP, takes 59s on Chunkhost.
(oh 61s that time)
[11:18]
hazardousi had a chunkhost box once
it was some kind of slow, horrid abomination
[11:22]
bryceclol
In the very beginning, they were cool
[11:22]
hazardousi think
they did some free server giveaway
ended up getting shit on because they didn't actually give any out apart from one or two
then ended up getting shit on more whe nthey spammed the poeple they didn't gvie anything out to
[11:23]
m0undschunk host...vomit host? sweet. [11:24]
brycecI'm unaware of that... They did offer (do still?) a "free" tier though, a way to pack host machines fuller, the instance was culled if detected idle for too long, and I think it was v6 only. [11:24]
hazardousthey paid for this ad http://www.reddit.com/comments/1lcxo4/get_a_superfast_cloud_server_with_ssd_hard_drives/ [11:25]
brycecAnd I do know people who took advantage of that (briefly) [11:25]
hazardousit was crapped on immediately
because it was a giant, blatant lie
[11:25]
brycecheh [11:25]
hazardousthe title was SSD hard drives and 8GB of RAM for just $9/month. WHAT?! (chunkhost.com)
it was $9/monht and something like $2500 setup fee
[11:25]
brycecYep, you "purchase" the hardware
or some crap like that
[11:26]
hazardousexcept you dont' actually own any of it
and it's still shared
so that's a huge douchemove
[11:26]
brycecYep.
Unique idea, but only becausepeople don't usually share the bad ideas.
[11:26]
hazardouswhat i really want to know is [11:26]
brycec(Okay, it's not DNS that's slowing fping down) [11:27]
hazardouswhat the hell was chunkhost thinking when they decided to put a blatant lie for an ad in r/programming and r/sysadmin
like, maybe minecraft or something with less competent people would work
but come on
[11:27]
m0undshttps://scontent-a.xx.fbcdn.net/hphotos-ash3/t1/1656358_278087589008207_569417404_n.jpg one of hte positive aspects of working @ a tribal enterprise [11:28]
hazardouswhat is that [11:28]
brycecindian tac!
*taco
[11:28]
hazardousi see beans and .. not sure if that's pita [11:28]
toddfwell, in the long term, is $2500 setup fee and $9/mo cheaper than $100/mo or so for equivalent elsewhere? [11:28]
brycecFried tortilla [11:28]
hazardousvegeterian taco? [11:28]
m0undsfrybread
yeah
[11:28]
hazardoustoddf: well it's an 8 gb vps, so yes [11:28]
toddfsounds like someone is playing with the pricing models to see who will bite at what pricepoints [11:28]
hazardousand it's also oversold to death and complete crap
what is frybread
[11:28]
brycecm0unds: looks like you got valentine's fruitsnacks too? [11:29]
toddfoversold, well, thats kindof par for the course with chunkhost right? [11:29]
hazardousi wiki'd it but i'm not sure what its equivalent is [11:29]
m0undsyeah, hahaha
http://en.wikipedia.org/wiki/Frybread
[11:29]
BryceBotFrybread :: Frybread (also spelled fry bread) is a flat dough fried or deep-fried in oil, shortening, or lard. The dough is generally leavened by yeast or baking powder.[citation needed] Frybread can be eaten alone or with various toppings such as honey, jam, or hot beef. Frybread can also be made into tacos, like Indian tacos. It is a simple complement to meals. History According to Navajo tradition, frybread was created in... [11:29]
hazardousis it like those things at county fairs that have tons of powdered sugar on them
or is that called something else
[11:29]
brycecm0unds: I work in a 1-man office. Now I feel very alone :( [11:29]
hazardous* nb: i've never travelled outside of ultra-urban areas anywhere in the US [11:29]
brycechazardous: Ever been to a carnival/fair? [11:29]
hazardousbrycec: too bad the valentines fruit snacks didn't have TO: and FROM: written in :< [11:30]
m0undsbummer. i rarely eat anything people bring, except today because i had no idea frybread was egg-free (i'm allergic to eggs) [11:30]
hazardousbrycec: nope, never [11:30]
***jpalmer has quit IRC (Quit: leaving) [11:30]
brycechazardous: so basically I have to put this in terms of taco bell, mcdonalds, and wendy's? [11:30]
m0undshahah [11:30]
hazardouspretty much, yeah [11:30]
***jpalmer has joined #arpnetworks [11:30]
hazardousi've seen a parade or two but those msotly consist of naked gay people and the sisters of perpetual indulgence here [11:30]
brycecm0unds: Okay, easy. IT"S TASTY.
m0unds: awesome, very lucky. I bet you get lunch for free too :(
[11:30]
hazardouslike i'm trying to picture fried bread, but the only thing i can think of is french toast [11:31]
m0undsi've had frybread before, but before i realized i was allergic to eggs (i grew up in CO/NM) [11:31]
hazardousm0unds: wow, that must .. suck.. [11:31]
m0undsthere was red chile too, but it had meat in it [11:31]
hazardousdoesn't that exclude almost everything made commercially [11:31]
m0undsyeah [11:31]
hazardousi remember seeing a lot of WARNING: MADE IN A FACILITY THAT ALSO PROCESSESS x,y,z,a
and it's something like nuts, wheat, eggs, gluten?
[11:32]
m0undsyeah, eggs are a common ingredient in stuff. it just means i eat less shit than most people, since crappy food tends to have a lot of egg in it
crappy as in lacking in any nutritional value
so we tend to buy vegan stuff mostly, as it doesn't require diligent label checking
[11:32]
hazardousi wish i could eat cilantro, but it tastes like soap and it sucks because i like mexican food and they never remember to leave it off when i ask
it's not an allergy or anything but it just tastes vomit inducing and revolting
maybe something is wrong with my tongue
[11:33]
m0undsyeah, cilantro is great
hahaha
DEFECTIVE
[11:33]
jpalmerhazardous: a lot of people think cilantro tastes like soap.. [11:33]
m0undsthey're WRONG WRONG WRONG [11:34]
jpalmerm0unds: no, it's actually in how they process the flavor [11:34]
m0undstheir processing of the flavor is wrong [11:34]
bryceccilantro is over-used as a crutch by crappy and wannabe "mexican" places
m0unds++
[11:34]
staticsafeit is a genetic thing no? [11:35]
m0undsit could be
http://www.huffingtonpost.com/2012/09/20/cilantro-aversion-gene-study_n_1901124.html
[11:35]
jpalmerhttp://www.nytimes.com/2010/04/14/dining/14curious.html?_r=0
staticsafe: there is no formal proof that it is genetic, but that is what they are studying. they suspect it might be.
[11:36]
staticsafestaticsafe nods [11:36]
jpalmerI often hear "soap" along with chemicals, hand lotion, etc.
(ie, it smells or tastes like chemicals)
[11:37]
m0undsA Japanese study ... suggested that crushing the leaves will give leaf enzymes the chance to gradually convert the aldehydes into other substances with no aroma.
cilantro pesto as a way to enjoy cilantro
oh man. http://www.huffingtonpost.com/2011/10/27/roasted-vegetable-tofu-a_n_1057197.html
[11:38]
.... (idle for 17mn)
hazardousm0unds: you're veegan? [11:57]
.... (idle for 18mn)
m0undsfood wise, yea
lifestyle stuff makes my head hurt
[12:15]
........ (idle for 37mn)
mercutiobrycec: you can judge some dns servers to set minimum ttl [12:52]
s/servers/recursors/ [13:05]
BryceBot<mercutio> brycec: you can judge some dns recursors to set minimum ttl [13:05]
mercutios/judge/set/ [13:05]
BryceBot<mercutio> brycec: you can set some dns servers to set minimum ttl [13:05]
mercutiooh i need both heh
i shouldn't be on irc before coffee
wow ubuntu is going to change to systemd apparently
it's not april 1st hmm
[13:05]
bryceclol
mercutio: But I've determined that it's not the DNS lookup choking that one host, so yay that.
Tested with time fping... a whole bunch of IP's, still took much longer on Chunkhost for no discernible reason (same results)
[13:20]
mercutioweird
you want to pastebin me the command?
i could see if i can figure out what causing it
i found a ntt looking glass
[13:25]
brycecI could, sure, but I don't think you'll see anything. I run smokeping on 5 hosts, and Chunkhost is the only one being poopy. </technicalterm> [13:28]
mercutiobut it seems to be going amazingly slow [13:28]
m0undshttp://www.us.ntt.net/support/looking-glass/ ? [13:28]
mercutioyeh [13:28]
m0unds@ddg ntt looking glass [13:28]
BryceBotm0unds: I'm sorry, the DDG Zero-Click API returned no results. As this is a Zero-click Info API, most deep queries (non topic names) will be blank. Quite frankly, I suck cocks. [13:28]
mercutioSorry, the Looking Glass is currently unavailable. Please try again later. [13:29]
m0undsoh, that' sright [13:29]
mercutioi thought it may show up in more detail the ntt issues
chunkhost along is making it take more than 60 seconds?
alone
as a destination?
[13:29]
brycecmercutio: source
running http://sprunge.us/eAhL takes forever
^ on a chunkhost VM
forever in the linked case is 61s
[13:31]
mercutiooh that was from a chunkhost vm not to chunkhost [13:32]
brycec(And yes that's one with hostnames, it was quick/easy for me to post) [13:32]
mercutiowhat do you get on arp? from NZ it took 37 seconds [13:33]
brycecreal 0m23.240s [13:34]
mercutionice
i should try agian
[13:34]
brycecI try it every 60 seconds :P [13:34]
mercutioyeh
so more are cached i meant
[13:34]
brycecheh, in my debugging earlier, I tcpdumped the DNS lookups, didn't take more than 3s
^ On Chunk
[13:35]
mercutiooh curious
22.7 seconds second time in nz
[13:35]
brycecI then tcpdumped icmp... that was a mistake :P [13:35]
mercutioso dns definitely makes a diff when a whole lot of uncached stuff
haha
i use strace sometimes
[13:35]
brycecYeah, but only 50% in your case.
I did that too, didn't show anything obvious
[13:36]
mercutioyou're doing 20 pings [13:36]
brycecBut strace doesn't have timing [13:36]
mercutiothat means it'll take 20 seconds for the pings at least
way less diff between cached/uncached on arp
and 22.7 sconds again heh
[13:36]
brycecnice [13:38]
mercutioso chunkhost is blowing chunks?
what dns server are you using on chunkhost?
and what location is it?
[13:38]
brycecmercutio: I was using HE's DNS earlier, switched to using Google's DNS. No difference.
mercutio: LAX1
(It's vps1.cobryce.com)
[13:38]
mercutioahh
8.8.8.8 isn't close in la :/
maybe try 4.2.2.2?
i think he is close but not very good
[13:39]
brycecAgain: DNS lag has been taken out of the equation completely. Tcpdump says DNS only took ~3s. Trying without DNS lookups at all still takes 50+ seconds. [13:39]
mercutioi did namebench ages ago [13:39]
brycecmercutio: ^^
(Oh my bad, I'm using Chunkhost's own DNS right now)
which ARE close :P
[13:40]
mercutiosbc global is the other one that was good
heh
maybe they throttle you
it's not openvz is it?
[13:40]
brycecMust be something like that.
mercutio: Xen
[13:41]
mercutiothat nz host was xen
xen should be fine normally
[13:41]
brycecbrycec is going to throw a ping -f at things and see how it does. [13:41]
mercutiouhh
add -c 10
or something
[13:41]
brycecmercutio: yes
but later
because I have important shit to get done when I'm on the clock ;p
[13:41]
mercutiocos ping -f could be considered abusive behaviour
ok
[13:42]
brycecNetwork throttling is the only thing that makes sense at this point, but I don't have a good way to test for that. [13:42]
***kevr has quit IRC (Ping timeout: 252 seconds) [13:43]
mercutiosmokeping? :)
umm usually http download and look at packet dumps
cos http has back off behaviour
it is recent version riht?
[13:43]
brycecrecent version of? [13:44]
mercutiofping
there were soem updates at some point
just reading changelog now
[13:45]
brycecAnd I was thinking they're only throttling icmp [13:45]
mercutioi doubt it [13:45]
brycecgood question, it's Debian Squeeze so no [13:45]
mercutiofping sohuld timeout
is it 3.2+ ?
* Performance optimization for big select timeouts (#10, Andrey Bondarenko)
i wonder if that make sa diff
[13:45]
brycecfping: Version 2.4b2_to $Date: 2002/01/16 00:33:42 $
hmm
[13:46]
mercutiois arp using same version? [13:46]
brycecarp is 3.2
as are my other nodes
[13:46]
mercutiohmm... [13:46]
bryceci'll tackle that later [13:46]
mercutioyou can probably just install the later debian package [13:47]
brycecbut good call [13:47]
mercutioif it breaks it only breaks fping :) [13:47]
brycecnewer fping not in backports, so there may be a reason [13:47]
mercutiothat someone didn't get around to it
you can often just wget the package and dpkg -i it
[13:48]
***kevr has joined #arpnetworks [13:48]
mercutiofor single programs with no dependencies [13:49]
brycecbrycec thinks mercutio is way more invested in this than he
real 0m34.709s
quite an improvement
[13:49]
mercutioheh
i like clean graphs :)
ok so fping bug :)
is mirrors.arpnetworks.com down?
weird it works from another location
[13:56]
brycecha! mercutio was slightly wrong, it did break fping slightly :P ERROR: fping ('/usr/bin/fping6 -C 1 ::1') could not be run: /usr/bin/fping6: /lib/libc.so.6: version LIBC_2.15' not found (required by /usr/bin/fping6) [14:01]
mercutiowhat version was that
glibc can be the biggest bitch
apt-get source fping may be necessary :(
if that's 3.2
[14:01]
brycecfuck, nvm... In spite of dpkg giving an error when I tried installing fping 3.8, it went ahead and installed it. [14:02]
mercutioit's whether smokeping minds [14:02]
brycec3.2 back installed and working
brycec shakes a fist at dpkg
[14:02]
***kevr has quit IRC (Read error: Operation timed out) [14:03]
brycecFun fact: http://sprunge.us/LZAT doesn't mean that it didn't install the package anyways. [14:04]
mercutiooh
i thought you were going to 3.2
3.2 should be fine
[14:04]
brycecmercutio: I did, then I tried 3.8 for shits and giggles. [14:04]
mercutioahh ok [14:05]
brycecIt shat more than he giggled [14:05]
mercutioit works with ubuntu from debian packages too sometimes
https://github.com/schweikert/fping/blob/master/ChangeLog
[14:05]
brycecI get the libc thing... I just figured "ERROR installing" meant "I didn't actually install it, you still have 3.2" [14:05]
mercutiothings are less interesting from 3.2 to 3.8
yip
but when you do other updates it can be a bitch
[14:05]
brycecIt's squeeze, there will never be another update to fping on it :P [14:06]
mercutiono ot other packages
what does apt-get -f install say atm?
[14:06]
brycec0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. [14:06]
mercutiooh cool
it not complaining
ok
the real question is did smokeping fix itself
[14:06]
brycecOnce 3.2 is on, yes
3 minutes' data and counting
[14:07]
mercutiocool [14:07]
***kevr has joined #arpnetworks [14:08]
......... (idle for 44mn)
up_the_ironsm0unds: brycec i like the NLNOG; since you get an actual shell, you have access to more tools; way better than a looking glass [14:52]
brycecup_the_irons: what about ARP hosting a RIPE probe?
(I'm skeptical, but worth asking)
s/worth asking/curious)
[14:58]
BryceBot<brycec> (I'm skeptical, but curious)) [14:58]
up_the_ironsbrycec: a possibility [14:59]
m0undshttps://atlas.ripe.net/targets/ringnodes/map/ [15:09]
brycecwhoa
there's already one at ARP!
[15:10]
m0undsthat's from nlnog [15:10]
brycecohduh [15:10]
m0undsit's the probe target map [15:10]
brycecso RIPE hits NLNOG? [15:10]
m0undsyeah, you can select those as user-configured probe destinations
or targets
[15:10]
brycecbrycec wonders why RIPE doesn't just link to NLNOG's site? [15:11]
m0undsthey do [15:11]
brycecit's kinda buried [15:11]
m0undsit's right at the top [15:11]
brycecLike RIPE has their own list of nodes
brycec should stop browsing with zoom=50%
[15:11]
m0undslol
it's the third line of text above the map "visit nlnog project" or something
hahaha
[15:12]
brycecYeah, but it's only ONE [15:12]
m0undsthe whole thing should be links [15:12]
brycecWhy did RIPE make their own list and own map? [15:12]
m0undswhy should someone click off the site to look at the map when it's right there? [15:12]
brycecheh okay so you understand my point :p [15:12]
m0undsBAD UX BRAH [15:12]
brycecwhy should someone take the time to scrape(?) another site for information when you could just send the user to the source?
BAD BRAH
[15:13]
staticsafeheh [15:13]
m0undsdoubt there's scraping going on
maybe some shovelling
[15:13]
brycecor they're paying some bloke to update it :( [15:14]
m0undsthe coordinator for nlnog is a commitee member at ripe
and he works for atrato
[15:22]
bryceccroneyism! [15:25]
m0undsMUST BE
also, i'm guessing he's trying to get exposure for nlnog
but mostly cronyism
[15:25]
up_the_ironswhoa, we're already in the Atlas target list [15:25]
m0undsyeah, nlnog ring target [15:26]
up_the_ironscool [15:26]
.... (idle for 17mn)
mercutiobrycec: there's a way to get output of root-servers locations
with dns
dig nodes.l.root-servers.org txt +tcp |
[15:43]
staticsafebrycec: just put in your Targets file with some modifications, smokeping takes a while to start up heh [15:45]
mercutios/|$//
s/|$//
gah
but how's that for a list bryce
[15:45]
m0undssmooooooooooooke [15:47]
mercutioit's ok to ping them i seem to recall too
i can't find confirmation of that though
it's kind of crazy how well the root serrvers are overprovisioned these days
i think it's pretty unlikely for the root servers to not be reachable, while the internet works now.
[15:47]
staticsafeFri Feb 14 15:51:44 2014 - FPing: probing 158 targets with step 300 s and offset 249 s.
Fri Feb 14 15:51:44 2014 - FPing6: probing 47 targets with step 300 s and offset 181 s.
hm
[15:53]
m0undszomg [15:53]
brycecwelcome to the smokeping age, staticsafe [15:54]
mercutioi wonder how much icmp traffic is on arp from people running smokeping :) [15:54]
brycecmercutio: nice idea [15:54]
m0undshahaha [15:55]
mercutiobrycec: i was surprised that was out there.
it's all isc infrastructure afaik
[15:55]
bryceclol well I'm currently responsible for about 150kb/s of ICMP traffic :P [15:55]
m0undsTRIPLE IT [15:56]
staticsafehm the smokeping webapp is pretty damn slow now [15:56]
brycecm0unds: You may recall I did more than triple it... [15:56]
staticsafewhat is the secret to your speed brycec? [15:56]
brycecstaticsafe: cgi? fastcgi? other?
staticsafe: fastcgi
[15:56]
staticsafeim passing it through fcgiwrap
nginx
[15:56]
brycecI'm not familiar with fcgiwrap
I'm running it from spawn-fcgi
I was responsible for at least 1mbps of ICMP traffic :P 15:54:42 < m0unds> TRIPLE IT
s'kb/'kbp
[15:57]
BryceBot<brycec> lol well I'm currently responsible for about 150kbps of ICMP traffic :P [15:58]
staticsafes/kbps/gbps [15:58]
m0undsTRIPLE IT AGAIN [15:58]
brycecI'd rather not, thanks. I came awful close to 20% of my monthly bandwidth at that rate
And by awful close, I do mean still far from
[15:59]
staticsafestaticsafe gives spawn-fcgi a shot [15:59]
m0undsbrycec: boooooooo [16:00]
brycecstaticsafe: here's mine http://sprunge.us/POVj
But I was on track to hitting 200GB/mo which is quite a bit more than I normally do
[16:01]
m0undsjeez [16:01]
mercutioi got my burst usage up quite high once, which disconcerted me, as i wnated to stay under 200gb/month. but it didn't last. [16:03]
brycecup_the_irons: "feature request" https://graphs.arpnetworks.com/ ought to redirect to https://graphs.arpnetworks.com/cacti/graph_view.php instead of giving a Forbidden error. Unless that's intentional? [16:03]
m0undsi've used...140GB since the middle of nov [16:03]
brycec254.9GB in the past 12mo
s/9/69
[16:03]
BryceBot<brycec> 254.69GB in the past 12mo [16:03]
brycec64.67GB according to vnstat. Hey what's the deal with this, is up_the_irons trying to rob me of my bandwidth? :p [16:05]
m0unds... https://twitter.com/ScanBC/status/434469138325831680 [16:08]
BryceBotTWITTER: #Coquitlam LaFarge park RCMP investigating a report of someone attempting to steal a duck (Fri Feb 14 23:28:09 +0000 2014, retweeted 123 times) [16:08]
brycecDAMNIT THAT'S MY DUCK [16:08]
m0undsUNHAND MR. MALLARD [16:10]
brycecSIR QUACKSALOT WILL NEVER BE TAKEN ALIVE [16:10]
m0undshttp://brewvival.com/beer/whiskey-chub?utm_source=twitterfeed&utm_medium=twitter
want.
[16:11]
staticsafewhee spawn-fcgi works much better brycec [16:14]
up_the_ironsbrycec: well i don't think there's anything at graphs.arp*; everyone's Cacti graphs link in the Portal shows a complete URL [16:14]
bryceccongrats staticsafe
up_the_irons: Maybe I'm not following a link from the portal ;)
Maybe I'm typing in the URL and expect to be greeted with the login
[16:14]
up_the_ironsbrycec: then put '/cacti' on it [16:15]
brycecmaybe I will! [16:15]
up_the_ironsbrycec: there are other things on graphs that are not meant for customers :) [16:15]
brycecheh, I thought that might be the case [16:16]
up_the_ironsbut since i was JUST in that apache config updating the SSL, maybe... [16:18]
bryceclol [16:18]
up_the_ironsup_the_irons scratches his head thinking about an Apache redirect
IT'S BEEN TOO LONG
[16:18]
brycecredirect ^/$ /cacti/
I think
(because it's been so long since I've had to much with Apache. NGINX FOREVER. Also lighttpd is okay.)
[16:18]
m0undsnginx por vida [16:19]
brycecstaticsafe: The reason fcgiwrap was slow is because it acts as fastcgi to nginx, but it's still executing the cgi for each and every request. [16:19]
m0undsbuddy of mine works for the utah MLS (real estate MLS) and lighttpd really shines at delivering static content on their web frontend
it was nearly 15% faster than nginx w/the same content
[16:19]
brycecstaticsafe: whereas fastcgi loads and runs once and leaves it running for multiple queries
faster than nginx? Wow
[16:19]
m0undsyeah, that's the only load where that was the case too [16:20]
up_the_ironsbrycec: i think that would also rewrite other things [16:20]
m0undshe uses nginx on everything else [16:20]
brycecup_the_irons: the pattern would match requests to / only
up_the_irons: would not match /pr0nstash for instance
[16:20]
up_the_ironsoh [16:20]
brycecm0unds: Is it "dynamic" static content? ie. something that could pass back X-SendFile? I've found that to be wicked fast in nginx. [16:21]
up_the_ironsup_the_irons hopes he won't lose his /pr0nstash [16:21]
brycec*yoink*
E_CONNREFUSED looks like graphs is down and up_the_irons has lost is /pr0nstash
*his
[16:21]
m0undsbrycec: i have no idea [16:23]
brycecWell, good for lighttpd anyways.
When I abandoned Apache, I went to lighttpd. It was a learning curve... But I liked it and its configurability over Apache immensely. When I upgraded to Wheezy, I moved to nginx and I'm instantly in love. I can't explain it, but I love nginx.
[16:23]
up_the_ironsbrycec: there, it works [16:25]
brycecup_the_irons: yep the redirect works. And now I have a Cacti "access denied" page [16:25]
m0undsi like contextual configs a lot [16:25]
up_the_ironsthat's all the Apache work I will do for a week [16:25]
brycecAnd clicking login gives me access denied again
stupid Cacti
[16:26]
m0undsnginx configs remind me of junos, so i like them
hahaha
[16:26]
bryceclol [16:26]
up_the_ironsbrycec: hmm.. i can login [16:26]
brycecup_the_irons: going directly to my graphs.php URL works fine
up_the_irons: So I'm gussing it's user permissions
[16:26]
m0undsthe only time i used lighttpd was just as an exercise to see how it worked, and that was just to handle static images for a site [16:26]
brycec"bryce is not allowed to see the Cacti index"
lighttpd and nginx configs are similar, which is good
[16:26]
m0undsi returned to my graph url and i'm seeing: GRAPH DOES NOT EXIST [16:27]
brycec(by no means interchangeable... but similar, esp. comared to Apache)
lol m0unds srsly?
[16:27]
m0undsyeah, just reloaded it
haha
ok, logged out and in and it's ok
m0unds shrugs
haha
[16:27]
up_the_ironsi dunno if you're able to go to graphs.php directly
/graph_view.php
is the one that works
and is what you get redirected to after login
[16:28]
m0undsargh, lastpass broke form filling again
dammit
[16:28]
up_the_ironsi'll leave it this way for a bit and if i get complaints, i'll just put it back
to the way it was
[16:28]
brycecYeah must be cacti perms. If I logout, I'm greeted with the ARP login page for Cacti at "/cacti/", if I'm already logged in and visit "/cacti/" Cacti gives me access denied and links me to login. [16:29]
up_the_ironsand brycec will simply have to type "/cacti" ;) [16:29]
brycecbut that's so much work! [16:29]
mercutioi've had cacti permission issues on multiple different cacti instances. i think cacti has some kind of cookie or such bug. [16:29]
brycec(mmmm cookies) [16:29]
mercutiosometimes if i login to cacti, and click on a graph, it will bring up the login again.
and then it'll work
[16:29]
up_the_ironsbrycec: i can't imagine it would be any different from before if you hit '/cacti' directly. i mean, the redirect just redirects to that way...
*anyway
[16:30]
m0undsit worked fine for me - logged out of the portal, then logged in and went through the links to get to graphs that way
logged in to cacti, saw my vlan and my graphs
[16:30]
brycecup_the_irons: I'm sure it's exactly the same as if I hit /cacti directly. [16:30]
up_the_ironsk [16:31]
brycecheh /home/garry/www/cacti/rra/
now we know where the RRDs are kept
[16:32]
Can anyone else from an ARP IP ping hitchcock.freenode.net ?
According to https://smokeping.cobryce.com/?target=Internet.FreeNode.hitchcock4 it has dropped off the map for two hosts, both hosts traceroute through NTT. (And another working host traceroutes through peer1+init7)
[16:42]
m0undsnada here [16:47]
mercutiocan't get to dest
i can't get to it from home either
[16:47]
m0undsit's in bulgaria? [16:47]
mercutioit may be the destination
can't telnet on port 6667 too
i'd say the host is just offline maybe due to ddos or hardware failure
[16:47]
brycecmaybe, but I can reach it from a TWC and Comcast connection :p [16:48]
mercutiowell i'll still blame them
actually maybe it ntt
quite a bit of stuff to me returns via ntt
[16:48]
m0undsyeah, it's fine via comcast for me
via level3
[16:49]
mercutiowell my forward route from home is via cogent :/ [16:49]
brycecMorelikely NTT, http://sprunge.us/aWhZ [16:49]
mercutiobut dunno what reverse route is [16:49]
brycecWhat's important to me is that it's not my Smokeping that's broken :p [16:50]
mercutiook
well i don't think it is
[16:50]
brycecI didn't either - that's the purpose to having a diverse set of slaves - but I wanted to check anyways [16:50]
m0undsSLAVES [16:51]
brycecHooray for slaves
There, I said it.
For history and ignominy
@log_search slaves
[16:51]
BryceBot11 results found. Here's #6 Feb 14, 2014 16:49:29 <brycec > I didn't either - that's the purpose to having a diverse set of slaves - but I wanted to check anyways [16:51]
staticsafei can reach hitchcock.freenode.net from my smokeping master but not from my 2 slaves [16:53]
m0undsntt is unable to reach it via their sofia pop (which is the last one i see in my traceroute before it fails) [16:53]
mercutiontt looking glass is working again? [16:53]
m0undsyep [16:53]
brycecAhaha I wonder if Comcast is doing this intentionally (and yes it's only the Comcast slave) https://smokeping.cobryce.com/images/__navcache/139242549325735_1392425493_1392382260.png [16:53]
m0undsit's gross htough
though
[16:53]
mercutioheh [16:54]
m0undshahahah
"it's that brycec guy again.."
[16:54]
mercutioit's not packet lossy [16:54]
brycecIt's actually pretty regular... https://smokeping.cobryce.com/images/__navcache/139242558525735_1392425585_1391820660.png [16:55]
m0undshahaha, awesome [16:55]
brycecTo be clear, I was accusing Comcast of "throttling" Netflix [16:55]
mercutiolooks like verizon graphs :/ [16:55]
m0undsbrycec: accuse netflix of not having enough capacity [16:55]
mercutioactually verizon changed a little
it seems shorter now
[16:55]
m0undsi don't think comcast participates in that cdn program they have though [16:55]
brycecmercutio: I would, but it's perfectly stable on TWC, Orbit, ARP, and Chunkhost
Actually, I think they must
[16:56]
staticsafewhy does spawn-fcgi just keep dying? o_o [16:56]
mercutioso is verizon on non ntt forward path :/ [16:56]
brycecstaticsafe: because aids? [16:56]
m0undsyou have to feed it or it starves
you can't just spawn things willy nilly
they'll die without care
[16:56]
mercutioit just depends if you hit some congested link or not [16:56]
bryceclol m0unds [16:56]
mercutiobut at least verizon got better partially [16:56]
m0unds"better" is a relative term w/vzw
err
vz
[16:57]
brycecprobably vzw too [16:57]
mercutioi think things will get worse over tiem not better [16:57]
m0undsyeah, but i didn't want to confuse anyone [16:57]
mercutioddos attacks are becmoing more and more common [16:57]
m0undsvzw is garbage in the SW US because they're the dominant carrier and are super oversubscribed [16:57]
mercutiono reason to change?
that'd cost money
[16:58]
brycecVZW is dominant in the SW? [16:58]
mercutiowelcome to net neutrality [16:58]
brycecIn the PHX area, it was not. [16:58]
mercutionow you can have degredation to everywhere [16:58]
m0undsit is in CO, UT, NM, and NV
for sure
no idea about PHX specifically, but i know more people and businesses that use vzw in flagstaff and tucson than anything else
[16:58]
brycecMy impression growing up in PHX was that ATT was dominant [16:59]
m0undsa ton of their subs in NM and CO are former alltel subs [16:59]
brycecOh well that's sortof cheating :p
"I'll become dominant, not by people choosing me, but buy buying up those that people chose.)
[17:00]
mercutioit's how the world works
unfortunately
[17:00]
brycec(Reviewing more of my Smokeping graphs, it's clear that Comcast has a few saturated routes. It's not limited to Netflix.) [17:01]
staticsafeComcast<->Cogent, Comcast<->Tata [17:02]
brycecbrycec has much work to do in order to make his Smokeping more useful and friendly, like finishing the traceroute integration... And aggregating graps by slave... [17:02]
mercutioheh
i had an idea about that btw bryce..
[17:02]
m0undsbrycec: it's because alltel's rural network was much larger than VZW or Sprint's, and VZW was having to eat tons of roaming charges (their customers just see everything as native) [17:03]
mercutioif ttl changes it probably means route changes
so if you don't want to do heaps of traceroutes, you can prioritise when ttl changes
it wouldn't always work
[17:03]
brycecheh, I could...
For now, I plan to run traceroutes hourly
[17:04]
mercutioand would do some false positives
i been planning to do my own monitoring thing
[17:04]
m0undswhat was the ping target for that ramped comcast graph? [17:04]
mercutiobut there's a lot of work involved...
so i mostly just been thinking about it so far
[17:04]
brycecm0unds: netflix's nlnog node
Just need to come up with a decent UI, and fix the config parser.
[17:04]
mercutioi want to start by doing passive monitoring of normal traffic
active monitoring doesn't necessarily hit the things that are failing
and it can be hard to know what to monitor
good luck :)
i think one problem with heaps of hosts is it's hard to get an overview of the state of things
if it can come up with something like "paths out via ntt seem broken"
it could be useful
and so if traceroute starts feeding out bound paths, you can start collecting information
and have some intelligence
[17:05]
brycecSo it would need to have some intelligence to udnertand the paths being taken [17:08]
mercutiobut return paths is still a pain unless you have multiple targets doing the same in reverse
yeh
traceroute -A may be the simplest..
i want something that can work in a couple of minutes
when there are isuses
and looking at smokeping for that is hard
so i'll probably start with just measuring at the time
using a mesh system to be able to communicate with hosts even if they're unreachable by one route...
[17:08]
m0undshttp://krebsonsecurity.com/2014/02/the-new-normal-200-400-gbps-ddos-attacks/ [17:12]
mercutiothere's also snmp problems
that aren't being exploited yet
i suspect if ntp gets widely patched they'll move on
there's probably some unknown vector too
i'm surprised there aren't bots that try and buy things on amazon or such from peoples hosts using generated credit cards or such
[17:13]
brycecIsn't there always an unknown vector? [17:15]
mercutioyeh
it seems most attacks target networks atm rather than hosts
that have lots of db activity etc
it's easier to block network attacks atm i expect
more people are ready and waiting to respond
whereas application level stuff could lead to longer outages.
amazon probably has people on hand though
but i imagine lots of banks don't for instance
well not at short notice at 3 am etc
[17:16]
m0undsnah, they do [17:18]
mercutioall of them? [17:18]
m0undsthat's silly [17:18]
mercutioi suppose large ones will
but what about those little in between things
[17:18]
m0undsi know people who work at local banks and national banks, and both have people on call to respond to that stuff [17:19]
mercutiothey don't necessarily have people with expertise to be able to figure out who is real and who isn't
and they can do application level filtering?
maybe i underestimate
[17:19]
m0undsyou do
absolutely
[17:19]
mercutiothat country where banking is really strong got hit by ddos's hard
slovenia?
somewehre near there at least
[17:20]
milkisweden? [17:20]
mercutioit was somewhere with really high internet banking penetration
so probably somewhere cold, but i don't think it was sweden
they had government enforced id cards i know
that had crytographic pgp type keys
for identifying
[17:21]
milkimilki wants [17:21]
mercutioso it's like everyone of their population had a pgp key
Lithuania
maybe have been there
https://www.lb.lt/cyber_attacks_were_used_to_interrupt_the_provision_of_the_bank_of_lithuanias_online_services

maybe it was that
not finding out much in search but that my country has hackable transport cards :)
ahh estonia
http://en.wikipedia.org/wiki/Estonian_ID_card
[17:22]
BryceBotEstonian ID card :: The Estonian identity card (Estonian: Isikutunnistus) is a chipped picture ID issued in the Republic of Estonia by the Citizenship and Migration Board of the Ministry of Internal Affairs. It is officially a primary-picture ID in Estonia, and is therefore recognised by all member states of the European Union or the Schengen Area and some other European countries as an official travel document. For travel outside the EU... [17:25]
mercutioToday 99.6% of banking transactions are done electronically and the number of users of the Estonian Internet banks is more than 1.8 million clients, a bit more than the whole population of Estonia ??? 1.3 million.
99.6%?!
[17:26]
m0unds_uhh [17:26]
mercutiomost older people don't seem to like electronic banking [17:26]
m0unds_my in-laws only recently started using any online banking [17:28]
mercutioexactly
apparnetly only 75% of people are on the internet there
but i suppose 99.6% includes non-internet electronic too
and 75% is still damn high
it's a country i know very little about
but they seem to be quite advanced techonology wise
[17:28]
m0undsi imagine it's not as hard to push tech when your country's population is that tiny [17:31]
mercutionot necessarily
i wonder if they're rich
[17:31]
m0undsit'd likely be easier to push stuff like that into service in a country w/1.3m population than it would be to push it into service in a country with half a billion poeple
people
[17:34]
mercutiotakes just as much effort to create
but distribution is easier
[17:35]
m0undsheh [17:35]
mercutioprobably more that there's less things in the way [17:35]
m0undsexactly [17:35]
mercutioapparently they're the richest east european country [17:35]
m0undscool, tallest midget [17:35]
mercutioi never really hear about east surope
europe
[17:36]
m0undsGDP of estonia is (according to 2012 data) 20bn USD [17:37]
mercutiois that high or load [17:38]
m0undsnominal [17:38]
mercutiothey're near east europe so they can prob get some stuff cheap
nz is 140 billion
with like 4 millionpopulation
[17:38]
m0undsmy state is 80bn with 2m pop [17:39]
mercutiohttp://www.numbeo.com/cost-of-living/country_result.jsp?country=Estonia
it doesnt' loko cheap
but i don'w know value of euros exact
[17:39]
m0undschange the exch rate [17:40]
mercutiohmm internet is like $24usd/month [17:40]
m0undserr, exch currency [17:40]
mercutiooh cool
these sites are often not very accurate i've found
when i look at my own country
[17:40]
brycec"And everybody looks like a hobbit" [17:41]
m0undshttps://www.youtube.com/watch?v=AGF5ROpjRAU [17:42]
BryceBotYouTube Music: "Leonard Nimoy - The Ballad of Bilbo Baggins [FULL VERSION] - best quality" by SputnikMonkey (2m 21s), 1,147,193 views, 9,488 likes and 105 dislikes. Uploaded 2011-09-07T06:23:34.000Z. [17:42]
mercutioit's a bit hit and miss
they have amazingly cheap rent epr month here compraed to what is normal
[17:42]
m0undsso some people look like hobbits and some don't? [17:43]
mercutiono hobbits here [17:43]
brycecm0unds++ [17:46]
m0undssigh. http://krqe.com/2014/02/07/recent-nm-skinwalker-photo-ignites-fear/ [17:47]
brycecStay New Mexican, New Mexico. [17:49]
m0undsyeah.
like the kinda shit that you'd hear about on coast to coast
[17:49]
mercutioheh
https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-prn2/t1/1891280_486670144770622_1185801086_n.jpg
[17:49]
m0undshahaha [17:50]
brycecSeems uncomfortable [17:50]
mercutioyeah probably not pratical
but if it's just to scare your neighbours :)
[17:53]
m0undsi'd have to do it with red paint because we haven't been getting any snow
sigh
[17:54]
mercutiodamn
it doesn't snow here
[17:54]
m0undswe usually get a bunch of snow from dec-feb [17:55]
mercutiowe just get rain [17:55]
m0undsbut it's gonna be 24C tomorrow [17:55]
mercutioi don't particularly like the snow used to get in my old city anyway
there was never enough to be fun, just enough to turn to slosh
24c in winter?
[17:55]
m0undsyeah [17:56]
mercutionice [17:56]
m0undsridiculous [17:56]
mercutioit's 24c here right now [17:56]
brycec@weather ABQ [17:56]
BryceBotThere is 1 weather alert in effect for your area! There is a Fire Weather Watch.
Albuquerque International Sunport, NM: Mostly Cloudy ☁ 62°F (17°C), Humidity: 23%, Wind: From the NNW at 12 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=35.04166794,-106.61472321 or re-request this with: @weather -v ABQ
[17:56]
brycec@weather Kiwi-land [17:56]
mercutiooh i read that at first as a "fine weather watch" [17:56]
BryceBotError, No cities match your search query [17:56]
mercutio@weather auckland [17:56]
BryceBotAuckland, New Zealand: Clear 72°F (22°C), Humidity: 60%, Wind: From the SW at 16 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=-37.00805664,174.79167175 or re-request this with: @weather -v auckland [17:56]
mercutiooh it says 22c.
i am sure the wind is not 16mph
@eather waitakere
@weather waitakere
[17:57]
BryceBotWaitakere, New Zealand: Clear 74°F (23°C), Humidity: 62%, Wind: From the NE at 1.0 MPH Gusting to 4.0 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=-36.857498,174.520554 or re-request this with: @weather -v waitakere [17:57]
bryceclol
windy in the big city
[17:57]
mercutioit's not really a separate city, it's just a part of the city. [17:57]
brycec@weather 99019 [17:57]
BryceBotThere is 1 weather alert in effect for your area! There is a Areal Flood Warning.
Liberty Lake, WA: Clear 44°F (7°C), Humidity: 72%, Wind: From the ESE at 6.9 MPH Gusting to 10.1 MPH -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=47.666508,-117.100792 or re-request this with: @weather -v 99019
[17:57]
m0undsi read that as "waithere" [17:57]
brycecbrycec wonders how to pronounce waitakere [17:58]
mercutioi think it's wai-tak-a-rear-e
s/wai/why/
[17:58]
BryceBot<mercutio> i think it's why-tak-a-rear-e [17:58]
brycecDon't you know? Aren't you there? [17:58]
mercutioi think wai is probably thought of as whey normally [17:59]
brycecYou know, just for that I'm going to put on Flight of the Conchords [17:59]
mercutioi'm not from around here
i live here
[17:59]
m0undslol [17:59]
mercutiolots of places are miscomonly pronounced here
err commonly
gah commounly mispronounced
my brain don't work right
[17:59]
m0undsweather underground is forecasting a much more (still not terribly) reasonable high tomorrow
of 72F vs 77F from accuweather
[17:59]
mercutioWhy-tahk-ah-reh)
see i was wrong
it's becuase they use native words, and where i was from didn't haev natives
http://www.shelleymunro.com/blog/2010/03/04/place-names-that-roll-off-the-tongue/
how's that for confusing
[18:00]
brycecI'm sure it's perfectly easy once you're accustomed to it.
Hawaiian names are similarly frightening to see, but easily picked up
[18:03]
mercutioi dunno, news reporters here mispronuce placenames
god i'm mistyping words even today
some place names have actually hard to say names, because they use uncommon speech
[18:04]
m0undsthere's a lot of wonky tiwa (rio grande valley-area dialect) native names near here
p'oswaugueh, tesuque, ohkay owingueh, tlur pa, ba shi'ie
[18:07]
mercutiopee-o-swaga?
no idea about second
tee-suck?
[18:09]
m0undsteh-soo-kay [18:09]
mercutiotee-sook?
ahh
i like the third one
okay oh-winga?
[18:09]
m0undsokay ohwingay
oh-ween-gay
[18:10]
mercutioi'm terrible at this [18:10]
m0undsthe spanish version of the first one is pojoaque, po-wah-kay
tlur-pa is t-lur pah, last one is bah shee-eh
[18:11]
mercutiohttp://www.itnews.com.au/News/372345,cisco-stung-by-switch-memory-glitch.aspx
uhh
sounds nasty
i wonder if that is at all related to up_the_irons issue :/
(cos it showing some memory issue, it hard to know if OS bug or memory problem though)
[18:17]
up_the_ironshmm
HMMMMMMMM
[18:20]
mercutioit's kind of scary, 7 years of memory problems [18:20]
up_the_ironsyeah
i still think it was an IOS bug. after removing the Any2 port, no more crashes. with the Any2 port, crashed 3 days in a row within 5 minutes of 05:30 PST. WAY too much of a coincidence for memory problems.
[18:21]
mercutiothat said most of these problems seem to be triggered by power cycling
yeh i think it probably ios bug too
i wonder who cisco uses for memory
[18:22]
m0undsomfg.
that's the issue i had with 3 whole stacks of 2960s in july
[18:24]
up_the_ironsgood thing i don't power cycle my cisco's much ;)
m0unds: ouch
[18:24]
m0undsall had multiple-years' worth of uptime, all went down because of an enormous systemic power failure during a huge storm [18:25]
up_the_ironsm0unds: looks like you'd get free replacement memory / gear now :) [18:25]
mercutiodo you still have them?
i suppose you just claimed insurance?
[18:25]
m0undswe still have 30 of them in service - i revived 9 out of 12 of them
nah, they just did RMAs
[18:25]
mercutioor does that not work there [18:25]
m0undsit was just a power down, no voltage craziness or anything [18:26]
mercutioahh
2960 is confusing, there are multiple versions of them :)
through many years
[18:26]
m0undsyeah [18:26]
mercutiowhen i think 2960 i think old
but they're still selling new ones
[18:26]
m0undsmine are....WS-C2960S-48FPS-L
3 years old
[18:27]
mercutios is newer than g i think
yeah g is 2005
[18:27]
up_the_ironsm0unds: so they powered down b/c of the power failure then refused to come back up properly?
that is f'in scary
[18:27]
mercutioit was a pre-emptive shut down?
or just protected feed?
cos you can still get spikes sometimes that are short
[18:28]
m0undsmanaged shutdown [18:29]
mercutioeek [18:30]
m0undsprior to UPS gear running out of juice (generator failed to start, it cooked its battery harnesses)
and yeah, powered down and when it came back up, it would dump the second IOS finished booting
full on crash and boot loop
[18:30]
mercutioup_the_irons: are you using 2960s? [18:31]
up_the_ironsmercutio: no [18:32]
mercutiojust straight into router? [18:32]
up_the_ironsmercutio: yeah our topology is very flat [18:32]
mercutionice
i don't like 2960s :)
[18:33]
up_the_ironsVM -> VM Host -> (trunk port) -> s1.lax (or s7.lax for the GigE ports) [18:33]
mercutioso when there's two ports on dedicated servers they both go to s1?
directly in?
or do they have switches?
[18:34]
up_the_ironsoh, no, the dedi's have an edge
s8.lax, s9.lax for dedi's (each have dual 1 Gbps, to each switch), then to either s1 or s7
[18:36]
mercutioahh [18:37]
m0undsthanks so much for linking that stuff (the blog post and whatnot)
just passed it on to our casino ops IT director and neteng guys
[18:37]
mercutiocool.
it sounded like another of those times when buying things at different periods of times or from different vendors could help
it's like when peopld do raid with mirrors, and all their hard-disks come from the same lot
and they all fail at once
[18:37]
up_the_ironsyeah
i use different vendors ;)
[18:38]
mercutiobut it's a pita to multivendor [18:39]
up_the_irons1x Hitachi, 1x WD RE4 [18:39]
m0undsyep, especially with a time-constrained large project [18:39]
up_the_irons(pair) [18:39]
mercutioahh cool. [18:39]
m0undsalso, we got our 2960s from 3 different vendors [18:39]
mercutioi didn't realise you were doing that [18:39]
m0undsi think our 6500s came from two different vendors too
but same timeframe
[18:40]
mercutioit can be hard to get diff lots. [18:40]
m0undsyep [18:40]
mercutioi have my ssd's with different lots :/
well at least diff looking model numbers
[18:40]
m0undswe had a bunch of cisco-brand SFPs just croak this year too
it's been a rough year for cisco at my facility
haha
[18:40]
mercutiowhy use cisco sfp?
oh you cisco shop?
[18:41]
m0undson top of our IT/ops guys having tons of issues with their new 7000-series stuff
because it was what was specced
[18:41]
mercutioheh [18:41]
m0undsif it was up to me, we wouldn't have used any cisco gear, but our VMS vendor prefers cisco and won't certify installs that don't use cisco gear [18:41]
mercutiocisco tax is quite a lot on sfp's though [18:41]
m0undsyep, it's absurd [18:41]
up_the_ironsm0unds: i heard bad stuff about the 7k series too [18:41]
mercutioi mean i'm more ok with cisco where it matters.
not that i necssarily agree iwth it, but i can understand not wanthing to change. buf sfp isn't going to be a problem with non cisco
[18:42]
m0undsup_the_irons: i've been hearing they're already trying to forget about it in favor of the newer 6xxx chassis' [18:42]
mercutiohave they considered changing vendors? [18:42]
m0undswell, cisco actually changed suppliers for their sfps because of rampant failures [18:43]
mercutioapparently huwai are trying quite hard to compete with cisco now days. not that i know anything about them
but juniper have qutie a bit of market share now
[18:43]
m0undshuawei tried really hard to reverse engineer a lot of competitors' products because their own os and stuff was garbage [18:43]
mercutioapparently juniper doesn't have smp support yet [18:43]
up_the_ironsm0unds: the newer 6xxx being the 65xx-E or..? [18:43]
mercutioi was surprised.
oh and apparently openbsd is adding smp support in kernel now
they done audio, now doing disks and network interfaces i think, before making the network and disk interfaces smp..
cos one of juniper's present-day issues is being slow at taking bgp tables, reconverging etc.
however much cisco may have issues, for the cpus they use they have a damn fast bgp implementation.
i mean you were saying bird is fast up_the_irons .. but the cpu on that bird machine i bet is way faster than the cisco.
[18:43]
up_the_ironsmercutio: well that's true too :) [18:46]
mercutiorpki is going to slow bgp down soon too if people start implementing it
that and when ipv6 takes off more
[18:46]
m0undsup_the_irons: 6800 [18:49]
up_the_ironsm0unds: ah [18:49]
m0undskinda weird considering the nexus stuff should just be better :/ [18:51]
mercutioshould
they are different platform
[18:51]
m0undsbut such horrible teething problems with it on the software side have wrecked credibility [18:51]
mercutiowould be nice if this stuff could be open source :)
wher epeople still might pay to get issues fixed
or for support
facebook was doing some open switch thing, i think that's as close as it gets
hmm ok probably expensive, it's 48 10 gigabit ports and 4 40 gigabit ports
[18:52]
grepidemicshuttleworth announced ubuntu will now go with systemd. [18:56]
up_the_ironsi heard of a juniper (or was it cisco?) version of that
grepidemic: godamnit
[18:56]
mercutiohttp://www.opencompute.org/blog/progress-in-opening-up-the-network-/ [18:57]
grepidemicso i guess upstart is gonna end [18:57]
mercutiogrepidemic: i actually said this earlier today :)
but yeah, it looks legit
[18:57]
up_the_ironswell, i didn't like upstart much either. so we're going from "meh" to "meh"
i won't panic
[18:57]
grepidemicit is on shuttleworth's b;pg [18:57]
mercutioi assume they don't want to focus on maintaining upstart
too busy with mir and ubuntu phone
there's mellanox intel boradcom and cumlus networks all doing open soruce switches
[18:57]
grepidemici don't know much about mir. i am guessing it is mostly for mobile linux? [18:58]
mercutioit's for desktop too
it's like wayland but different
with a lot of ambiguity about why
https://github.com/Mellanox/SwitchX-interfaces/tree/master/SwitchX-2/source
[18:58]
grepidemicthe only things i know about wayland are from a conference video 1 hour. the guy is mostly talking about how it is better than X. [18:59]
mercutiogrepidemic: yeh i heard about better than X stuff being necessary for 15 years
but X is still around, and doesn't seem to be struggling that badly
[19:00]
grepidemici have no problems with X. but maybe i'll disagree with myself after using wayland. [19:00]
mercutioX can be kind of bad over high latency links
but for local desktop it's fine
it may be more efficient whatever they're doing
but i assume they'll cover that all up in layers of abstraction.
there were some security concerns about applications being able to impede other applications once upon a time, i assume they're still valid.
i know netscape 4.7 used to sometimes lock X up until you sshed into from remote host nad killed it.
[19:01]
grepidemici think wayland will have something similar to "ssh -X" which will be more efficient than X11 forwarding. [19:03]
mercutioi was playing with X forwarding the other day. skype seems to resize slowly when using it.
but other than that it seems just like being local.
but that's on lan.
on wan it still sucks.
but i think people moving to web for that stuff now anyway
[19:04]
grepidemicmaybe desktop OS's will just become a client for web apps. [19:05]
mercutiolike chromeos? [19:05]
grepidemicand everything will be cloud. [19:05]
mercutiohaha
now i know you taking the mickey.
[19:05]
grepidemicthe new MS guy is a cloud person. [19:05]
mercutiomy condolences. [19:05]
m0undsheh [19:06]
mercutiowell private cloud is fine [19:06]
grepidemici never used chromeos, but i'm not really too fond of cloud services. i like using a vps for storage and running my own services, but i don't like the idea of EVERYTHING being stored over the internet. [19:07]
mercutioit seems even technology focused companies are sometimes using things like AWS now.
grepidemic: i kind of do.. but..
as soon as your net goes down you lose everything.
bbl
[19:07]
grepidemicthe only cloud thing i use (that im aware of) is Steam client. [19:09]
m0undshaha. http://t.co/S3ZKrxcf2g [19:13]
BryceBothttp://t.co/S3ZKrxcf2g -> http://twitter.com/HistoricalPics/status/434488522704441344/photo/1 [19:13]
...... (idle for 28mn)
mercutiosteam's cloud sync is terribly unreliable
but their normal stuff is just straight cdn i'm prettty sure
[19:41]
m0undsorigin does a pretty good job of detecting and resolving local/remote save conflicts [19:43]
brycecbrycec has never had an issue with Steam Cloud Sync [19:43]
m0undssteam just says "there's a conflict, if you start the game you'll lose your save" [19:43]
brycecHow can there be a conflict? It's not like you can play the game in two places at once [19:43]
m0undslike..you were playing and the game crashed, and steam didn't sync
so your save is different than the one in TEH KLAUD
[19:44]
brycecHuh. Odd. [19:44]
m0undsyeah, only game i've seen it with was the walking dead (telltale)
it ended up eating my save so i had to start over
[19:44]
brycecIt's always appeared as the sync was part of Steam, not the game, and when Steam detects the game has closed, it syncs the relevant data. But that's total speculation based on what I've seen (notably, the "sync" popup is Steam's, and it shows in the Steam "Downloads" page briefly)
Bummer
[19:45]
m0undsyeah, i think that's how it does it
it has like a local glob of save data
and it pushes it up when the title closes
but when your game mangles it or it gets mangled in transit somehow, then you have that weird condition that might lead to losing the save altogether
i just flipped off steam cloud saves for that game and set spideroak to back it up since it versions files
[19:45]
mercutiom0unds: not ime [19:47]
m0undss/versions files/saves versions of files/ [19:47]
BryceBot<m0unds> i just flipped off steam cloud saves for that game and set spideroak to back it up since it saves versions of files [19:47]
mercutiobut i dunno my games are stuck in not syncing [19:47]
m0undsdunno what to tell you
blow the hobbits out of your cpe
[19:47]
mercutioi dunno i just choose don't sync
then next time i choose don't sync again
the syncing is just key bindings etc for things like dota2
i did kind of want my key bindings, and once it actually worked.
steam has various issues
like i used to find every time i started steam it wanted to download a huge update to do with big picture
which i don't even use.
[19:47]
bryceclol, a nuissance but not an "issue" :p
they just keep their program up to date
[19:49]
mercutionow i play dota2 heaps i use steam way more, so steam updates seem less frequent, but dota2 updates can still be multiple updates a week
well it's like 80mb+ download i think
i like it how chrome got their update sizes down.
i'm not necssarily opposed to updates, but when they're huge, and you have to wait for them at startup, they're kind of annoying
chrome is good, because it doesn't make you wait for the update, and they're small.
[19:49]
up_the_ironshttps://twitter.com/bsdvps/status/434532130480082945 [19:50]
BryceBotTWITTER: OpenBSD 5.4 is now available to order (Sat Feb 15 03:38:27 +0000 2014, retweeted 1 times) [19:50]
brycec*finally* [19:50]
mercutionow if only ubuntu would do dif udpates [19:50]
brycecJust as 5.5 is on the verge [19:50]
mercutio5.5 is going to break things [19:50]
brycecYes :D
Can't wait
[19:51]
mercutioit's suggested that people wanting to update soon go to snapshots atm
which is curious
[19:51]
brycec(if i really couldn't wait, I'd install a snap...) [19:51]
mercutioi'm using 5.3 [19:51]
brycecWhy not 5.4? [19:51]
mercutiowhy 5.4?
4:50PM up 257 days, 4:15, 1 user, load averages: 0.19, 0.17, 0.16
[19:51]
brycecBecause you're s'posed'a upgrade! :P
Also 5.3 will be dropped from most mirrors once 5.5 is out
[19:52]
mercutioi dunno i have one host much further behind [19:52]
brycecnot to mention "support" [19:52]
mercutioonly if you have problems [19:52]
up_the_ironsnow i have to do FreeBSD 10 [19:52]
mercutioi quite like freebsd 10 [19:52]
brycecFreeBSD doesn't really matter... :P [19:52]
mercutiohow do you find out openbsd release?
oh right uname -a
ok this host is openbsd 4.8
uptime of over 1000 days
going to replace hardware and shift to freebsd 10 though
the mirrors thing is intresting
i haven't installed anything on it recently but id idn't have issues getting packages last time i did
curiously where i was installing packages has 4.9 packages but not 4.8
PKG_PATH=http://mirror.internode.on.net/pub/OpenBSD/4.8/packages/i386/
oosp
PKG_PATH=http://mirror.internode.on.net/pub/OpenBSD/4.8/packages/i386/
gah
cut and paste issues
http://ftp.vim.org/OpenBSD/
that has old openbsd to 2.0
[19:53]
m0undsfreebsd 10 has been good so far - only issue i had was w/pkgng which i don't really use too frequently [20:04]
mercutioi reading openbsd 5.4 changes
nothign really strikes out as requiring me to update
i liked pkgng
i hit a compile time bug with trafficserver
due to clang
but that was fixable, and due to bad code
http://www.openbsd.org/cgi-bin/man.cgi?query=ppoll&sektion=2
[20:05]
m0undsthe only pkg i've installed was to work around a compile time issue with something, i don't remember what [20:07]
mercutioactually that looks cool
and is new in 5.4
i might jump to a snapshot of 5.5 now
[20:07]
hmm em0 watchdog timeout [20:16]
CaZedisable mpbios [20:17]
mercutioit's in the installer
is it just mpbios that causes that?
[20:17]
CaZeOh, weird. [20:17]
mercutioi'm pretty sure i had it a while back without mpbios [20:17]
CaZeIve never had it hapen in the installer. [20:18]
mercutiomaybe it was doing it before, and i didn't do enough network traffic [20:18]
CaZeI just installed a snapshot a few weeks ago. [20:18]
mercutioi'm downloading from mirrors.arpnetworks.com
ahh cool, i doing snapshot
figure it's not worth jumping to 5.4
i normally don't do update using bsd.rd
but 5.5 requires it, so i figure may as well.
err encourages it heavily.
[20:18]
CaZeI always use bsd.rd. [20:19]
mercutioit varies for me [20:19]
CaZeI can't think of a reson not to, as long as you have a bootloader installed, and network access. [20:20]
mercutiohmm got another on boot
and some pf syntax errors
damnit, it wont' login
first time i've had an issue actually
oh 5.5 i smeant to not actually run earlier binaries iirc
so it prboably doesn't liek my zsh
[20:20]
m0undszsh is for wizards [20:23]
BryceBotYER A WIZARD m0unds [20:23]
m0undsahahahah
i love zsh
[20:23]
mercutioheh so do i
damn
still not working
so it's working for you CaZe ?
[20:27]
ohhh
my fault
i didn't read instructions, apparently you have to do some stuff upgrade doesn't do
[20:36]
network isn't working at all hmm
caze: are you on virtio or legacy?
[20:43]
mnathani@smokeping [20:45]
BryceBothttps://smokeping.cobryce.com/ [20:45]
mnathani@uptime [20:45]
BryceBotBot uptime: 15 days, 21 hours, 45 minutes, and 17 seconds. [20:45]
mnathani@mnathani [20:45]
BryceBotmnathani: Have you checked whether Windows Firewall is enabled and dropping packets?
451 results found. Here's #366 Feb 09, 2014 13:06:24 <mnathani > 5 packets transmitted, 0 received, 100% packet loss, time 4101ms ( 0.v.freedaemon.com )
[20:45]
brycec@uptime host [20:45]
BryceBothost uptime: 141 days, 21 hours, 58 minutes, and 44.800000000745 seconds. [20:45]
mercutiooh that was just needing to disable mpbios [20:45]
m0undsprecision
almost get to walk a quarter mile to my car. muahahaha.
[20:45]
mercutioi'm half tempted to shfit to freebsd now :)
apparently downgrading to 5.4 isn't supported too
[20:49]
m0undsdoh [20:52]
brycecDowngrading is never supported by OpenBSD. Might be "possible" but never "supported." [20:53]
mercutioi'm hopeful i can make it work anyway
i think it's something to do with the password database
[20:55]
step by step..
it seems i wasn't forcing shell right
[21:02]
woot got it [21:09]
CaZemercutio: legacy [21:23]
mercutioit was just mpbios making ethernet fail in the end
i've used self-compiled kernels for ages so haven't really noticed
i dunno why installer works
and editing /etc/passwd and /etc/master.passwd didn't seem to work to change shell from zsh
in the end i just upgraded zsh
[21:23]
CaZeYou have to regenerate the db. [21:25]
mercutioi did
that's the strange thing
otherwise i think things wouldn't work at all?
i did pwd_mkdb i think
yeh pwd_mkdb
[21:25]
CaZeWell, there's vipw. [21:26]
mercutioweird, now it works [21:26]
CaZebbl [21:26]
mercutioi ran it again and it set it to zsh
yeh i normally do vipw
[21:26]
.... (idle for 18mn)
i think openbsd 5.5 is slightly quicker [21:45]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)