[00:23] *** Webhostbudd has quit IRC (Quit: Leaving) [01:00] *** meingtsla has quit IRC (Quit: Leaving) [01:04] *** meingtsla has joined #arpnetworks [01:29] *** Mexicainvexed has quit IRC (Ping timeout: 260 seconds) [01:39] *** [FBI] starts logging #arpnetworks at Wed Oct 17 01:39:38 2012 [01:39] *** [FBI] has joined #arpnetworks [01:46] why is the fbi logging this channel? [01:48] *** Ehtyar has joined #arpnetworks [02:36] kvr27 (beta) server going down for one last reboot... [02:41] crap, one more time... [02:54] heh [02:54] what are you doing to it? :) [02:55] just curiosity not waiting on it or anything [02:56] oh it's up anyway [02:56] says uptime of 6 minutes [03:06] *** hive-mind has quit IRC (Ping timeout: 244 seconds) [03:08] *** hive-mind has joined #arpnetworks [03:21] mercutio: kernel patches [03:24] while we may still need some openbsd / freebsd testing for the beta VMs, my cacti server VM has been running on it for over two months without issues, so I'm declaring kvr27 production ready. I have provisioned the first Linux customer on it. [03:24] (cacti server runs Linux if that wasn't obvious ;) [03:46] mercutio: can you re-run the test you mentioned was "it's quite reproducable" [03:46] i found the RAID controller write cache was off; this was one reason iowait was high; now fixed [03:47] no rush, just whenever... [04:36] Do you have any process by which you evaluate a guest OS on a beta system? [04:36] (features to try, stress testing, etc?) [04:50] I know some issues I've had in FreeBSD VMWare guests that seem fairly reproducable under higher IO situations. [04:50] They seem to crop up more frequently when ZFS is involved as well. [05:47] easymac: i want to see 1) no guest panic, 2) clean network throughput, 3) decent disk performance [05:48] i've seen frequent guest freezing with ZFS and load [05:48] less so if a lot of RAM is involed [05:55] zfs necessarily requires lots of ram for optimimum performance (I know this and I only admin'ed a freebsd system that used it for a very short time) [05:58] yeah [06:56] recommendations on a good Intel NIC for FreeBSD? (dual port, gigabit) [06:57] so that it uses the em0 driver [06:57] igb0 is driving me insane [06:57] INSANE [07:21] http://www.freebsd.org/cgi/man.cgi?query=em&apropos=0&sektion=0&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html [07:21] I'd presume that would be a good list to work from [07:22] I can't say I have hardware knowledge from a freebsd perspective ;-) [07:29] *** cmeiklejohn has joined #arpnetworks [07:32] up_the_irons: don't look at me at 6:47PM i'll be gone :-) [07:32] up_the_irons: but yes i'll help you rock that app. [07:47] *** cmeiklejohn has quit IRC (Quit: Computer has gone to sleep.) [08:23] *** er|c has quit IRC (Ping timeout: 276 seconds) [08:29] *** er|c has joined #arpnetworks [08:29] *** er|c has quit IRC (Changing host) [08:29] *** er|c has joined #arpnetworks [08:34] *** er|c has quit IRC (Ping timeout: 276 seconds) [08:39] *** er|c has joined #arpnetworks [08:43] *** Webhostbudd has joined #arpnetworks [09:04] *** chmod\ has quit IRC (Ping timeout: 246 seconds) [09:05] *** chmod\ has joined #arpnetworks [09:27] *** gcw|mbpro has quit IRC (Remote host closed the connection) [10:45] *** HighJinx has quit IRC (Quit: Computer has gone to sleep.) [11:17] is there an ipv6 routing issue im not aware of? [11:19] up_the_irons: you there? [11:20] my traceroutes are hitting 120 hops [11:20] that definitely can't be right [11:20] the packets just never get there [11:20] looooop [11:20] exactly [11:20] that's what im thinking [11:20] does arp control this router? [11:20] 13 2607:f2f8:0:102::1 (2607:f2f8:0:102::1) 105.860 ms 105.698 ms 105.629 ms [11:20] after that it looks like it loops [11:22] *** Ehtyar has quit IRC (Remote host closed the connection) [11:23] Webhostbudd: do you have a sample traceroute with source / destination [11:23] I'm reaching google.com via ipv6 with no problem [11:23] RandalScwartz: same here [11:23] from the vps [11:24] however, from my home ipv6 to the vps [11:24] i get that loop [11:24] just a sec on the full trace [11:24] and a few others I tried [11:24] *** HighJinx has joined #arpnetworks [11:24] ahh, yes, let me try from my home to here. [11:25] http://sprunge.us/AFAY [11:25] I can ping6 from home to red.stonehenge.com [11:26] traceroute is good too [11:26] what's your host? [11:26] 2607:f2f8:a9e4::2 [11:26] it looks like it is one of the intermediate routers [11:26] because it can ping out fine [11:27] could i somehow be causing this? [11:27] I'm checking... [11:28] I'm getting stars after a bunch of hops. [11:28] let me try from red. [11:28] yea [11:28] Yeah - I can't get to yours from even nearby [11:28] I think it [11:28] is your end perhaps. [11:28] weird [11:28] alright [11:28] I cant ping 2607:f2f8:a9e4::2 from home [11:28] oddly enough i can ping red just fine [11:28] I'd still file a support request in case something is broke [11:28] perhaps ICMP are blocked? [11:29] mnathani: nope [11:29] it has an almost identical pf ruleset to my other ipv6 box [11:29] which can be pinged just fine [11:29] pass inet6 proto icmp6 all icmp6-type {echoreq unreach neighbradv neighbrsol routeradv} [11:30] maybe your default route is busted? [11:30] Well even the ipv6 ping sourced from Sprints looking glass : Success rate is 0 percent (0/5) [11:30] maybe [11:30] hmmmmm [11:31] try pinging my home IP: 2001:470:b148::25 [11:31] it just has to be something on my end [11:31] works fine from the box [11:31] no problem reaching you from red [11:31] (your home, that is) [11:31] see, it's weird, i can ping out and get response [11:31] so routing may be working? [11:32] sounds like wonky firewall then [11:32] yes it does [11:32] by the way, if you haven't seen mtr, get it. [11:32] it's far slicker than ping or traceroute [11:32] from my ARP VPS: 18 packets transmitted, 0 received, 100% packet loss, time 17582ms [11:32] i just turned off pf [11:32] still nothing [11:33] ... http://en.wikipedia.org/wiki/MTR_(software) [11:33] yea i've used it [11:33] and long since forgotten [11:33] =p [11:35] my rc.conf is pretty simple [11:35] ipv6_enable="YES" [11:35] ipv6_defaultrouter="2607:f2f8:a8e4::1" [11:35] ifconfig_em0_ipv6="2607:f2f8:a8e4::2/64" [11:41] nothing? [11:41] hmmmm [11:42] I'm using the 0 address for my box. :) [11:43] red.stonehenge.com has address 208.79.95.2 red.stonehenge.com has IPv6 address 2607:f2f8:3080:: [11:43] shorter to type [11:45] =p [11:45] yea [11:45] i should actually use the 0 [11:45] anyway [11:45] ill see what i can try and change [11:46] I also have ipv6_gateway_enable=YES [11:46] I think that was to try openvpn over ipv6 though [12:37] *** dzup has joined #arpnetworks [12:45] up_the_irons: well the test runs a lot faster :) [12:45] round-trip min/avg/max/std-dev = 0.559/1.208/7.285/1.409 ms [12:45] and that's not as bad [12:46] up_the_irons: speed increase of roughly 4x for my test [12:47] where it's twice as fast without the sync commands [13:23] *** Ehtyar has joined #arpnetworks [13:28] anyone else seeing packet loss to their VPS? [13:28] *** Ehtyar has quit IRC (Quit: Going!) [13:29] twobithacker: not noticing anything here, mtr? [13:29] yeah, it's clean to ge0-arpnet.cust.lax07.mzima.net then about 25~30% packet loss [13:30] twobithacker: hostname/IP? [13:31] Comcast on the way there, trying to get a trace back now [13:32] 174.136.99.74 is my vps, I'm on 17216.146.45.2343 [13:32] err 216.146.45.243 [13:32] IRCing from my VPS too, the packet loss is more than mosh can deal with nicely [13:33] ah yes [13:34] I have loss over v4 as well [13:34] 0% loss from Toronto [13:34] ah, v6 is much better [13:35] *** mercutio has quit IRC (Ping timeout: 252 seconds) [13:36] *** mercutio has joined #arpnetworks [13:36] looks like it's the return path, packet loss as soon as it hits Comcast [13:36] maybe someone saturating a link :/ [13:39] go comcast [13:45] i think the loss just got worse o_o [13:46] heh [13:46] i lost connection to freenode [13:46] which seems to happen to lots of people frequently [13:46] http://pastie.org/private/agllzswbtezlpyyjtz2a [13:46] but i was just thinking yesterday how i seem to avoid it [13:46] from both sides [13:46] wow [13:46] I think thats why my friend was complaining about not being able to reach his znc session [13:46] that looks like congestion to mzima in genreal [13:47] which most outgoing routes go over [13:47] until up_the_irons get's his new router [13:47] znc is still connected to IRC because its over IPv6 [13:47] hmm [13:47] happsne here too it seems [13:47] ge0-15.as01.lax07.mzima.net [13:47] as starting hop [13:47] ahh that's same as you [13:48] yea seems like mzima is having issues [13:48] im sshing in through another VPS via IPv6 [13:48] notice how it doesn't seem consistent between hops [13:48] i feel like i have slight lag [13:49] but smooth [13:49] *** solj has joined #arpnetworks [13:49] hmm it's routing via trit [13:49] oh hey [13:50] how do i do a paste? [13:50] pastie.org [13:50] http://pastie.org/5075292 [13:50] I'm guessing v6 is mostly over peering with HE [13:50] who's saturating my linkz?! [13:50] i think it's an incoming issue not outgoing [13:50] based on that [13:51] but the other things i'd found were screwed in both directions [13:51] err i mean packet loss all the way [13:51] telstra must have a different return path that's hitting an unsaturated link [13:51] man i am gettin lagielagg [13:51] i can do the reverse path from that ip [13:52] oh wow 1 Gbps of traffic, who will i shit can today... [13:52] http://pastie.org/5075303 [13:52] up_the_irons: O_o [13:53] twobithacker: not telstra return path but yeah [13:53] gah [13:53] this wind is creepy [13:53] let's hope the power doesn't go out again [13:59] up_the_irons: who? Thought you limited us to 100Mbps, so that would be 10 users to shitcan... and grammatically speaking, "whom" [13:59] things look clearer now [13:59] brycec: it's prob incoming ddos [13:59] which can go way over that [13:59] probs [13:59] * brycec would be interested to know who all is saturating up_the_irons' links too [14:00] some loser kid in his basement i imagine [14:00] brycec: you can't limit incoming though [14:00] target has been identified and null routed [14:00] yeah, that's much better from here [14:00] swoot [14:00] it does look like it cleared up [14:01] yeah I know up_the_irons (unless you either have friends on their end, or like borking the BGP periodically) [14:01] up_the_irons: that write cache thing doubled my performance with and without sync [14:01] err actually i think it was 4x [14:03] up_the_irons: outgoing packet flood? :P [14:04] static: i think it just happened incoming was the same as outgoing for the hops that showed all loss [14:05] brycec: we basically give the target IP a null route community and certain important peers / transits cease forwarding traffic to this IP [14:05] bingo [14:05] mercutio: yeah i'm not surprised, that write cache rocks [14:05] up_the_irons: well, it use to be slower than the old node... [14:05] but not by a lot [14:05] and a bit variable [14:05] mercutio: would you say the performance is on par with the regular VPS' ? [14:05] and now it's obviously faster :) [14:05] up_the_irons: i /think/ so [14:05] staticsafe: incoming :) [14:06] ah [14:06] but i ssh into the old vps to get into the new vps [14:06] or via ipv6 [14:06] ipv6 is longer route though [14:06] mercutio: oh so it's faster now? [14:06] tar definitely seems faster [14:06] nice [14:06] i was disconcerted by the high network ping times when untarring before htough [14:06] mercutio: do those high network ping times still occur? [14:06] and vmstat was showing very few interrupts when that was happening [14:06] up_the_irons: well highest was 7 msec [14:06] in my untar [14:07] i assume it's probably a bug somewehre where it's running out of queue slots or something [14:07] * staticsafe e-mails friend [14:07] and spinning in the inrerrupt handler [14:07] and not receiving network packets while that happens [14:07] so teh write cache thing really might alleviate it, and it may not happen at lower load anyway [14:08] but yeah it was over 200 msec before the write cache peak [14:08] mercutio: roger [14:08] oh the other weird thing is that ping times in general are higher than old vps [14:09] like if i ping the ipv6 gateway [14:09] but that may be going through an extra switch or something [14:09] mercutio: nah same switch [14:10] mercutio: how much higher are we talking about [14:10] about 100% [14:10] err 50% [14:10] so double? [14:10] yeh [14:10] but [14:10] that's still like half a msec [14:10] --- 2607:f2f8:add0::1 ping6 statistics --- [14:10] 29 packets transmitted, 29 packets received, 0.0% packet loss [14:10] round-trip min/avg/max/std-dev = 0.394/0.622/1.158/0.178 ms [14:10] that's old one [14:10] yeah i don't think i'm gonna be able to do much about [14:11] it may be ethernet coalescing setting [14:11] --- 2607:f2f8:add0::1 ping6 statistics --- [14:11] 27 packets transmitted, 27 packets received, 0.0% packet loss [14:11] round-trip min/avg/max/std-dev = 0.696/0.928/1.821/0.263 ms [14:11] that's new one [14:11] btw, this recent UDP flood was targetting an IP on kvr26, and I recently used my new NOTRACK rules (that we talked about) on that host. Man, even with 1 Gbps incoming, shell over SSH to kvr26 was clean as a whistle, no delays at all. [14:12] * up_the_irons is happy about that one [14:12] sweet [14:12] up_the_irons: nice [14:12] my beta kvm is on kvr27 isn't it? [14:12] mercutio: yup [14:12] does that have notrack too? [14:12] mercutio: yeah looks like double [14:12] mercutio: try IPv4, not IPv6 though. IPv4 is hardware accelerated [14:12] up_the_irons: any specific target IP on kvr26? [14:12] i first noticed it when thinking it was weird that gateway had much lower ping than the other vm [14:12] yeah kvr27 has notrack [14:13] yeh well the ipv4 gateway is congested atm [14:13] solj: yeah, the target was identified as a single IP [14:13] err i mean pinging next hop is deprioritised [14:13] * solj hopes it wasn't mine then :-/ [14:13] and the ipv4 from the new vps goes through the first vm [14:14] so would need to contrast it against something [14:14] solj: if you can still reach your vps, it wasn't you :) [14:14] mercutio: ah right [14:14] oh up_the_irons ... is kvr27 on 100 megabit atm btw? [14:14] yes [14:14] cool [14:15] well ipv6 performance to another host in los angeles was basically the same [14:15] cool [14:15] up_the_irons: awesome, thanks! [14:15] solj: np [14:16] oh, and your ethernet card issues on freebsd would probably be alleviated by Intel® PRO/1000 PT Server Adapter [14:16] they're standard pci-e cards [14:16] without the fancy stuff [14:16] err withotu the really fancy stuff [14:17] latency to another LAX VPS from my arpnetworks vps - http://sprunge.us/iVFI [14:17] :) [14:17] heh under 1 msec is nice static [14:18] yep [14:19] we interviewed the KVM guys today on FLOSS Weekly [14:19] pretty active chat room [14:19] that sounds like a dentist magazine [14:19] oooh, and the audio is already out [14:19] Webhostbudd: nothing on ipv6 changed btw. looping is a sign that my router can't see a route to you [14:19] FLOSS = free libre open source software [14:20] hmm [14:20] my podcast that gets about 50k downloads a week [14:20] mercutio: yeah after a bit of research i also found the PT card and NewEgg'd it already :) [14:20] latency over v6 is a bit higher cause the other LA host does not have native v6 and is using a HE tunnel [14:21] is it audio only? [14:21] up_the_irons: oh cool [14:21] RandalSchwartz: nice! i'll have to listen to that one [14:21] static: he.net tunnels aren't likely to be that slow actually [14:22] it isn't [14:22] for just los angeles traffic [14:22] staticsafe: nice traceroute! that VPS at Cyberverse is about 1 mile down the street, so yeah, pings are way low :) [14:22] :D [14:22] thats my friend's box with Chunkhost [14:22] mercutio: http://sprunge.us/iRTH [14:23] a lot of that latency's coming from the arp ipv6 router having higher latency [14:23] if i trace from another host it's 0.8 msec to that destination [14:24] although more hops [14:24] http://pastie.org/5075448 [14:24] gonna get some air [14:24] why is it so hot in LA now? It's almost november! [14:24] * up_the_irons takes a walk [14:25] mercutio: interesting [14:25] welcome to global warming [14:25] i'm listening to this web cast [14:25] about kvm [14:26] twit.tv/floss ? [14:26] yeh [14:26] yeah. that's what I sound like :) [14:26] so kvm was made for windows [14:26] no - there's a version of it for windows [14:27] ok this is too basic for me :) [14:27] no - wait... we get deeper [14:27] I just had to give gradual overview [14:27] i skipped ahead :0 [14:28] gah i need more coffee [14:28] the developer is hard to understand/follow for me [14:28] probably partially accent [14:28] Yeah - they failed to follow our instructions about audio [14:28] it's also partially i think cos of being a geek :) [14:28] and they're israeli [14:29] rather than public talker [14:29] ahh ok [14:29] *** valleyfox has joined #arpnetworks [14:30] My instructions are to have a mic that is no further than six inches from the mouth. they shared a single mic between them that was 2-3 feet away [14:30] that makes the sound very muddy [14:31] oh [14:32] it reminds me of skype [14:32] but with skype you have feedback loop [14:32] i use skype on my laptop rather than desktop with some inbuilt mic [14:32] it *is* skype [14:32] so it probably sounds terrible [14:32] oh real [14:33] yeah [14:33] skype isn't great for audio quality [14:33] it is if you have enough bandwidth [14:33] really? [14:33] i only have 1 megabit upload i suppsoe [14:33] in fact, it's better than nearly anything else [14:33] but i've never noticed skype being that great [14:33] the SILKv3 codec is amazing [14:33] i find voip better [14:33] better than G.729 [14:33] voip doesn't use g729 [14:34] voip uses g711 [14:34] or g722 [14:34] uh - I have voip that goes 729 [14:34] it depends on your clients [14:34] .. http://voip.about.com/od/voipbasics/a/voipcodecs.htm [14:34] yeh some people do voip with g729 [14:34] g729 sucks [14:34] gsm is even worse [14:34] g729 is used in some call centres [14:34] that's why the music sounds terrible [14:35] er [14:35] g729 on *good* bandwidth sounds great [14:35] I'll just let you keep digging a hole for yourself though [14:36] what he said. [14:37] I've been to astricon. I've hung out on vuc.me a number of times. [14:39] I've even had "the asterisk voice lady" make a promo for me for FLOSS Weekly. :) [14:39] I haven't, but I do have vanity commits in Asterisk, for whatever that's worth ;) [14:39] RandalSchwartz: that's pretty cool. [14:41] g729 isn't bad [14:41] but you notice it on music [14:41] gsm is bad [14:41] but g711 is only like 80kbit with sip [14:42] ... http://soundcloud.com/randal-l-schwartz/allison-smith-floss-weekly [14:42] there it is [14:42] mercutio: and when you only run one channel of voice, that's wonderful. [14:43] well if you run lots of channels you can get a e1/t1 cheap anyway [14:43] i use voip over adsl with single channel for my normal claling [14:43] it does depend on the client a bit [14:43] it's pretty good with my linksys phone [14:44] jdoe - have you seen https://www.tropo.com/home.jsp ? [14:44] as long as you don't upload you're sweet without evne having qos [14:44] they have a develop-for-free setup... I have a couple of demo apps :) [14:45] RandalSchwartz: heard of it, I use twilio though. [14:45] unfortunately I don't really have a business use for it, I just somehow scored ~$80 in credit, so I use it for personal things. [14:45] tropo is more flexible, I think [14:46] could be... same idea though, looks like. Programmable voice/sms stuff? [14:46] basically, it binds together any of POTS, VOIP, SMS, and Jabber. [14:46] along with voice recognition and menu systems [14:46] *** solj has left [14:47] ah. Jabber is neat, I don't think twilio does that. [14:47] though nothing prevents you from writing some glue between twilio and jabber, I guess. [14:47] what do you apps do? [14:57] ... http://www.quora.com/What-if-anything-separates-Twilio-from-its-competitors [14:58] that top answer makes me cringe [14:59] well, the third answer down or so is interesting [15:00] tropo seems to be more of a glue [15:00] skype and AIM and twitter I forgot about :) [15:00] the Colin one? yeah. [15:00] wow, that guy's name is almost the same as mine [15:01] RandalSchwartz: yeah that's reasonable. I dunno, I think if that was something I was super concerned about I might just DIY the glue bit... You've got me thinking about gluing twilio to my jabber bot now :P [15:10] ... but right now, I just use it for stupid shit. [15:11] like I can text myself a number, and it calls a script which prods asterisk to find me, call the number, and bridge. [15:11] call (424) 235-1666 [15:12] that was one of the demos, but it's still cute [15:32] *** Ehtyar has joined #arpnetworks [15:32] up_the_irons: any idea why your router might not "see" my node? [15:33] up_the_irons: my ipv6 config is the provided config [15:33] up_the_irons: so I'm not really sure what I'm doing wrong here [16:05]