[01:34] *** milki_ has joined #arpnetworks [01:36] *** mkb_ has joined #arpnetworks [01:41] *** nathani has quit IRC (*.net *.split) [01:41] *** mkb has quit IRC (*.net *.split) [01:41] *** mhoran has quit IRC (*.net *.split) [01:41] *** mrsaint_ has quit IRC (*.net *.split) [01:41] *** KILLALLH1MANS01 has quit IRC (*.net *.split) [01:41] *** karstensrage has quit IRC (*.net *.split) [01:41] *** toddf has quit IRC (*.net *.split) [01:41] *** staticsafe has quit IRC (*.net *.split) [01:41] *** milki has quit IRC (*.net *.split) [01:49] *** nathani has joined #arpnetworks [01:49] *** mhoran has joined #arpnetworks [01:49] *** mrsaint_ has joined #arpnetworks [01:49] *** KILLALLH1MANS01 has joined #arpnetworks [01:49] *** karstensrage has joined #arpnetworks [01:49] *** toddf has joined #arpnetworks [01:49] *** staticsafe has joined #arpnetworks [01:49] *** tepper.freenode.net sets mode: +o toddf [04:39] *** fIorz has joined #arpnetworks [04:42] up_the_irons: any chance you could provision the replacement VM in parallel with the old one still running (with temporary addresses, v6 would be sufficient)? the old one is still running fine, so I could restore my backup and then cut over once that's done [04:43] fIorz: should be doable. just tell me which IPv6 I should assign. [04:46] up_the_irons: you mean from my subnet? just take a random one, chances are negligible that you'll hit one that's in use? ;-) [04:48] i suppose you're right ;) [04:48] OK, I have 1 VPS ahead of you, then I'll work on yours [04:50] sure, it's not particularly urgent as the old vm seems to be working just fine so far, and I'll be afk in a moment anyway ... [04:50] ok [04:52] another customer said a similar thing, that the VM was running fine. But we did a ZFS snapshot of all VMs on that host and tried to transfer the data off, and OH MAN, was it corruption city. 'zpool -v status' reports your volume as affected. [04:55] well, it's mostly a tunnel endpoint, openvpn is probably in the pagecache, so chances are you could more or less remove the disk completely and it would still work fine ;-) [04:55] hahaha [04:55] fair enough [04:58] but, yeah, bad luck indeed with two disks failing at the same time ... same manufacturer/manufacturing batch? [05:06] it all came as a set. a bunch of HP disks in a 25-bay server. _really_ weird the failures were from the same raidz group. [05:07] makes me want to put a Ceph cluster out in Frankfurt [05:07] alas the initial server count needed is high [05:28] MIne just froze right now when tried to do ps aux :) [05:28] i'm not surprised [05:29] or it died even [05:29] yeppers. died hehe [05:29] oh well.. wanted to backup bgpd.conf [05:30] *** mrsaint_ is now known as mrsaint [05:31] yeah timeout wait for disk kinda.. kernel outputs. [05:31] want me to reply to email I guess? [05:32] if you feel you're done with it and there's no more saving data, you can just tell me here [05:32] Well I cant save any data as it died :) [05:32] up_the_irons: but if you want you can put it to centos6.7? [05:32] or 6 :) [05:33] hangon.. I just got in again. [05:33] let me rsync what I need. [05:34] done :) [05:37] crap, I don't have the 6.x line in our templates in Frankfurt [05:38] I can work on that later today [05:38] if you don't mind [05:38] ah ok. Np. [05:38] Ill shutdown bgp while its having related issues so i dont run into more diskusage perhaps. [05:41] ok [07:12] up_the_irons: my box is up again now. Thanks. [07:12] mrsaint: great! [07:12] no problem [07:16] up_the_irons: now syncing 5000 domains again :) [07:16] what do you do? :) [07:17] is that for like.. dns.. or? [07:17] think it's time for me to finally sleep... [07:18] g'nite all [08:02] *** mkb_ is now known as mkb [10:01] up_the_irons: yeah dns and webanycasted service. [11:25] mrsaint: Anycast web stuff? How stable do you find the internet for that? Many problems with TCP sessions dying because they change routing? [11:31] plett: works just fine.. static content and varnish that do cache alot. [11:32] and I choose my providers for my servers with care. Specially take them with direct connects to either me or my providers.. [11:39] there's actually a lot less varying with routes than there used to be. [11:40] and a lot of them come from brief outages where web page not loading right sucks anyway [11:40] and people reload [11:40] or go somewhere else [11:42] yeah.. And would be same problem anycasted or not.. Only times I would have connection problem would be when the node cant reach backend and it has not yet happend.. :) [11:43] yeah that seemse to be normal cloudflare issue [11:43] the node can't reach the backend [11:43] (cloudflare are anycasting tcp too) [11:43] yeah [11:44] reddit is the site i most have issues with [11:44] but their backend usually responds but not positively :) [11:44] my own site is dynamic but alot of it is static so works fine anyway. so we see. I only have gamesites and their png/gif content sofar. [11:45] but the dns works lovely really nice.. [11:47] Yeah, anycast dns is a good solution. I'm not sold on doing it for web traffic though :) [11:48] i'm not sold on tcp for web traffic [11:49] with people expecting 2 second load times etc, slow resend etc delays really hurts [11:49] and it'd be better if the client knew about multiple places to source the same data from, and to have good web caches etc [11:50] mercutio: It's fine, just don't live somewhere a long RTT away from the web server ;) [11:50] linux decreasing the initial timeout from 3 seconds to 1 seconds really helps [11:50] but yeah high rtt connections really don't want packet loss [11:51] but then you end up with 1 second+ pings on 3g [11:51] because they don't want to drop packets [11:51] 3G networks here are typically 200ms ping times. 4G are closer to 10 or 20ms [11:52] dc hsdpa or whatever is around 60 to 80 msec [11:52] but often there's some congestion [11:52] i found it went up when there was a power cut.. [11:52] which is when i wanted to use it, and it was 1+ second. [11:52] i tried a few random times afterwards and wasn't nearly as bad [11:53] 4g is here now at least. [11:55] plett: if you have a web you want to try you can try it for free if you wish. Prefer backend at a location where I have good connectivity to tho. To secure uptime. [12:33] *** dj_goku has quit IRC (Ping timeout: 260 seconds) [13:23] aw, got momentarily excited to see that gandi has .nets for $2, but the promo doesn't apply to renewals. :P [13:23] mrsaint: ah cool [13:29] up_the_irons: yeah works well. [13:30] *** dj_goku has joined #arpnetworks [13:30] *** dj_goku has quit IRC (Changing host) [13:30] *** dj_goku has joined #arpnetworks [13:34] nice [13:35] *** dj_goku has quit IRC (Ping timeout: 260 seconds) [14:32] *** dj_goku has joined #arpnetworks [14:32] *** dj_goku has quit IRC (Changing host) [14:32] *** dj_goku has joined #arpnetworks [14:38] *** dj_goku has quit IRC (Ping timeout: 248 seconds) [15:35] *** dj_goku has joined #arpnetworks [15:35] *** dj_goku has quit IRC (Changing host) [15:35] *** dj_goku has joined #arpnetworks [15:40] *** dj_goku has quit IRC (Ping timeout: 265 seconds) [16:02] *** dj_goku has joined #arpnetworks [16:02] *** dj_goku has quit IRC (Changing host) [16:02] *** dj_goku has joined #arpnetworks [16:33] *** mkb has quit IRC (Remote host closed the connection) [16:58] *** KILLALLH1MANS01 has quit IRC (Ping timeout: 249 seconds) [16:59] *** KILLALLHUMANS01 has joined #arpnetworks [16:59] *** KILLALLHUMANS01 has quit IRC (Changing host) [16:59] *** KILLALLHUMANS01 has joined #arpnetworks [16:59] *** KILLALLHUMANS01 has quit IRC (Changing host) [16:59] *** KILLALLHUMANS01 has joined #arpnetworks [17:23] *** nathani has quit IRC (Ping timeout: 241 seconds) [17:24] *** staticsafe has quit IRC (Ping timeout: 240 seconds) [17:24] *** nathani has joined #arpnetworks [17:29] *** staticsafe has joined #arpnetworks [17:52] *** mkb has joined #arpnetworks [18:20] up_the_irons: re ceph: yeah, probably the best solution to that problem--but be careful with the crypto, it's terrible (unless they have redesigned that in the last year or so since when I did a rough review of the code) [18:47] we aren't using the crypto in ceph [18:50] that's probably the right thing to do :-) [18:51] :) [22:52] *** toddf has quit IRC (Ping timeout: 240 seconds) [22:52] *** toddf has joined #arpnetworks [22:52] *** ChanServ sets mode: +o toddf [23:41] *** mnathani has quit IRC (Read error: Connection reset by peer)