[00:01] man, this site is *still* busted: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide [00:05] up_the_irons: whichever one my vps is on... sec, lemme dig up my portal info so I can see where I am :P [00:06] huh... ipv6 48/64 thing... that's new, could have sworn my /48 was routed before. [00:07] up_the_irons: kvr06, about the time I pasted that. [00:07] (ditto for yesterday) [00:08] jdoe: roger that. hrm, kvr06 is a host i haven't touched in a while [00:10] yeah, you mentioned that, nobody ever quits. [00:12] it's weird. I was frustrated with IO a few weeks ago, so I cron'd that dd+sync to run every 5 minutes and log stats. Problem disappeared. It's only when I'm not waiting for it that I get poor throughput. [00:30] lol "nobody ever quits" [01:00] another order from an IPv6 address [01:00] from GB [01:00] it's taking off... [01:09] up_the_irons: yeh that was beta vm [01:10] up_the_irons: when you've got some time i'd like my I/O changed to virtio, and if that doesn't work scsi. [01:10] err probably even if it does work, cos it's probably pointful testing it [01:12] mercutio: done, give it a hard reboot. i change disk and NIC to virtio. [01:12] cool [01:13] you can't force a hard reboot can you [01:13] chrome for some reason doesn't want to let me click on login [01:13] i'm confused [01:14] err the web page became unresponsive [01:15] oh dislabing threaded compisitin gfixed chrome [01:16] mercutio: i can hard reboot if u want [01:16] it's booting now [01:16] it looks good so far [01:16] but just kernel bit [01:17] oh [01:17] i should have shutdown inside the vm properly :/ [01:17] it's fscking [01:17] but good sign [01:18] it has vic0 now [01:18] isntead of em0 [01:18] i'm making symlink and rebooting [01:19] btw is it possible to have more cpu flags passed to the vm? [01:19] cos there's something that improves ssh isn't there? [01:21] hmm it's about 8mb/sec for dd now [01:22] as long as it doesn't stall i don't care too much [01:23] (it was about that on the other vps) [01:24] and network works too [01:24] network speed is similar [01:33] mercutio: so you have both disk and nic working with virtio? [01:33] mercutio: about cpu flags, i don't know of anything specific that makes ssh faster [01:34] yip [01:34] 41 seconds to extract baes52.tgz [01:34] i thought there was aesni or osmething [01:35] wow nice, good job. was the procedure hard? i don't even think toddf has it working yet (but i might be wrong) [01:35] at the moment it's the default qemu base cpu thing [01:35] umm well i'm using recent snapshot [01:35] which is why i thought i should test scsi [01:35] cos it seems that 5.2 won't have the virtio stuff [01:36] going to compare against the old vm now [01:37] oh it also changes from wd0 to sd0 [01:39] it's using uuid type stuff by default though so you don't have to do anything [01:39] tar vxzpf /root/64bit-base52.tgz 1.89s user 20.59s system 65% cpu 34.232 total [01:39] it's faster on the old system [01:39] but less cpu% on the new system [01:40] *** jnq has joined #arpnetworks [01:43] looks like it's slower for writes [01:45] maybe it's doing sync versus async [01:45] both file systems have noatime,softdep [01:45] mercutio: the new system has a really busy cacti VM on it that i should actually migrate somewhere else so we get a better picture of the performance [01:45] hmm [01:46] there's not constant disk load though is there? [01:46] err constant high [01:46] iostat -x should tell you what it's like [01:46] no, it's not constant, but like every 5 minutes, there a load burst [01:46] yeh [01:46] i'm turning on VIRTIO_DEBUG [01:47] i think it tells buffer sizes etc [01:48] my kvr26 host [01:48] $ iostat -x [01:48] Linux 2.6.28-19-server (kvr26) 10/11/2012 _x86_64_ (8 CPU) [01:48] avg-cpu: %user %nice %system %iowait %steal %idle [01:48] 13.70 0.00 10.84 0.79 0.00 75.01 [01:48] so less than 1% iowait [01:48] but on kvr27 (new box) [01:48] avg-cpu: %user %nice %system %iowait %steal %idle [01:48] 5.24 0.00 2.86 12.15 0.00 79.75 [01:48] so > 12% [01:48] oh :) [01:48] quite busy [01:49] what's kvr15 like? [01:49] cos that's what i'm testing against [01:52] kvr15: [01:52] avg-cpu: %user %nice %system %iowait %steal %idle [01:52] 8.29 0.00 7.07 0.80 0.00 84.29 [01:52] allocated 45056 byte for virtqueue 0 for I/O request, size 128 [01:52] using 36864 byte (2304 entries) indirect descriptors [01:52] vioblk_attach: ; qsize: 128 seg_max: 18 [01:52] ahh so similar to kvr26 [01:52] actually less load than kvr26 [01:53] so if you're finding the i/o the same speed as the old server (but on virtio), then that's actually not bad since kvr27 has 12x the IO load on it already [01:53] your iostat -x doesn't show msec [01:53] it shows averages [01:53] weird [01:54] Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util [01:54] sda 0.06 0.23 0.76 19.35 7.91 97.64 10.50 0.01 0.66 0.63 0.66 0.05 0.11 [01:54] oh [01:54] its' below it? [01:54] that's what i find more useful [01:54] that is just my router so super low load [01:55] err my router is on sdd too [01:55] ssd [01:56] mercutio: that cuz on my end it shows every individual VM disk as "dm-XX' and I don't wanna paste all that ;) [01:56] oh [01:56] what about the md1? [01:56] or whatever that lvm is on top of [01:56] ah ok, here: [01:56] sdb1 28.20 215.77 9.82 38.22 316.84 2032.17 48.89 0.10 2.02 1.09 5.22 [01:57] yip that's high [01:57] that's 15 [01:57] wow [01:57] see how much higher write load is than read load [01:57] actually it's not that high [01:57] here's kvr27: [01:57] sdb 2.09 298.10 2.29 177.67 296.07 3836.57 22.96 5.75 31.96 1.71 30.74 [01:58] you have two less fields than me [01:58] iostat added too fields [01:59] lol [01:59] in some ubuntu version [01:59] here are the headings: [01:59] Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util [01:59] and ubuntu takes packages freom debian [01:59] ahh [01:59] so it's just plain average wait [01:59] not split by read/write [01:59] looks like the await was split into read/write [01:59] it has all 3 :) [01:59] :) [01:59] well having high write average time doesnt' matter too much [02:00] compared to read [02:00] welll "it depends" [02:00] yeah [02:00] so according to that there's an average delay of nearly 32 msec on kvr27 [02:00] and just over 2 msec on kvr15 [02:01] each request takes about 10 msec [02:01] *** Ehtyar has joined #arpnetworks [02:01] so it's like queueing behind 3 other requests [02:01] mm [02:01] i c [02:01] i'll try extracting that tarball again [02:02] see how the numbers compare.. [02:03] wow [02:03] tar vxzpf /root/base52.tgz 2.74s user 6.53s system 42% cpu 21.820 total [02:03] munin-graph is currently running and being a usual hog [02:03] heh [02:03] but i got half the time taken as last time [02:03] nice :) [02:03] i have turned on VIRTIO_DEBUG... [02:03] and rebooted [02:04] so it shouldn't be cached [02:04] and may go slower if aynthing [02:04] so yeh benchmarking could be hindered by other activity [02:05] yeah [02:05] my testzero write is going slow still [02:06] 104857600 bytes transferred in 36.133 secs (2901970 bytes/sec) [02:06] that's 100mb [02:06] it's actually slower than last time [02:06] so yeh it's prob bouncing up and down a bit [02:07] actually where it says 177.67 on that iostat... [02:07] that's quite a lot of writes/sec [02:09] i suppose it's making a write per graph made [02:10] but that's basically the kind of work load where raid 5 will go faster than raid 10 [02:10] err it could be [02:10] it depends how much seeking involved [02:22] mm, it also doesn't seem to be write stalling [02:22] ie - you can ssh in while it's busy doing stuff [02:23] aesni is in bulldozer cpus [02:24] it mostly speeds up ssh and vpns [02:24] and supported since openbsd 4.9 [02:24] and in recent linux openssl [02:24] err recent linux kernel, and recent openssl [02:25] i can't remember how recent, but probably within the last year [02:27] [02:27] in qemu apparently [02:30] apparently they don't pass all cpu features through so that you can live migrate to a cpu that doesn't ahve those features [02:30] yeah [02:31] it may make scp faster [02:31] i don't see aes in my "virsh capabilities" [02:31] but it probably mostly just reduces cpu usage [02:31] boo [02:31] oh you're using virsh [02:32] can virsh pass through the cpu? [02:32] err host cpu [02:32] i don't believe you can set arbitrary qemu command line params until a later version [02:32] '-cpu host' is not something that we intend to support in libvirt, because it is an opaque blob whose semantics cannot be determined, queried, analysed. Furthermore it is completely redundant given the capabilities already present in the libvirt CPU description framework. [02:32] libvirt development is fast, so i'll need yet another upgrade before i get some of the newer features [02:32] hmm [02:33] You can get the equivalent of -cpu host by running 'virsh capabilities' (or the API) to query the current host description, and then copying that unchanged into the guest XML. At most we could add a syntactic sugar at the XML input stage where host gets automatically expanded into the full XML from the capabilities. [02:33] on dom0 cat /proc/cpuinfo | grep -i aes [02:33] shows aes right? [02:34] you mean "grep -i aes /proc/cpuinfo" ;) [02:34] and yeah, it has aes [02:35] brb [02:35] yeah same diff [02:40] http://wiki.libvirt.org/page/Libvirt_identifies_host_processor_as_a_different_model_from_the_hardware_documentation [02:40] this says libvirt can sometimes see cpu as lower model than it is and lose features, in two cases aes [03:07] *** Webhostbudd has quit IRC (Quit: Leaving) [03:07] *** lazard has quit IRC (Ping timeout: 256 seconds) [03:12] boo [03:16] *** lazard has joined #arpnetworks [03:50] lol http://i.imgur.com/LZ73H.png [03:51] heh [03:52] Been meaning to mess with clang/llvm, but I'm lazy and have no real demand for C code right now. [03:59] C was first language I learned after BASIC; it always holds a place in my heart even though I never have a chance to use it anymore [04:04] heh It was my first language. [04:05] I love C, but I never really have demand for it doing network/infrastructure stuff. It's usually easier/faster to script something. [04:11] *** Ehtyar has quit IRC (Quit: +++ OK ATH OK) [04:22] yeah for sure [04:57] C was my first language, too, and I've never actually used it. Better than asm, though. [05:08] Sorry we can't all be using your hippy fad languages, mike-burns. heh [05:12] * CaZe is idling in ##c [05:12] I find that the more help channels I idle in the more discouraged I am with help channels. [05:40] i dont help, dont talk, just idle, am a irc bloodsucker leech [05:41] I help in one of them, the rest are just a waste. [05:49] I give bad advice in #haskell. [05:51] Just for fun? [05:54] Yes, and also because I don't know better. [05:54] Just like how I give bad directions on the street: it's fun, and I can't give good directions anyway. [06:31] *** gcw|mbpro has quit IRC (Remote host closed the connection) [07:09] *** heavysixer has quit IRC (Remote host closed the connection) [07:16] *** root\pwned has quit IRC (Ping timeout: 246 seconds) [07:19] haha [07:19] fair enough [07:21] *** root\pwned has joined #arpnetworks [07:32] *** root\pwned has quit IRC (Quit: root\pwned) [07:34] *** chmod\ has joined #arpnetworks [08:19] Ah thanks for including my username in the "your debit card is expiring" email. I log in so rarely that I completely forgot. [08:26] 179 days uptime apparently. [08:32] [phlux@bryant ~]$ uptime [08:32] 8:32AM up 446 days, 12:40, 3 users, load averages: 0.02, 0.01, 0.00 [08:32] my arp uptime :P [09:35] *** toddf has quit IRC (Remote host closed the connection) [09:42] *** nukefree has quit IRC (Ping timeout: 246 seconds) [09:42] *** toddf has joined #arpnetworks [09:42] *** ChanServ sets mode: +o toddf [09:42] *** kraigu has quit IRC (Read error: Connection reset by peer) [09:43] *** kraigu has joined #arpnetworks [09:44] *** nukefree has joined #arpnetworks [10:22] *** heavysixer has joined #arpnetworks [10:22] *** ChanServ sets mode: +o heavysixer [10:32] *** HighJinx has quit IRC (Quit: Computer has gone to sleep.) [11:26] heh [11:26] I guess I don't have to worry a ton about bandwidth when my _yearly_ bandwidth is 1/5 my monthly allowance ;-) [11:35] *** HighJinx has joined #arpnetworks [12:32] o.O [12:49] *** chmod\ has quit IRC (Ping timeout: 246 seconds) [12:49] *** chmod\2 has joined #arpnetworks [13:16] heh [13:16] toddf up_the_irons was saying that you couldn't get virtio to go on openbsd? [13:16] mercutio: not on 'current production' he has, but there is a beta system that boots openbsd just fine [13:16] todd: oh right [13:16] caveat is virtio is not on the install media [13:17] so you have to install first, then reboot, and never need a bsd.rd [13:17] or just wait until someone fixes the install media [13:17] mm [13:17] was that with 5.2 or snapshot? [13:17] the current production systems lack a bios hook to show the virtio disk as a disk, therefore the bootloader can't see the disk [13:17] oh 5.2 isn't out yet is it? [13:17] current only has virtio, beyond 5.2 [13:17] yeh [13:17] i have snpashot working on the beta system [13:17] with snapshot [13:18] but the host has to be set to virtio :( [13:18] with xen it's just automatic [13:18] so in theory you could email support@arpnetworks.com and give the uuid of your beta vps and you too can have virtio [13:18] i have virtio :) [13:18] be aware mv /etc/hostname.em0 /etc/hostname.vio0 [13:18] well you can use virtio [13:18] i symlinked it [13:18] so now i should be able to boot with either [13:18] if it works [13:19] yeh it works [13:19] performance was slightly better [13:19] not amazing though [13:19] performance is already amazing [13:19] dunno if you did many tests? [13:19] i got 8 mb/sec on dd? [13:19] I'm moving my colo this week, will do more thorough testing next week [13:19] but on the new system without virtio there were these weird i/o stalls [13:20] and virtio seemed to fix that [13:20] apparently there are some disk heavy things on there too though [13:21] i suspect it's about read ahead or such [13:21] although i'd think dd with bs=1024k would fix that [13:36] 8MB/s is not amazing :P [13:48] the linear read is not as impressive as seek times which takes more than a simple dd to measure [13:48] I had a system at the prompt because fsck wanted attention [13:48] yeh [13:49] I typed 'fsck -y' and about 5 seconds later it was done due to fast seeks [13:49] write was actually faster [13:49] hmm [13:49] it's also had "bad" it gets [13:58] like if you get high i/o latency it can be way less than 8mb/sec [13:58] on a normal system [14:04] i have some serious permissions issues on my freebsd vps, and i can't think of a way to fix it without just starting over [14:04] o.O [14:04] did you do a recursive chmod from / [14:04] ive done that before... [14:05] no [14:05] but for some reason, web servers can't reach anyone's files without setting them 755 or similar [14:05] it doesn't end there, either [14:05] sounds like the web server's fault [14:05] unless i'm doing something as root, i get permission denied for several things [14:05] the web server is just the most prevalent [14:05] got a transcript? [14:05] yeah i was thinking that, too, but i don't understand how [14:06] i've tried two different servers (apache and nginx) [14:06] neither can get proper access [14:06] you verified the user the server process runs as? [14:08] hmm [14:08] that may be an issue [14:08] nginx:nginx (user:group)..the group doesn't exist in /etc/group [14:08] :o [14:09] i ownder how that happened [14:09] i figured that would've been taken care of during the installation [14:09] * milki too [14:09] i wonder how it's even running [14:09] lol [14:09] maybe it defaulted to the nobody group [14:11] * phlux shrugs [14:11] i'll see if this handles it then heh [14:57] oh, my initial thought about the 8mb/sec virtio seems to be wrong [14:58] rer, or limited, openbsd goes slow if you don't use the raw device, and dd straight from the hard-disk [14:58] but if you dd frmo a file on a mounted file system, or the raw device it's fast [14:59] use ps auxw? [14:59] to check what it's running as [14:59] usually on linux files use :www-data [14:59] with read permissions [14:59] and you own the files by the user hosting the web conent [15:00] which has read write [15:00] you don't really want your web server being able to write to hosted files normally [15:00] as it means if you get any kind of exploit of the web server that people can "hack" your site [15:02] perhaps, but that depends on the server setup [15:02] its very explicit in the configuration [15:03] yeh sometimes people su to the user that owns the files for php i think [15:42] 1/ [15:42] oops [15:43] DDevine: yeah, the username in that email was requested a few times with people who have several separate accounts [15:46] phlux: i remember there is a prog on freebsd to fix all permissions from / and downward, but i don't remember the name... [16:04] *** wallshot has joined #arpnetworks [16:05] what did i miss? did i trip over the internet power cable and unplug it again? [16:05] * wallshot wins @ tripping over cables [16:14] *** Ehtyar has joined #arpnetworks [16:36] *** cmeiklejohn has joined #arpnetworks [16:46] *** nixbag has joined #arpnetworks [17:02] *** gcw|mbpro has joined #arpnetworks [17:03] *** gcw|mbpro has quit IRC (Remote host closed the connection) [17:04] *** gcw|mbpro has joined #arpnetworks [17:42] not perhaps very relevant to arpnetwroks (but we do discuss virtualisation every now and then), but here's a pretty good primer from hendrik volkmer on smartos: http://blog.hendrikvolkmer.de/2012/09/07/why-smartos-as-an-openstack-base-os/ [17:44] that's neat, thanks [17:46] *** cmeiklejohn has quit IRC (Quit: ["Textual IRC Client: www.textualapp.com"]) [18:04] *** wallshot has quit IRC (Remote host closed the connection) [18:46] up_the_irons: did you get that redis majobber going? [19:01] up_the_irons: are you talking about mtree? [19:01] *** HighJinx has quit IRC (Quit: Computer has gone to sleep.) [19:03] *** gcw|mbpro has quit IRC (Remote host closed the connection) [19:03] *** gcw|mbpro has joined #arpnetworks [19:09] *** warex has joined #arpnetworks [19:11] I'm a new customer. I've selected OpenBSD 5.1 but there is no xbase and x* installed. I remember that in OpenBSD it can just be updated, rebooting from the cd-rom. Is there a way to do it on my virtual computer? [19:12] You can request to have the CD installed for you. Just send a support request. [19:38] qbit: negative [19:38] phlux: mtree? [19:39] warex: yeah, here are the instructions to boot from the CD-ROM: http://support.arpnetworks.com/kb/vps/custom-partitions [19:41] 17:46:15 @up_the_iron+│ phlux: i remember there is a prog on freebsd to fix all permissions from [19:41] │ / and downward, but i don't remember the name... [19:41] sounds like you're talking about mtree [19:42] got it thanks! [19:44] oo, i didnt know you could use mtree like that [19:48] phlux: oh! ok, yeah maybe it was that [19:48] warex: np! [19:52] warex: you don't need to boot off cd to install xbase [19:53] you can basically download xbase51.tgz then cd / && tar vxzpf /location/xbase51.tgz [19:54] you shouldn't need all of X if things if it's just things like the font thing [19:54] font library that gd and stuff need [19:55] *** gcw|mbpro has quit IRC (Ping timeout: 255 seconds) [19:59] *** gcw|mbpro has joined #arpnetworks [21:16] *** HighJinx has joined #arpnetworks [21:17] *** gcw|mini1 has joined #arpnetworks [21:18] *** mjp has quit IRC (Quit: leaving) [21:21] *** gcw|mbpro has quit IRC (Ping timeout: 248 seconds) [21:30] *** mjp has joined #arpnetworks [21:31] *** andol has quit IRC (Quit: leaving) [21:37] *** andol has joined #arpnetworks [22:13] f i clic in a http://ipv6:ip:adress enable webserver should work right? [22:31] I believe it needs to be http://[ipv6:ip:address]/ [22:34] i had try http://2001:5c0:1000:b::c7b1/ no luck, can you see me? [22:35] try with brackets [22:35] cuz if i try to telnet me ipv6 80 i get apache response [22:35] but no over my bwoser, send me to the google search deal [22:59] *** Sapote has joined #arpnetworks [23:05] *** Sapote has left [23:09] *** gcw|mini1 has quit IRC (Remote host closed the connection) [23:10] *** gcw|mbpro has joined #arpnetworks [23:21] *** warex has quit IRC (Quit: Page closed) [23:55] *** gcw|mini1 has joined #arpnetworks [23:58] *** gcw|mbpro has quit IRC (Ping timeout: 240 seconds)