man, this site is *still* busted: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide up_the_irons: whichever one my vps is on... sec, lemme dig up my portal info so I can see where I am :P huh... ipv6 48/64 thing... that's new, could have sworn my /48 was routed before. up_the_irons: kvr06, about the time I pasted that. (ditto for yesterday) jdoe: roger that. hrm, kvr06 is a host i haven't touched in a while yeah, you mentioned that, nobody ever quits. it's weird. I was frustrated with IO a few weeks ago, so I cron'd that dd+sync to run every 5 minutes and log stats. Problem disappeared. It's only when I'm not waiting for it that I get poor throughput. lol "nobody ever quits" another order from an IPv6 address from GB it's taking off... up_the_irons: yeh that was beta vm up_the_irons: when you've got some time i'd like my I/O changed to virtio, and if that doesn't work scsi. err probably even if it does work, cos it's probably pointful testing it mercutio: done, give it a hard reboot. i change disk and NIC to virtio. cool you can't force a hard reboot can you chrome for some reason doesn't want to let me click on login i'm confused err the web page became unresponsive oh dislabing threaded compisitin gfixed chrome mercutio: i can hard reboot if u want it's booting now it looks good so far but just kernel bit oh i should have shutdown inside the vm properly :/ it's fscking but good sign it has vic0 now isntead of em0 i'm making symlink and rebooting btw is it possible to have more cpu flags passed to the vm? cos there's something that improves ssh isn't there? hmm it's about 8mb/sec for dd now as long as it doesn't stall i don't care too much (it was about that on the other vps) and network works too network speed is similar mercutio: so you have both disk and nic working with virtio? mercutio: about cpu flags, i don't know of anything specific that makes ssh faster yip 41 seconds to extract baes52.tgz i thought there was aesni or osmething wow nice, good job. was the procedure hard? i don't even think toddf has it working yet (but i might be wrong) at the moment it's the default qemu base cpu thing umm well i'm using recent snapshot which is why i thought i should test scsi cos it seems that 5.2 won't have the virtio stuff going to compare against the old vm now oh it also changes from wd0 to sd0 it's using uuid type stuff by default though so you don't have to do anything tar vxzpf /root/64bit-base52.tgz 1.89s user 20.59s system 65% cpu 34.232 total it's faster on the old system but less cpu% on the new system looks like it's slower for writes maybe it's doing sync versus async both file systems have noatime,softdep mercutio: the new system has a really busy cacti VM on it that i should actually migrate somewhere else so we get a better picture of the performance hmm there's not constant disk load though is there? err constant high iostat -x should tell you what it's like no, it's not constant, but like every 5 minutes, there a load burst yeh i'm turning on VIRTIO_DEBUG i think it tells buffer sizes etc my kvr26 host $ iostat -x Linux 2.6.28-19-server (kvr26) 10/11/2012 _x86_64_ (8 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 13.70 0.00 10.84 0.79 0.00 75.01 so less than 1% iowait but on kvr27 (new box) avg-cpu: %user %nice %system %iowait %steal %idle 5.24 0.00 2.86 12.15 0.00 79.75 so > 12% oh :) quite busy what's kvr15 like? cos that's what i'm testing against kvr15: avg-cpu: %user %nice %system %iowait %steal %idle 8.29 0.00 7.07 0.80 0.00 84.29 allocated 45056 byte for virtqueue 0 for I/O request, size 128 using 36864 byte (2304 entries) indirect descriptors vioblk_attach: ; qsize: 128 seg_max: 18 ahh so similar to kvr26 actually less load than kvr26 so if you're finding the i/o the same speed as the old server (but on virtio), then that's actually not bad since kvr27 has 12x the IO load on it already your iostat -x doesn't show msec it shows averages weird Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.06 0.23 0.76 19.35 7.91 97.64 10.50 0.01 0.66 0.63 0.66 0.05 0.11 oh its' below it? that's what i find more useful that is just my router so super low load err my router is on sdd too ssd mercutio: that cuz on my end it shows every individual VM disk as "dm-XX' and I don't wanna paste all that ;) oh what about the md1? or whatever that lvm is on top of ah ok, here: sdb1 28.20 215.77 9.82 38.22 316.84 2032.17 48.89 0.10 2.02 1.09 5.22 yip that's high that's 15 wow see how much higher write load is than read load actually it's not that high here's kvr27: sdb 2.09 298.10 2.29 177.67 296.07 3836.57 22.96 5.75 31.96 1.71 30.74 you have two less fields than me iostat added too fields lol in some ubuntu version here are the headings: Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util and ubuntu takes packages freom debian ahh so it's just plain average wait not split by read/write looks like the await was split into read/write it has all 3 :) :) well having high write average time doesnt' matter too much compared to read welll "it depends" yeah so according to that there's an average delay of nearly 32 msec on kvr27 and just over 2 msec on kvr15 each request takes about 10 msec so it's like queueing behind 3 other requests mm i c i'll try extracting that tarball again see how the numbers compare.. wow tar vxzpf /root/base52.tgz 2.74s user 6.53s system 42% cpu 21.820 total munin-graph is currently running and being a usual hog heh but i got half the time taken as last time nice :) i have turned on VIRTIO_DEBUG... and rebooted so it shouldn't be cached and may go slower if aynthing so yeh benchmarking could be hindered by other activity yeah my testzero write is going slow still 104857600 bytes transferred in 36.133 secs (2901970 bytes/sec) that's 100mb it's actually slower than last time so yeh it's prob bouncing up and down a bit actually where it says 177.67 on that iostat... that's quite a lot of writes/sec i suppose it's making a write per graph made but that's basically the kind of work load where raid 5 will go faster than raid 10 err it could be it depends how much seeking involved mm, it also doesn't seem to be write stalling ie - you can ssh in while it's busy doing stuff aesni is in bulldozer cpus it mostly speeds up ssh and vpns and supported since openbsd 4.9 and in recent linux openssl err recent linux kernel, and recent openssl i can't remember how recent, but probably within the last year in qemu apparently apparently they don't pass all cpu features through so that you can live migrate to a cpu that doesn't ahve those features yeah it may make scp faster i don't see aes in my "virsh capabilities" but it probably mostly just reduces cpu usage boo oh you're using virsh can virsh pass through the cpu? err host cpu i don't believe you can set arbitrary qemu command line params until a later version '-cpu host' is not something that we intend to support in libvirt, because it is an opaque blob whose semantics cannot be determined, queried, analysed. Furthermore it is completely redundant given the capabilities already present in the libvirt CPU description framework. libvirt development is fast, so i'll need yet another upgrade before i get some of the newer features hmm You can get the equivalent of -cpu host by running 'virsh capabilities' (or the API) to query the current host description, and then copying that unchanged into the guest XML. At most we could add a syntactic sugar at the XML input stage where host gets automatically expanded into the full XML from the capabilities. on dom0 cat /proc/cpuinfo | grep -i aes shows aes right? you mean "grep -i aes /proc/cpuinfo" ;) and yeah, it has aes brb yeah same diff http://wiki.libvirt.org/page/Libvirt_identifies_host_processor_as_a_different_model_from_the_hardware_documentation this says libvirt can sometimes see cpu as lower model than it is and lose features, in two cases aes boo lol http://i.imgur.com/LZ73H.png heh Been meaning to mess with clang/llvm, but I'm lazy and have no real demand for C code right now. C was first language I learned after BASIC; it always holds a place in my heart even though I never have a chance to use it anymore heh It was my first language. I love C, but I never really have demand for it doing network/infrastructure stuff. It's usually easier/faster to script something. yeah for sure C was my first language, too, and I've never actually used it. Better than asm, though. Sorry we can't all be using your hippy fad languages, mike-burns. heh I find that the more help channels I idle in the more discouraged I am with help channels. i dont help, dont talk, just idle, am a irc bloodsucker leech I help in one of them, the rest are just a waste. I give bad advice in #haskell. Just for fun? Yes, and also because I don't know better. Just like how I give bad directions on the street: it's fun, and I can't give good directions anyway. haha fair enough Ah thanks for including my username in the "your debit card is expiring" email. I log in so rarely that I completely forgot. 179 days uptime apparently. [phlux@bryant ~]$ uptime 8:32AM up 446 days, 12:40, 3 users, load averages: 0.02, 0.01, 0.00 my arp uptime :P heh I guess I don't have to worry a ton about bandwidth when my _yearly_ bandwidth is 1/5 my monthly allowance ;-) o.O heh toddf up_the_irons was saying that you couldn't get virtio to go on openbsd? mercutio: not on 'current production' he has, but there is a beta system that boots openbsd just fine todd: oh right caveat is virtio is not on the install media so you have to install first, then reboot, and never need a bsd.rd or just wait until someone fixes the install media mm was that with 5.2 or snapshot? the current production systems lack a bios hook to show the virtio disk as a disk, therefore the bootloader can't see the disk oh 5.2 isn't out yet is it? current only has virtio, beyond 5.2 yeh i have snpashot working on the beta system with snapshot but the host has to be set to virtio :( with xen it's just automatic so in theory you could email support@arpnetworks.com and give the uuid of your beta vps and you too can have virtio i have virtio :) be aware mv /etc/hostname.em0 /etc/hostname.vio0 well you can use virtio i symlinked it so now i should be able to boot with either if it works yeh it works performance was slightly better not amazing though performance is already amazing dunno if you did many tests? i got 8 mb/sec on dd? I'm moving my colo this week, will do more thorough testing next week but on the new system without virtio there were these weird i/o stalls and virtio seemed to fix that apparently there are some disk heavy things on there too though i suspect it's about read ahead or such although i'd think dd with bs=1024k would fix that 8MB/s is not amazing :P the linear read is not as impressive as seek times which takes more than a simple dd to measure I had a system at the prompt because fsck wanted attention yeh I typed 'fsck -y' and about 5 seconds later it was done due to fast seeks write was actually faster hmm it's also had "bad" it gets like if you get high i/o latency it can be way less than 8mb/sec on a normal system i have some serious permissions issues on my freebsd vps, and i can't think of a way to fix it without just starting over o.O did you do a recursive chmod from / ive done that before... no but for some reason, web servers can't reach anyone's files without setting them 755 or similar it doesn't end there, either sounds like the web server's fault unless i'm doing something as root, i get permission denied for several things the web server is just the most prevalent got a transcript? yeah i was thinking that, too, but i don't understand how i've tried two different servers (apache and nginx) neither can get proper access you verified the user the server process runs as? hmm that may be an issue nginx:nginx (user:group)..the group doesn't exist in /etc/group :o i ownder how that happened i figured that would've been taken care of during the installation i wonder how it's even running lol maybe it defaulted to the nobody group i'll see if this handles it then heh oh, my initial thought about the 8mb/sec virtio seems to be wrong rer, or limited, openbsd goes slow if you don't use the raw device, and dd straight from the hard-disk but if you dd frmo a file on a mounted file system, or the raw device it's fast use ps auxw? to check what it's running as usually on linux files use :www-data with read permissions and you own the files by the user hosting the web conent which has read write you don't really want your web server being able to write to hosted files normally as it means if you get any kind of exploit of the web server that people can "hack" your site perhaps, but that depends on the server setup its very explicit in the configuration yeh sometimes people su to the user that owns the files for php i think 1/ oops DDevine: yeah, the username in that email was requested a few times with people who have several separate accounts phlux: i remember there is a prog on freebsd to fix all permissions from / and downward, but i don't remember the name... what did i miss? did i trip over the internet power cable and unplug it again? not perhaps very relevant to arpnetwroks (but we do discuss virtualisation every now and then), but here's a pretty good primer from hendrik volkmer on smartos: http://blog.hendrikvolkmer.de/2012/09/07/why-smartos-as-an-openstack-base-os/ that's neat, thanks up_the_irons: did you get that redis majobber going? up_the_irons: are you talking about mtree? I'm a new customer. I've selected OpenBSD 5.1 but there is no xbase and x* installed. I remember that in OpenBSD it can just be updated, rebooting from the cd-rom. Is there a way to do it on my virtual computer? You can request to have the CD installed for you. Just send a support request. qbit: negative phlux: mtree? warex: yeah, here are the instructions to boot from the CD-ROM: http://support.arpnetworks.com/kb/vps/custom-partitions 17:46:15 @up_the_iron+│ phlux: i remember there is a prog on freebsd to fix all permissions from │ / and downward, but i don't remember the name... sounds like you're talking about mtree got it thanks! oo, i didnt know you could use mtree like that phlux: oh! ok, yeah maybe it was that warex: np! warex: you don't need to boot off cd to install xbase you can basically download xbase51.tgz then cd / && tar vxzpf /location/xbase51.tgz you shouldn't need all of X if things if it's just things like the font thing font library that gd and stuff need f i clic in a http://ipv6:ip:adress enable webserver should work right? I believe it needs to be http://[ipv6:ip:address]/ i had try http://2001:5c0:1000:b::c7b1/ no luck, can you see me? try with brackets cuz if i try to telnet me ipv6 80 i get apache response but no over my bwoser, send me to the google search deal