man, this site is *still* busted: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
up_the_irons: whichever one my vps is on... sec, lemme dig up my portal info so I can see where I am :P
huh... ipv6 48/64 thing... that's new, could have sworn my /48 was routed before.
up_the_irons: kvr06, about the time I pasted that.
(ditto for yesterday)
jdoe: roger that.  hrm, kvr06 is a host i haven't touched in a while
yeah, you mentioned that, nobody ever quits.
it's weird. I was frustrated with IO a few weeks ago, so I cron'd that dd+sync to run every 5 minutes and log stats. Problem disappeared. It's only when I'm not waiting for it that I get poor throughput.
lol "nobody ever quits"
another order from an IPv6 address
from GB
it's taking off...
up_the_irons: yeh that was beta vm
up_the_irons: when you've got some time i'd like my I/O changed to virtio, and if that doesn't work scsi.
err probably even if it does work, cos it's probably pointful testing it
mercutio: done, give it a hard reboot.  i change disk and NIC to virtio.
cool
you can't force a hard reboot can you
chrome for some reason doesn't want to let me click on login
i'm confused
err the web page became unresponsive
oh dislabing threaded compisitin gfixed chrome
mercutio: i can hard reboot if u want
it's booting now
it looks good so far
but just kernel bit
oh
i should have shutdown inside the vm properly :/
it's fscking
but good sign
it has vic0 now
isntead of em0
i'm making symlink and rebooting
btw is it possible to have more cpu flags passed to the vm?
cos there's something that improves ssh isn't there?
hmm it's about 8mb/sec for dd now
as long as it doesn't stall i don't care too much
(it was about that on the other vps)
and network works too
network speed is similar
mercutio: so you have both disk and nic working with virtio?
mercutio: about cpu flags, i don't know of anything specific that makes ssh faster
yip
41 seconds to extract baes52.tgz
i thought there was aesni or osmething
wow nice, good job.  was the procedure hard?  i don't even think toddf has it working yet (but i might be wrong)
at the moment it's the default qemu base cpu thing
umm well i'm using recent snapshot
which is why i thought i should test scsi
cos it seems that 5.2 won't have the virtio stuff
going to compare against the old vm now
oh it also changes from wd0 to sd0
it's using uuid type stuff by default though so you don't have to do anything
tar vxzpf /root/64bit-base52.tgz  1.89s user 20.59s system 65% cpu 34.232 total
it's faster on the old system
but less cpu% on the new system
looks like it's slower for writes
maybe it's doing sync versus async
both file systems have noatime,softdep
mercutio: the new system has a really busy cacti VM on it that i should actually migrate somewhere else so we get a better picture of the performance
hmm
there's not constant disk load though is there?
err constant high
iostat -x should tell you what it's like
no, it's not constant, but like every 5 minutes, there a load burst
yeh
i'm turning on VIRTIO_DEBUG
i think it tells buffer sizes etc
my kvr26 host
$ iostat -x
Linux 2.6.28-19-server (kvr26)  10/11/2012      _x86_64_        (8 CPU)
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          13.70    0.00   10.84    0.79    0.00   75.01
so less than 1% iowait
but on kvr27 (new box)
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.24    0.00    2.86   12.15    0.00   79.75
so > 12%
oh :)
quite busy
what's kvr15 like?
cos that's what i'm testing against
kvr15:
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           8.29    0.00    7.07    0.80    0.00   84.29
allocated 45056 byte for virtqueue 0 for I/O request, size 128
using 36864 byte (2304 entries) indirect descriptors
vioblk_attach: ; qsize: 128 seg_max: 18
ahh so similar to kvr26
actually less load than kvr26
so if you're finding the i/o the same speed as the old server (but on virtio), then that's actually not bad since kvr27 has 12x the IO load on it already
your iostat -x doesn't show msec
it shows averages
weird
Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sda               0.06     0.23    0.76   19.35     7.91    97.64    10.50     0.01    0.66    0.63    0.66   0.05   0.11
oh
its' below it?
that's what i find more useful
that is just my router so super low load
err my router is on sdd too
ssd
mercutio: that cuz on my end it shows every individual VM disk as "dm-XX' and I don't wanna paste all that ;)
oh
what about the md1?
or whatever that lvm is on top of
ah ok, here:
sdb1             28.20   215.77    9.82   38.22   316.84  2032.17    48.89     0.10    2.02   1.09   5.22
yip that's high
that's 15
wow
see how much higher write load is than read load
actually it's not that high
here's kvr27:
sdb               2.09   298.10    2.29  177.67   296.07  3836.57    22.96     5.75   31.96   1.71  30.74
you have two less fields than me
iostat added too fields
lol
in some ubuntu version
here are the headings:
Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
and ubuntu takes packages freom debian
ahh
so it's just plain average wait
not split by read/write
looks like the await was split into read/write
it has all 3 :)
:)
well having high write average time doesnt' matter too much
compared to read
welll "it depends"
yeah
so according to that there's an average delay of nearly 32 msec on kvr27
and just over 2 msec on kvr15
each request takes about 10 msec
so it's like queueing behind 3 other requests
mm
i c
i'll try extracting that tarball again
see how the numbers compare..
wow
tar vxzpf /root/base52.tgz  2.74s user 6.53s system 42% cpu 21.820 total
munin-graph is currently running and being a usual hog
heh
but i got half the time taken as last time
nice :)
i have turned on VIRTIO_DEBUG...
and rebooted
so it shouldn't be cached
and may go slower if aynthing
so yeh benchmarking could be hindered by other activity
yeah
my testzero write is going slow still
104857600 bytes transferred in 36.133 secs (2901970 bytes/sec)
that's 100mb
it's actually slower than last time
so yeh it's prob bouncing up and down a bit
actually where it says 177.67 on that iostat...
that's quite a lot of writes/sec
i suppose it's making a write per graph made
but that's basically the kind of work load where raid 5 will go faster than raid 10
err it could be
it depends how much seeking involved
mm, it also doesn't seem to be write stalling
ie - you can ssh in while it's busy doing stuff
aesni is in bulldozer cpus
it mostly speeds up ssh and vpns
and supported since openbsd 4.9
and in recent linux openssl
err recent linux kernel, and recent openssl
i can't remember how recent, but probably within the last year
<feature name='aes'/>
in qemu apparently
apparently they don't pass all cpu features through so that you can live migrate to a cpu that doesn't ahve those features
yeah
it may make scp faster
i don't see aes in my "virsh capabilities"
but it probably mostly just reduces cpu usage
boo
oh you're using virsh
can virsh pass through the cpu?
err host cpu
i don't believe you can set arbitrary qemu command line params until a later version
'-cpu host' is not something that we intend to support in libvirt, because it is an opaque blob whose semantics cannot be determined, queried, analysed. Furthermore it is completely redundant given the capabilities already present in the libvirt CPU description framework.
libvirt development is fast, so i'll need yet another upgrade before i get some of the newer features
hmm
You can get the equivalent of -cpu host by running 'virsh capabilities' (or the API) to query the current host <cpu> description, and then copying that <cpu> unchanged into the guest XML.  At most we could add a syntactic sugar at the XML input stage where <cpu><model>host</model></cpu> gets automatically expanded into the full <cpu> XML from the capabilities.
on dom0 cat /proc/cpuinfo | grep -i aes
shows aes right?
you mean "grep -i aes /proc/cpuinfo" ;)
and yeah, it has aes
brb
yeah same diff
http://wiki.libvirt.org/page/Libvirt_identifies_host_processor_as_a_different_model_from_the_hardware_documentation
this says libvirt can sometimes see cpu as lower model than it is and lose features, in two cases aes
boo
lol http://i.imgur.com/LZ73H.png
heh
Been meaning to mess with clang/llvm, but I'm lazy and have no real demand for C code right now.
C was first language I learned after BASIC; it always holds a  place in my heart even though I never have a chance to use it anymore
heh It was my first language.
I love C, but I never really have demand for it doing network/infrastructure stuff. It's usually easier/faster to script something.
yeah for sure
C was my first language, too, and I've never actually used it. Better than asm, though.
Sorry we can't all be using your hippy fad languages, mike-burns. heh
I find that the more help channels I idle in the more discouraged I am with help channels.
i dont help, dont talk, just idle, am a irc  bloodsucker leech
I help in one of them, the rest are just a waste.
I give bad advice in #haskell.
Just for fun?
Yes, and also because I don't know better.
Just like how I give bad directions on the street: it's fun, and I can't give good directions anyway.
haha
fair enough
Ah thanks for including my username in the "your debit card is expiring" email. I log in so rarely that I completely forgot.
179 days uptime apparently.
[phlux@bryant ~]$ uptime
 8:32AM  up 446 days, 12:40, 3 users, load averages: 0.02, 0.01, 0.00
my arp uptime :P
heh
I guess I don't have to worry a ton about bandwidth when my _yearly_ bandwidth is 1/5 my monthly allowance ;-)
o.O
heh
toddf up_the_irons was saying that you couldn't get virtio to go on openbsd?
mercutio: not on 'current production' he has, but there is a beta system that boots openbsd just fine
todd: oh right
caveat is virtio is not on the install media
so you have to install first, then reboot, and never need a bsd.rd
or just wait until someone fixes the install media
mm
was that with 5.2 or snapshot?
the current production systems lack a bios hook to show the virtio disk as a disk, therefore the bootloader can't see the disk
oh 5.2 isn't out yet is it?
current only has virtio, beyond 5.2
yeh
i have snpashot working on the beta system
with snapshot
but the host has to be set to virtio :(
with xen it's just automatic
so in theory you could email support@arpnetworks.com and give the uuid of your beta vps and you too can have virtio
i have virtio :)
be aware mv /etc/hostname.em0 /etc/hostname.vio0
well you can use virtio
i symlinked it
so now i should be able to boot with either
if it works
yeh it works
performance was slightly better
not amazing though
performance is already amazing
dunno if you did many tests?
i got 8 mb/sec on dd?
I'm moving my colo this week, will do more thorough testing next week
but on the new system without virtio there were these weird i/o stalls
and virtio seemed to fix that
apparently there are some disk heavy things on there too though
i suspect it's about read ahead or such
although i'd think dd with bs=1024k would fix that
8MB/s is not amazing :P
the linear read is not as impressive as seek times which takes more than a simple dd to measure
I had a system at the prompt because fsck wanted attention
yeh
I typed 'fsck -y' and about 5 seconds later it was done due to fast seeks
write was actually faster
hmm
it's also had "bad" it gets
like if you get high i/o latency it can be way less than 8mb/sec
on a normal system
i have some serious permissions issues on my freebsd vps, and i can't think of a way to fix it without just starting over
o.O
did you do a recursive chmod from /
ive done that before...
no
but for some reason, web servers can't reach anyone's files without setting them 755 or similar
it doesn't end there, either
sounds like the web server's fault
unless i'm doing something as root, i get permission denied for several things
the web server is just the most prevalent
got a transcript?
yeah i was thinking that, too, but i don't understand how
i've tried two different servers (apache and nginx)
neither can get proper access
you verified the user the server process runs as?
hmm
that may be an issue
nginx:nginx (user:group)..the group doesn't exist in /etc/group
:o
i ownder how that happened
i figured that would've been taken care of during the installation
i wonder how it's even running
lol
maybe it defaulted to the nobody group
i'll see if this handles it then heh
oh, my initial thought about the 8mb/sec virtio seems to be wrong
rer, or limited, openbsd goes slow if you don't use the raw device, and dd straight from the hard-disk
but if you dd frmo a file on a mounted file system, or the raw device it's fast
use ps auxw?
to check what it's running as
usually on linux files use :www-data
with read permissions
and you own the files by the user hosting the web conent
which has read write
you don't really want your web server being able to write to hosted files normally
as it means if you get any kind of exploit of the web server that people can "hack" your site
perhaps, but that depends on the server setup
its very explicit in the configuration
yeh sometimes people su to the user that owns the files for php i think
1/
oops
DDevine: yeah, the username in that email was requested a few times with people who have several separate accounts
phlux: i remember there is a prog on freebsd to fix all permissions from / and downward, but i don't remember the name...
what did i miss?  did i trip over the internet power cable and unplug it again?
not perhaps very relevant to arpnetwroks (but we do discuss virtualisation every now and then), but here's a pretty good primer from hendrik volkmer on smartos: http://blog.hendrikvolkmer.de/2012/09/07/why-smartos-as-an-openstack-base-os/
that's neat, thanks
up_the_irons: did you get that redis majobber going?
up_the_irons: are you talking about mtree?
I'm a new customer. I've selected OpenBSD 5.1 but there is no xbase and x* installed. I remember that in OpenBSD it can just be updated, rebooting from the cd-rom. Is there a way to do it on my virtual computer?
You can request to have the CD installed for you. Just send a support request.
qbit: negative
phlux: mtree?
warex: yeah, here are the instructions to boot from the CD-ROM: http://support.arpnetworks.com/kb/vps/custom-partitions
17:46:15 @up_the_iron+│ phlux: i remember there is a prog on freebsd to fix all permissions from
                      │ / and downward, but i don't remember the name...
sounds like you're talking about mtree
got it thanks!
oo, i didnt know you could use mtree like that
phlux: oh! ok, yeah maybe it was that
warex: np!
warex: you don't need to boot off cd to install xbase
you can basically download xbase51.tgz then cd / && tar vxzpf /location/xbase51.tgz
you shouldn't need all of X if things if it's just things like the font thing
font library that gd and stuff need
f i clic in a http://ipv6:ip:adress enable webserver  should work right?
I believe it needs to be http://[ipv6:ip:address]/
i had try http://2001:5c0:1000:b::c7b1/ no luck, can you see me?
try with brackets
cuz if i try to telnet me ipv6 80 i get apache response
but no over my bwoser, send me to the  google search deal