#arpnetworks 2012-10-11,Thu

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)
WhoWhatWhen
up_the_ironsman, this site is *still* busted: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide [00:01]
jdoeup_the_irons: whichever one my vps is on... sec, lemme dig up my portal info so I can see where I am :P
huh... ipv6 48/64 thing... that's new, could have sworn my /48 was routed before.
up_the_irons: kvr06, about the time I pasted that.
(ditto for yesterday) 		[00:05]
up_the_ironsjdoe: roger that. hrm, kvr06 is a host i haven't touched in a while [00:08]
jdoeyeah, you mentioned that, nobody ever quits.
it's weird. I was frustrated with IO a few weeks ago, so I cron'd that dd+sync to run every 5 minutes and log stats. Problem disappeared. It's only when I'm not waiting for it that I get poor throughput. 		[00:10]
.... (idle for 18mn)
up_the_ironslol "nobody ever quits" [00:30]
....... (idle for 30mn)
another order from an IPv6 address
from GB
it's taking off... 		[01:00]
mercutioup_the_irons: yeh that was beta vm
up_the_irons: when you've got some time i'd like my I/O changed to virtio, and if that doesn't work scsi.
err probably even if it does work, cos it's probably pointful testing it 		[01:09]
up_the_ironsmercutio: done, give it a hard reboot. i change disk and NIC to virtio. [01:12]
mercutiocool
you can't force a hard reboot can you
chrome for some reason doesn't want to let me click on login
i'm confused
err the web page became unresponsive
oh dislabing threaded compisitin gfixed chrome 		[01:12]
up_the_ironsmercutio: i can hard reboot if u want [01:16]
mercutioit's booting now
it looks good so far
but just kernel bit
oh
i should have shutdown inside the vm properly :/
it's fscking
but good sign
it has vic0 now
isntead of em0
i'm making symlink and rebooting
btw is it possible to have more cpu flags passed to the vm?
cos there's something that improves ssh isn't there?
hmm it's about 8mb/sec for dd now
as long as it doesn't stall i don't care too much
(it was about that on the other vps)
and network works too
network speed is similar 		[01:16]
up_the_ironsmercutio: so you have both disk and nic working with virtio?
mercutio: about cpu flags, i don't know of anything specific that makes ssh faster 		[01:33]
mercutioyip
41 seconds to extract baes52.tgz
i thought there was aesni or osmething 		[01:34]
up_the_ironswow nice, good job. was the procedure hard? i don't even think toddf has it working yet (but i might be wrong) [01:35]
mercutioat the moment it's the default qemu base cpu thing
umm well i'm using recent snapshot
which is why i thought i should test scsi
cos it seems that 5.2 won't have the virtio stuff
going to compare against the old vm now
oh it also changes from wd0 to sd0
it's using uuid type stuff by default though so you don't have to do anything
tar vxzpf /root/64bit-base52.tgz 1.89s user 20.59s system 65% cpu 34.232 total
it's faster on the old system
but less cpu% on the new system 		[01:35]
***jnq has joined #arpnetworks [01:40]
mercutiolooks like it's slower for writes
maybe it's doing sync versus async
both file systems have noatime,softdep 		[01:43]
up_the_ironsmercutio: the new system has a really busy cacti VM on it that i should actually migrate somewhere else so we get a better picture of the performance [01:45]
mercutiohmm
there's not constant disk load though is there?
err constant high
iostat -x should tell you what it's like 		[01:45]
up_the_ironsno, it's not constant, but like every 5 minutes, there a load burst [01:46]
mercutioyeh
i'm turning on VIRTIO_DEBUG
i think it tells buffer sizes etc 		[01:46]
up_the_ironsmy kvr26 host
$ iostat -x
Linux 2.6.28-19-server (kvr26) 10/11/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
13.70 0.00 10.84 0.79 0.00 75.01
so less than 1% iowait
but on kvr27 (new box)
avg-cpu: %user %nice %system %iowait %steal %idle
5.24 0.00 2.86 12.15 0.00 79.75
so > 12% 		[01:48]
mercutiooh :) [01:48]
up_the_ironsquite busy [01:48]
mercutiowhat's kvr15 like?
cos that's what i'm testing against 		[01:49]
up_the_ironskvr15:
avg-cpu: %user %nice %system %iowait %steal %idle
8.29 0.00 7.07 0.80 0.00 84.29 		[01:52]
mercutioallocated 45056 byte for virtqueue 0 for I/O request, size 128
using 36864 byte (2304 entries) indirect descriptors
vioblk_attach: ; qsize: 128 seg_max: 18
ahh so similar to kvr26
actually less load than kvr26 		[01:52]
up_the_ironsso if you're finding the i/o the same speed as the old server (but on virtio), then that's actually not bad since kvr27 has 12x the IO load on it already [01:53]
mercutioyour iostat -x doesn't show msec
it shows averages
weird
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.06 0.23 0.76 19.35 7.91 97.64 10.50 0.01 0.66 0.63 0.66 0.05 0.11
oh
its' below it?
that's what i find more useful
that is just my router so super low load
err my router is on sdd too
ssd 		[01:53]
up_the_ironsmercutio: that cuz on my end it shows every individual VM disk as "dm-XX' and I don't wanna paste all that ;) [01:56]
mercutiooh
what about the md1?
or whatever that lvm is on top of 		[01:56]
up_the_ironsah ok, here:
sdb1 28.20 215.77 9.82 38.22 316.84 2032.17 48.89 0.10 2.02 1.09 5.22 		[01:56]
mercutioyip that's high [01:57]
up_the_ironsthat's 15 [01:57]
mercutiowow
see how much higher write load is than read load
actually it's not that high 		[01:57]
up_the_ironshere's kvr27:
sdb 2.09 298.10 2.29 177.67 296.07 3836.57 22.96 5.75 31.96 1.71 30.74 		[01:57]
mercutioyou have two less fields than me
iostat added too fields 		[01:58]
up_the_ironslol [01:59]
mercutioin some ubuntu version [01:59]
up_the_ironshere are the headings:
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 		[01:59]
mercutioand ubuntu takes packages freom debian
ahh
so it's just plain average wait
not split by read/write 		[01:59]
up_the_ironslooks like the await was split into read/write [01:59]
mercutioit has all 3 :) [01:59]
up_the_irons:) [01:59]
mercutiowell having high write average time doesnt' matter too much
compared to read
welll "it depends"
yeah
so according to that there's an average delay of nearly 32 msec on kvr27
and just over 2 msec on kvr15
each request takes about 10 msec 		[01:59]
***Ehtyar has joined #arpnetworks [02:01]
mercutioso it's like queueing behind 3 other requests
mm 		[02:01]
up_the_ironsi c [02:01]
mercutioi'll try extracting that tarball again
see how the numbers compare..
wow
tar vxzpf /root/base52.tgz 2.74s user 6.53s system 42% cpu 21.820 total 		[02:01]
up_the_ironsmunin-graph is currently running and being a usual hog [02:03]
mercutioheh
but i got half the time taken as last time 		[02:03]
up_the_ironsnice :) [02:03]
mercutioi have turned on VIRTIO_DEBUG...
and rebooted
so it shouldn't be cached
and may go slower if aynthing
so yeh benchmarking could be hindered by other activity 		[02:03]
up_the_ironsyeah [02:05]
mercutiomy testzero write is going slow still
104857600 bytes transferred in 36.133 secs (2901970 bytes/sec)
that's 100mb
it's actually slower than last time
so yeh it's prob bouncing up and down a bit
actually where it says 177.67 on that iostat...
that's quite a lot of writes/sec
i suppose it's making a write per graph made
but that's basically the kind of work load where raid 5 will go faster than raid 10
err it could be
it depends how much seeking involved 		[02:05]
mm, it also doesn't seem to be write stalling
ie - you can ssh in while it's busy doing stuff
aesni is in bulldozer cpus
it mostly speeds up ssh and vpns
and supported since openbsd 4.9
and in recent linux openssl
err recent linux kernel, and recent openssl
i can't remember how recent, but probably within the last year
<feature name='aes'/>
in qemu apparently
apparently they don't pass all cpu features through so that you can live migrate to a cpu that doesn't ahve those features 		[02:22]
up_the_ironsyeah [02:30]
mercutioit may make scp faster [02:31]
up_the_ironsi don't see aes in my "virsh capabilities" [02:31]
mercutiobut it probably mostly just reduces cpu usage [02:31]
up_the_ironsboo [02:31]
mercutiooh you're using virsh
can virsh pass through the cpu?
err host cpu 		[02:31]
up_the_ironsi don't believe you can set arbitrary qemu command line params until a later version [02:32]
mercutio'-cpu host' is not something that we intend to support in libvirt, because it is an opaque blob whose semantics cannot be determined, queried, analysed. Furthermore it is completely redundant given the capabilities already present in the libvirt CPU description framework. [02:32]
up_the_ironslibvirt development is fast, so i'll need yet another upgrade before i get some of the newer features [02:32]
mercutiohmm
You can get the equivalent of -cpu host by running 'virsh capabilities' (or the API) to query the current host <cpu> description, and then copying that <cpu> unchanged into the guest XML. At most we could add a syntactic sugar at the XML input stage where <cpu><model>host</model></cpu> gets automatically expanded into the full <cpu> XML from the capabilities.
on dom0 cat /proc/cpuinfo | grep -i aes
shows aes right? 		[02:32]
up_the_ironsyou mean "grep -i aes /proc/cpuinfo" ;)
and yeah, it has aes
brb 		[02:34]
mercutioyeah same diff [02:35]
http://wiki.libvirt.org/page/Libvirt_identifies_host_processor_as_a_different_model_from_the_hardware_documentation
this says libvirt can sometimes see cpu as lower model than it is and lose features, in two cases aes 		[02:40]
...... (idle for 27mn)
***Webhostbudd has quit IRC (Quit: Leaving)
lazard has quit IRC (Ping timeout: 256 seconds) 		[03:07]
up_the_ironsboo [03:12]
***lazard has joined #arpnetworks [03:16]
....... (idle for 34mn)
up_the_ironslol http://i.imgur.com/LZ73H.png [03:50]
easymacheh
Been meaning to mess with clang/llvm, but I'm lazy and have no real demand for C code right now. 		[03:51]
up_the_ironsC was first language I learned after BASIC; it always holds a place in my heart even though I never have a chance to use it anymore [03:59]
easymacheh It was my first language.
I love C, but I never really have demand for it doing network/infrastructure stuff. It's usually easier/faster to script something. 		[04:04]
***Ehtyar has quit IRC (Quit: +++ OK ATH OK) [04:11]
up_the_ironsyeah for sure [04:22]
........ (idle for 35mn)
mike-burnsC was my first language, too, and I've never actually used it. Better than asm, though. [04:57]
easymacSorry we can't all be using your hippy fad languages, mike-burns. heh [05:08]
CaZeCaZe is idling in ##c [05:12]
easymacI find that the more help channels I idle in the more discouraged I am with help channels. [05:12]
...... (idle for 28mn)
dzupi dont help, dont talk, just idle, am a irc bloodsucker leech [05:40]
easymacI help in one of them, the rest are just a waste. [05:41]
mike-burnsI give bad advice in #haskell. [05:49]
easymacJust for fun? [05:51]
mike-burnsYes, and also because I don't know better.
Just like how I give bad directions on the street: it's fun, and I can't give good directions anyway. 		[05:54]
........ (idle for 37mn)
***gcw|mbpro has quit IRC (Remote host closed the connection) [06:31]
........ (idle for 38mn)
heavysixer has quit IRC (Remote host closed the connection) [07:09]
rootpwned has quit IRC (Ping timeout: 246 seconds) [07:16]
easymachaha
fair enough 		[07:19]
***rootpwned has joined #arpnetworks [07:21]
rootpwned has quit IRC (Quit: rootpwned)
chmod has joined #arpnetworks 		[07:32]
.......... (idle for 45mn)
DDevineAh thanks for including my username in the "your debit card is expiring" email. I log in so rarely that I completely forgot. [08:19]
179 days uptime apparently. [08:26]
phlux[phlux@bryant ~]$ uptime
8:32AM up 446 days, 12:40, 3 users, load averages: 0.02, 0.01, 0.00
my arp uptime :P 		[08:32]
............. (idle for 1h3mn)
***toddf has quit IRC (Remote host closed the connection) [09:35]
nukefree has quit IRC (Ping timeout: 246 seconds)
toddf has joined #arpnetworks
ChanServ sets mode: +o toddf
kraigu has quit IRC (Read error: Connection reset by peer)
kraigu has joined #arpnetworks
nukefree has joined #arpnetworks 		[09:42]
........ (idle for 38mn)
heavysixer has joined #arpnetworks
ChanServ sets mode: +o heavysixer 		[10:22]
HighJinx has quit IRC (Quit: Computer has gone to sleep.) [10:32]
........... (idle for 54mn)
toddfheh
I guess I don't have to worry a ton about bandwidth when my _yearly_ bandwidth is 1/5 my monthly allowance ;-) 		[11:26]
***HighJinx has joined #arpnetworks [11:35]
............ (idle for 57mn)
milkio.O [12:32]
.... (idle for 17mn)
***chmod has quit IRC (Ping timeout: 246 seconds)
chmod2 has joined #arpnetworks 		[12:49]
...... (idle for 27mn)
mercutioheh
toddf up_the_irons was saying that you couldn't get virtio to go on openbsd? 		[13:16]
toddfmercutio: not on 'current production' he has, but there is a beta system that boots openbsd just fine [13:16]
mercutiotodd: oh right [13:16]
toddfcaveat is virtio is not on the install media
so you have to install first, then reboot, and never need a bsd.rd
or just wait until someone fixes the install media 		[13:16]
mercutiomm
was that with 5.2 or snapshot? 		[13:17]
toddfthe current production systems lack a bios hook to show the virtio disk as a disk, therefore the bootloader can't see the disk [13:17]
mercutiooh 5.2 isn't out yet is it? [13:17]
toddfcurrent only has virtio, beyond 5.2 [13:17]
mercutioyeh
i have snpashot working on the beta system
with snapshot
but the host has to be set to virtio :(
with xen it's just automatic 		[13:17]
toddfso in theory you could email support@arpnetworks.com and give the uuid of your beta vps and you too can have virtio [13:18]
mercutioi have virtio :) [13:18]
toddfbe aware mv /etc/hostname.em0 /etc/hostname.vio0
well you can use virtio 		[13:18]
mercutioi symlinked it
so now i should be able to boot with either 		[13:18]
toddfif it works [13:18]
mercutioyeh it works
performance was slightly better
not amazing though 		[13:19]
toddfperformance is already amazing [13:19]
mercutiodunno if you did many tests?
i got 8 mb/sec on dd? 		[13:19]
toddfI'm moving my colo this week, will do more thorough testing next week [13:19]
mercutiobut on the new system without virtio there were these weird i/o stalls
and virtio seemed to fix that
apparently there are some disk heavy things on there too though
i suspect it's about read ahead or such
although i'd think dd with bs=1024k would fix that 		[13:19]
.... (idle for 15mn)
jdoe8MB/s is not amazing :P [13:36]
toddfthe linear read is not as impressive as seek times which takes more than a simple dd to measure
I had a system at the prompt because fsck wanted attention 		[13:48]
mercutioyeh [13:48]
toddfI typed 'fsck -y' and about 5 seconds later it was done due to fast seeks [13:49]
mercutiowrite was actually faster
hmm
it's also had "bad" it gets 		[13:49]
like if you get high i/o latency it can be way less than 8mb/sec
on a normal system 		[13:58]
phluxi have some serious permissions issues on my freebsd vps, and i can't think of a way to fix it without just starting over [14:04]
milkio.O
did you do a recursive chmod from /
ive done that before... 		[14:04]
phluxno
but for some reason, web servers can't reach anyone's files without setting them 755 or similar
it doesn't end there, either 		[14:05]
milkisounds like the web server's fault [14:05]
phluxunless i'm doing something as root, i get permission denied for several things
the web server is just the most prevalent 		[14:05]
milkigot a transcript? [14:05]
phluxyeah i was thinking that, too, but i don't understand how
i've tried two different servers (apache and nginx)
neither can get proper access 		[14:05]
milkiyou verified the user the server process runs as? [14:06]
phluxhmm
that may be an issue
nginx:nginx (user:group)..the group doesn't exist in /etc/group 		[14:08]
milki:o
i ownder how that happened 		[14:08]
phluxi figured that would've been taken care of during the installation [14:09]
milkimilki too [14:09]
phluxi wonder how it's even running [14:09]
milkilol
maybe it defaulted to the nobody group 		[14:09]
phluxphlux shrugs
i'll see if this handles it then heh 		[14:11]
.......... (idle for 46mn)
mercutiooh, my initial thought about the 8mb/sec virtio seems to be wrong
rer, or limited, openbsd goes slow if you don't use the raw device, and dd straight from the hard-disk
but if you dd frmo a file on a mounted file system, or the raw device it's fast
use ps auxw?
to check what it's running as
usually on linux files use :www-data
with read permissions
and you own the files by the user hosting the web conent
which has read write
you don't really want your web server being able to write to hosted files normally
as it means if you get any kind of exploit of the web server that people can "hack" your site 		[14:57]
milkiperhaps, but that depends on the server setup
its very explicit in the configuration 		[15:02]
mercutioyeh sometimes people su to the user that owns the files for php i think [15:03]
........ (idle for 39mn)
easymac1/
oops 		[15:42]
up_the_ironsDDevine: yeah, the username in that email was requested a few times with people who have several separate accounts
phlux: i remember there is a prog on freebsd to fix all permissions from / and downward, but i don't remember the name... 		[15:43]
.... (idle for 18mn)
***wallshot has joined #arpnetworks [16:04]
wallshotwhat did i miss? did i trip over the internet power cable and unplug it again?
wallshot wins @ tripping over cables 		[16:05]
***Ehtyar has joined #arpnetworks [16:14]
..... (idle for 22mn)
cmeiklejohn has joined #arpnetworks [16:36]
nixbag has joined #arpnetworks [16:46]
.... (idle for 16mn)
gcw|mbpro has joined #arpnetworks
gcw|mbpro has quit IRC (Remote host closed the connection)
gcw|mbpro has joined #arpnetworks 		[17:02]
........ (idle for 38mn)
jbergstroemnot perhaps very relevant to arpnetwroks (but we do discuss virtualisation every now and then), but here's a pretty good primer from hendrik volkmer on smartos: http://blog.hendrikvolkmer.de/2012/09/07/why-smartos-as-an-openstack-base-os/ [17:42]
wallshotthat's neat, thanks [17:44]
***cmeiklejohn has quit IRC (Quit: ["Textual IRC Client: www.textualapp.com"]) [17:46]
.... (idle for 18mn)
wallshot has quit IRC (Remote host closed the connection) [18:04]
......... (idle for 42mn)
qbitup_the_irons: did you get that redis majobber going? [18:46]
.... (idle for 15mn)
phluxup_the_irons: are you talking about mtree? [19:01]
***HighJinx has quit IRC (Quit: Computer has gone to sleep.)
gcw|mbpro has quit IRC (Remote host closed the connection)
gcw|mbpro has joined #arpnetworks 		[19:01]
warex has joined #arpnetworks [19:09]
warexI'm a new customer. I've selected OpenBSD 5.1 but there is no xbase and x* installed. I remember that in OpenBSD it can just be updated, rebooting from the cd-rom. Is there a way to do it on my virtual computer? [19:11]
mhoranYou can request to have the CD installed for you. Just send a support request. [19:12]
...... (idle for 26mn)
up_the_ironsqbit: negative
phlux: mtree?
warex: yeah, here are the instructions to boot from the CD-ROM: http://support.arpnetworks.com/kb/vps/custom-partitions
[19:38]
phlux17:46:15 @up_the_iron+│ phlux: i remember there is a prog on freebsd to fix all permissions from
│ / and downward, but i don't remember the name...
sounds like you're talking about mtree 		[19:41]
warexgot it thanks! [19:42]
milkioo, i didnt know you could use mtree like that [19:44]
up_the_ironsphlux: oh! ok, yeah maybe it was that
warex: np! 		[19:48]
mercutiowarex: you don't need to boot off cd to install xbase
you can basically download xbase51.tgz then cd / && tar vxzpf /location/xbase51.tgz
you shouldn't need all of X if things if it's just things like the font thing
font library that gd and stuff need 		[19:52]
***gcw|mbpro has quit IRC (Ping timeout: 255 seconds)
gcw|mbpro has joined #arpnetworks 		[19:55]
................ (idle for 1h17mn)
HighJinx has joined #arpnetworks
gcw|mini1 has joined #arpnetworks
mjp has quit IRC (Quit: leaving)
gcw|mbpro has quit IRC (Ping timeout: 248 seconds) 		[21:16]
mjp has joined #arpnetworks
andol has quit IRC (Quit: leaving) 		[21:30]
andol has joined #arpnetworks [21:37]
........ (idle for 36mn)
dzupf i clic in a http://ipv6:ip:adress enable webserver should work right? [22:13]
.... (idle for 18mn)
reardencodeI believe it needs to be http://[ipv6:ip:address]/ [22:31]
dzupi had try http://2001:5c0:1000:b::c7b1/ no luck, can you see me? [22:34]
toddftry with brackets [22:35]
dzupcuz if i try to telnet me ipv6 80 i get apache response
but no over my bwoser, send me to the google search deal 		[22:35]
..... (idle for 24mn)
***Sapote has joined #arpnetworks [22:59]
Sapote has left
gcw|mini1 has quit IRC (Remote host closed the connection)
gcw|mbpro has joined #arpnetworks 		[23:05]
warex has quit IRC (Quit: Page closed) [23:21]
....... (idle for 34mn)
gcw|mini1 has joined #arpnetworks
gcw|mbpro has quit IRC (Ping timeout: 240 seconds) 		[23:55]
↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)