jlgaddis: no web frontend to powerdns, it just powers my Reverse DNS manager jdoe: mercutio : the comparable Intel stuff is sexy too, but way expensive up_the_irons: do the new cpus support bulldozer? err i mean are they bulldozer i actually have no ideahow much cpu ssh would be taking up but i like the idea of it being accelerated :0 mercutio: they are bulldozer, yes. would ssh even take advantage of the hardware aes-ni? like, how would it know it's there.. probably need a new version of ssh if it's anything like the old VIA AES instructions it just requires the right openssl setup for most things to use it ah cool 'openssl engine' up_the_irons: openssl needs to be up to date lt: ubuntu doesn't show it in openssl engine now do you have one of those cpus now? openssl speed -evp aes-256-cbc you want to run that command type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 202878.63k 208678.91k 206311.52k 225509.62k 222816.90k if it's anything like as high as that it's accelerated if it's more like: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 73896.75k 74157.50k 75189.17k 75321.54k 75582.84k then it's not hmm rhel stilll shows it in engine aes-256-cbc 614184.36k 684397.85k 696434.09k 699525.12k 701133.83k that's accelerated numbers :) actually i wonder if that first one is accelerating properly type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 164786.10k 183756.50k 189753.15k 190754.71k 191059.91k cos that's the number of non accelerated core 2 duo actually mine wasn't accelerated... really? how come? options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) # openssl engine (rsax) RSAX engine support (dynamic) Dynamic engine loading support is there something else that has to be done too? hmm... you have me wondering now the first number is xeon e5420 actually xeon E5345 model name : Intel(R) Xeon(R) CPU E5345 @ 2.33GHz and you''re like nearly 10 as fast type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 134099.06k 152144.10k 157358.49k 158168.27k 157409.28k e5420 is quite a bit faster actually i wonder if i'm accelerated properly https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1001424 type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 213802.95k 687449.25k 1094622.32k 1083138.24k 1088128.52k oh yeah it is accelerating, I thought I could trick it by setting -engine to something else but it seems to load the engine anyway it's a lot quicker when you pass -decrypt -decrypt time decryption instead of encryption (only EVP). The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 59636.47k 41641.90k 141532.70k 150321.79k 152577.39k but e5345 still sucks at decrypt aes-256-cbc 613619.75k 1452257.90k 1860791.98k 1988995.41k 2028642.30k lt: what cpu is that? X5690 hmm oh that's really high ghz isn't it? 3.47ghz ? versus 2 ghz it's the first intel series to support aes-ni i think wow those things are expensive :/ yes... but useful if you're stuck running something single threaded like dos? the new e5s latency reduction thing sounds intersting i suppose it'll come into desktops soon opteron 6276 seems similar speed to x5650 e5-2690 is like 50% faster but e5-2690 is like what the recent version of what you're using ie: expensive :/ it's also power hungry http://www.anandtech.com/print/5553 reading that what opteron 6276 is expensive too :/ oh it's cheaper than X5670 and X5690 isn't even sold here :/ the latest new thing always is expensive - we tend to wait until the next new thing is out then buy up the best of the last generation on the cheap ahh i see i prefer lower spec newer stuff generally but end up with lower spec older stuff :/ the e5-2620 is actually pretty good value and all the opterons seem to be branded hp up_the_irons: what cpus are in the kvm nodes atm? kvm hides the cpu type it says 2.641 ghz but that's about as much x3450? i'm guessing it's 2.66ghz or e5430 or e5150 you have to tell ssh to use aes otherwise it defaults to blowfish without acceleration aes256-cbc and blowfish are about the same speed and aes128-cbc is faster and ptuty defaults to aes rater than blowfish openssh defaults to aes128... does it? aes128-cbc? Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc i actually foudn this commented out blowfish isn't even in that last? but if i change that first entry to be aes128-cbc ssh to localhost with dd'ed zero data is faster oh hmm that really isn't a good test is it default is in the ssh_config man page... probably varies due to version, but aes has been preferred for many many versions it's aes128-ctr versus aes128-cbc? what's the diff i was reading something old that said blowfish default :) one is ctr mode one is cbc mode :p ctr is meant to be easily parallelisable, cbc isn't 209715200 bytes (210 MB) copied, 6.54909 s, 32.0 MB/s versus 209715200 bytes (210 MB) copied, 2.4191 s, 86.7 MB/s and non-zero file oh? first is ctr second is cbc dd if=/var/www/testfile.zip bs=1024k | ssh -c aes128-cbc root@localhost 'dd of=/dev/null' hmm# gzip -cv /var/www/testfile.zip| wc /var/www/testfile.zip: -0.0% 820474 4625767 209749312 so non compressable it would be nice to be like more than 50% faster than that :) so that ssh can be liek gigabit speeds with encrpytion if you want fast ssh you need the hpn version - that actually has a parallel version of aes-ctr oh? i don't really need faster ssh i was just curious how much diff these acceleration things made and i'm not getting a lot of diff aes128-cbc is faster than aes128-ctr on xeon e5435 too though.. maybe there's no acceleration for ctr but there is for cbc? dunno does the hpn version work on openbsd? xeon 5435 has no acceleration aynway err i meant 5345 but i think that could be right too LT cos i foudn a rhel bug about such, i cant' acecss it though to see more about it cbc is a pretty standard way of doing things that's been around for ages, but ctr is newer and I think the ssh people might even have invented it hmm well the ssh people took out the none cypher i like the idea of encrypting auth but not data :) the hpn version adds none back ahh i do slightly remember hpn from ages back when ssh had window size limitations that made international transfer speeds always suck back then it was always behind ssh by quite a few verisons http://www.cl.cam.ac.uk/~rja14/book.html is a great free book for understanding the differences between cbc, ctr, etc as an aside cool i'll bookmark it hmm dd over ssh is faster than scp i do actualyl want to understand encrpytion sometime .. especially with regards to interactive ssh type applications (ie not much data.. data could be guessed by delays between keystrokes etc) type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 279385.84k 277993.37k 301771.51k 318107.50k 319316.99k mercutio: ^^ AMD 6216 mercutio: the kvm nodes are dual E5430's these are numbers on the E5430's (no acceleration) type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 95563.29k 106319.22k 110646.64k 112421.96k 106064.11k jeez Intel Xeon E5-2660 Sandy Bridge-EP 2.2GHz $1329 all i can is Fuck That(tm) *say is 3:28PM up 391 days, 17:36, 4 users, load averages: 0.05, 0.06, 0.01 ^My ARPNetworks FreeBSD VPS...I think I'm going to be starting it from scratch soon :( $ uptime 13:49:07 up 958 days, 13:37, 3 users, load average: 4.09, 4.76, 4.38 kvr04 went up to 983 days, my longest box ever good lord maybe this one will surpass it am I on that box, by chance? that's kvr05 check your vnc host, says what box you're on I honestly don't even know what I'm on Roger that, one second bah I don't know my VNC host :\ I thought it would be in the dashboard somewhere oh wait, it is I'm on kvr06 ah that's a long running host too $ uptime 13:07:43 up 934 days, 10:01, 2 users, load average: 3.10, 3.23, 3.31 nice Hopefully it keeps it up :P :) all that time with no kernel security patches :D My ARPNetworks VPS is the hub of my entire network because of its stability I've got VPSes from elsewhere, and they don't really compare imo I've considered moving my entire network to ARPNetworks, but I don't know how much sense that would make for me. phlux: :) phlux: i say "do it" haha, I bet! If I could ensure that my servers were all on different hosts, I will be convinced to swap over everything. twobitha1ker: i knew someone would say that... Someone says that to me about my VPS, too I tell them to go ahead and exploit it for me right quick that's what you always hear when u try to tout uptime phlux: LOL hahaha They always answer "Sure give me a shell." It doesn't work that way. If I did that, I'd be giving you a head start. phlux: well, we *do* put VMs of the same customer on different hosts, by default. No kidding? So if I picked up 3 more at the same time, all 3 would be on separate hosts? this one dude got like 15 VMs, in one day, and that was the only time it was hard to make sure they all got spread out phlux: sure of course If so, I am going to make that change on Sep 1 (which also happens to be my birthday, yay me!) Awesome It's going to be so much easier paying one company :P phlux: i understand the concern about not having all eggs in one basket, i practice that myself, so why not do it for the customers? phlux: see if you can negotiate for an arpnetworks tshirt up_the_irons: I'm glad to hear that you do it that way. lol fink YEAH 3 servers for 1 T-shirt whaddya say?! phlux: i wish i knew that was your only concern, i would have addressed it earlier :) up_the_irons: No worries. I originally signed up a long time ago with the intention of "testing the waters," and I ended up using that VPS for *everything* phlux: haha cool Speaking of which, I'm teaching a guy some things, and he's most likely going to pick up a VPS from ARP in the coming days. Hoping his introduction to programming goes smoothly..he wants to be an IT Commisioned Officer in the USCG. I'm waiting for him so we can try to go to OCS together. We're both already enlisted, so we wouldn't really be competing with eachother. I got invited to OCS, then saw the age cutoff was 29. I officially felt old (I'm 33) i hunger, cd $food Dang I didn't know that the cutoff was 29 I'm only 23 (as stated earlier, 24 soon) so I haven't had to worry about age limits for anything, yet up_the_irons: child :D I was considering re-careering here as a direct entry officer, but the Canadian Forces dropped signing bonuses and I couldn't afford the pay cut without one Man, if I'd have joined a year earlier, I'd have got a signing bonus with the CG. I became an MST (Marine Science Technician) I deal with oil spills, HAZMAT releases, waterfront facility regulations, foreign and domestic vessel inspections, and container inspections. That's a mouthfull. also I suggested this possibility to my wife and her reaction was nearly as violent as I could expect in Kandahar :\ o.O ? up_the_irons: Given that kvr05 has 934 days uptime, how is its health? kraigu: lol arenlor: health looks good; there are a couple drives with reallocated sectors, but i did a raid verify (twice) on it, and everything checked out; so, i'm just watching it phlux: that sounds interesting, actually up_the_irons: Alright, given that my important eggs are on it, don't care to have it go down too suddenly ^_^ phlux: you will have to worry about age limit for American Idol soon (26, i think ;) arenlor: i hear ya. i don't want a repeat of kvr04, where i spent all day and night at the data center nursing it back to health up_the_irons: well they do say natural breastfeeding is better than the bottle, so good job lol up_the_irons: e5-2620 is a lot cheaper than e5-2660 and your e5430 results seem a little low ie i got higher results than that from e5420 it may be openssl version, or it may be load / context switching overhead mercutio: well, that was on a production server, hardly idle yeh load could create that it could be related to openssl version too OpenSSL 1.0.1c 10 May 2012 is what i was using err actually it was 1.01 1.0.1 i think openssl changed a bit recently mercutio: i was using 0.9.8something up_the_irons: less talking.. more testing centos images or spending time with your family. ;) i'm researching ways of exporting a block device from one host to another (say an lvm volume from one host to some /dev/foo on another). i've used AoE before, but things these days seem to be leaning toward iSCSI. Anyone have any thoughts on the matter? jpalmer: lol jpalmer: the kids are asleep:) same with the wife iscsi is the new hotness. do it! brb I think I'ma head to bed. night all 'nite up_the_irons: think newer versions have improved by a bit up_the_irons: with iscsi you want to have seperate network for it and use large mtus balancing can be annoying, you c an use either multiple gigabit or expensive 10 gigabit but with gigabit you limit top speed to 110mb/sec or something fibre channel is the other option that is affordable but most of the affordable stuff ise second hadn and 10 gigabit ethernet is coming down in cost that said 100mb/sec is racing, if access times are good mercutio: i c mercutio: separate network is no problem, large mtus no problem pretty easy to bond 2 gige's, but fc would be better mercutio: have you used any iscsi targets that you would deem reliable? only opensolaris interesting and that's in state of "uhh what's happening" wut?:) well when oracle bought sun everyone left err a lot of the higher up people left there was a fork to illumos ah the illumos camp is hopping openindiana is going between prereleases... it's the only place to be oh, would you look at that... who has illumosvps.com (me) openindiana seems stable with ok hardware stable as in reliable they have a lot of sun people working there so we should see some good stuff but not stable as in "one version for years" yeh well, no smartos sounds interesting too smartos is a great concept dang, i have openindianavps.com too md3000i is around for iscsi support but is slow i musta been looking toward the future... ;) there are higher cost options too hmm well joyent ported kvm to illumos it runs very well holy shit, they did? yes yeh wow runs good? that's what i have heard the base tools haven't had the work they need but the raw support is stable yes exactly once they port libvirt even though they ported it quickly it is going to be very attractive showing that they have good engineers they have a lot of the old sun guys because of the mass exodus from oracle by good poeple leaving oracle people like the guy who made java left mhmm smartos _does_ sound interesting all of the zfs, dtrace and other solaris kernel specialists left and said how he was unhappy with the direction oracle were taking sun and you'd think someone like that isn't likely to be someone who would motuh off randomly they aren't they LEFT http://www.eweek.com/c/a/Application-Development/Java-Creator-James-Gosling-Why-I-Quit-Oracle-813517/ yeh but i mean it's not like a straight business man leaving or something and saying about his new venture no, they just don't want to be a part of the lawnmower up_the_irons: i could totally see you running smartos one day up_the_irons: assuming it becomes stable Webhostbudd_: i'm checking it out right now what's funny is that their port of KVM requires the use of zones so each kvm instance is wrapped in a zone oddly enough it prevents a lot of kvm exploits because the attacker can't break the zone i think that was a sensible move yes it was i mean, why not it gives the administrator even more control over the virtual instance and with zfs you can clone file systems and do copy on write zfs is amazing so it's not as high overhead having heaps of zones smartos is a killer combination of technologies i would love to see it get bigger webhost: marketing on the other hand... have you seen their videos? no? they seem tacky it's pretty hard to market it at this point though let's be honest, they don't have a product to ship i think they do? joyent are doing private clouds already so does smartos run illumos underneath? http://joyent.com/solutions/privatecloud/ no one who is concerned with how they market a product is going to call that a product up_the_irons: yeh up_the_irons: yes and is one of the major contributors and illumos is an opensolaris fork? illumos is the kernel if i am not mistaken or do they do userland work? illumus is like the base system like netbsd etc i think yea, that was always my impression yeah i think i get it except they don't actually assemble any of it openindiana is more like opensolaris i wanna say it would be like GNU finishing a kernel there's also nexenta or something which uses apt-get the way i see it, GNU = illumos and OI / nexenta are distros IF GNU had a kernel so, with zones, i've always thought of them to be the solaris equivalent of freebsd jails. and since they are just processes that share the same base OS kernel, is it true the only thing you can put in a zone is another illumos instance? up_the_irons: mostly true up_the_irons: i think it's like freebsd jails, but a bit more brokenout (and that instance could run kvm, yeah, but in the end, it's still illumos) yes roger but there is probably work to emulate linux / bsd systemcalls i know freebsd jails can run centos with linux emulation web: i think that work is still behind maybe smartos is getting a new package manager i think i don't use zones so i wouldn't know and packages can be old at present well, all of the illumos stuff seems to be using pkgsrc so illumos is only the kernel, and openindiana is kinda like kernel + userland (e.g. GNU) but i think in general it's under quite a lot of active work freebsd jails can run cent? LOL up_the_irons: yea, i think that is probably more correct up_the_irons: bottom line, you don't run illumos Webhostbudd_: yeah makes sense, at least roughly openindiana has both sun and gnu userland directly yea ls you have /usr/gnu/bin/dd and /usr/bin/dd and the normal dd doesn't support things like conv=fdatasync there is a really cool fork out there which im really digging stormos debian userland + illumos kernel webhost: what have they done? similar to debian kfreebsd oh? i think i'm gonna load this up on one of my microcloud blades give it a shot smartos is doiugn things new and fancy like they want you to network boot even more reason to finish that box and haul it to the d/c mercutio: which is great and have all the storage on a machine running under zfs imagine how easy it is to deploy nodes which isn't a new concept nope don't the big guys do it? people have been doing such with esxi etc for a while vmware yea but like they're working towards "well thought out" ways of doing things which i like but also allowing a steeper learning curve and doing things diffrently like they're not trying to just support a whole lot of legacy ways of thinking nope they're trying to streamline things etc i really appreciate the effort for a solid, opensource hypervisor linux really doesn't fit the bill well, the distros of today don't illumos is a very solid kernel though hardware support is probably shaky though outside of your common server hardware it's better than esxi for hardware support i would only want to run common server hardware ;) which is outstandingly sad the uncommon stuff is expensive... up_the_irons: one of the older hp's was broken a while back hp's make me cringe cpqarwy o rwhatever scsi controllers cpqary3 if it'll run on an Ivy Bridge 3.4GHz + plain ol' SATA + Intel GB, then that's all i need :) http://www.greenviolet.net/articles/2011/09/21/openindiana-151a-and-hp-smart-array-controllers.gv anyone here in the market for a sas expander? up_the_irons: yeh it'll do that mercutio: yeah i didn't figure that basic of hardware would be a challenge up_the_irons: you need nehalem or whatever or newer cpus for kvm under illumos mercutio: well sure, u need that for kvm in general yes well the hp thing would effect quite a lot of older hardware up_the_irons: nah ehhhh nah wut up_the_irons: e5430 won't run kvm on illumos i run kvm on adm phenoms mercutio: oh? why? because they're relying on some newer intel feature ah yes uhh it hikn it's EPT interesting i can't remember what it was that sounds correct they don't support amd at all do they? oh, extended page tables? web: that was the case last i knew no amd support?! nope up_the_irons: that's what it stands for but i don't know if that' the case or if it was anothe rfeature k up_the_irons: someone may fix the amd stuff sometime but smartos developers focused on intel i'm loving my AMD 6212 system... Currently KVM on OpenIndiana only supports recent Intel VT chipsets. Webhostbudd_: ah gotcha bulldozer opterons huh? yup do you get 16+ cores on those? the 6212's have 8 oh but two sockets on my board, so 16 altogether Finally, there is no support currently for AMD SVM. This is not a value judgement of AMD's technology, but rather a reflection of limited engineering and testing resources. (In the spirit of full disclosure, it should be said that the sponsor of illumos-kvm, Joyent, is an Intel-funded company -- but the lack of AMD support reflects only engineering prioritization and lack of testing infrastructure; AMD SVM support would be most welcome should someone in the it sucks that those 8 cores get beaten so badly by 4 intel cores Webhostbudd_: my cpus: http://i.imgur.com/Ed6nw.png Webhostbudd_: in benchmarks or real world operation? actually more in real world cuz i'm finding them to be quite awesome but a lot of server stuff is multithreaded and integer intensive so bulldozer should be a good fit yeah you've also got context switches to worry about and having more cores can cut down in that when virtualising exactly context switching is super high in virt i've never seen any good benchmarks for high load virtualisation of multiple diff applications no, and that is a problem i find that benchmarks are like job interview questions; they look good on paper but do not reflect how the thing will behave in real life situations exactly up_the_irons: well it depends if you find the right benchmarks.. you can also find the right interview questions like you cna easily find game benchmarks to know if a video card will be a good fit for you mercutio: maybe i'm looking at the wrong stuff then but finding cpu benchmarks for virtualisation ... yeh.. mercutio: that's pretty hard to say though mercutio: yeah, games are specialized, cpus are generalized mm that's actually the problem with game benchmarks though i'd look at kernel compile time benchmarks probably unless you always play the same games... but they're still not going to be accurate webhost: you're comparing video cards to see what would be "good enoguh" for you :) mercutio: kernel compile time might be a good overall heuristic, it uses disk + cpu compile benchmarks have a lot of floating point though right? webhost: nope i would imagine no? why would compilers be floating point? video encoding etc would be but compiling is gnerally memory/integer heavy see, i would assume that if you add 2.5 + 3.5 on a compiler using float point numbers the compiler only has to generate the assembler that doesn't require it actually doing the maths. oh im not talking about that mercutio: perhaps at the optimize phase there is fp (but not lexer, parser, etc...) *optimization yes this is what i always thought well, if you multiply by 1.5 it probably does a shift and adds it to the number rather than actually doing it in floatin gpoint im not talking about arthimetic in code at all hmm there's not that much optimisation work these days. there's some, but it's not really as fancy as people often seem to believe. obviously fancy stuff breaks heh people like to believe a lot of cool stuff is at work it's mostly stuff like cache line optimisation that is easier for compiler to do than by hand mhmm and reordering to prevent stalls. yep i'm still surprised by how big the linux kernel is freaking massive i'd like a kernel < 1 megabyte compressed really i have one with lots of stuff disabled and it's still 3.4mb compressed back with linux 2.1 it wasl ike 500 or 600k compressed lol freebsd kernel so small how big is freebsd kernel? that's not real bullshit 630k really openbsd is 5.1 mb did freebsd go to modules? i do have a lot of stuff disabled in my kernel freebsd does have modules but i also removed a ton of built in device support openbsd isn't compressed nor modular well my freebsd kernel doesn't even have ufs support oh i mean, i compiled everything out it can't be zfs with that size what fs are you using? zfs doesn't build directly into the kernel and zfs is only 120k oh it's not huge that's good nice sec i've got this thing against bloat :) alright, i was massively mistaken =p 2mb for zfs.ko lol heh i thought zfs was big :) it'll have support code etc well, i knew it was big, i thought most people just overestimate heh zfs is a bit of a memory hog too it's nice ram got cheaper :) oh god yea zfs is quite a memory hog zfs is fast with lots of ram though blazing i love that arc cache weird this page says kvm just needs vmx which kvm? illumos linux kvm probably oh maybe they added support recently? ARC Size: Current Size: 3598 MB (arcsize) Target Size (Adaptive): 3598 MB (c) Min Size (Hard Limit): 638 MB (zfs_arc_min) Max Size (Hard Limit): 5111 MB (zfs_arc_max) and maybe that OI wiki is out of date "memory hog" is another word for "aggressive caching" ;) https://github.com/joyent/illumos-kvm/ it goes real slow with 2gb of ram so wait.... that thing works on amd! oh what? derp svm is amd yea that page makes installing kvm really hard it's actually simple on openindiana just use smartos =p EPT is supported on E5430 maybe it does work on earlier cpus? interesting what's this omnios people love illumos jeese anyone know of a good iscsi target for linux? i see iscsitarget and open-iscsi (and i actually don't know of open-iscsi can be a target...) no idea god, i can't believe illumos is built with gcc 3.4.3 that just feels super old now