rawr for the record, yes, I changed default partition between 7.2 and 8.0. 7.2 had slices for /usr /var, etc.. 8.0 just has one whopping / slice (ad0s1a) and swap (b) even if someone puts custom partitioning in the order form, I can't do it. Default installs come from a template. Custom partitioning would require me to actually do the install from scratch, and given there is no setup fee... not gonna do it ;) Everyone can do their own custom partitioning by performing a re-install over VNC up_the_irons, is it possible if i have a 400 gig /dev/md0 raid1 partition to create a LVM using some of it's space? jeev: software raid? sure i've never used software raid other than dismantling it anyway, that's not the point the point is it possible to create a lvm using free space in an existing Linux partition? not LVM not a lvm partition jeev: not really ok, if i have to reformat this box one more time im gonna be pissed jeev: you want part of the partition to go to lvm and not the rest? are you high? ;) maybe rather than installing with kvm over ip that's slow as shit with pxeboot that takes forever i should dismantle hte raid, set up lvm and copy the data over? given i don't know your requirements, i'm not going to comment its a mess around box at the datcenter datacenter heh... sprocketnetworks just got wind of my move now they say "oh, we offer FreeBSD VPS" idiots... why aren't you listing that on your website? :) do you see anythign but Linux or Windows on http://www.sprocketnetworks.com/virtual.htm no... neither do I. :) is that mr spacely's space sprockets? and their 60GB disk plan is $149/month and they don't have VNC consoles which I have now come to love RandalSchwartz, that's a lot of adult videos I don't think their disk is raided either so - I pointed them at the ARP page... let's see if they want to compete here's the other thing - PING blue.stonehenge.com (209.223.236.162): 56 data bytes 64 bytes from 209.223.236.162: icmp_seq=0 ttl=241 time=73.464 ms PING red.stonehenge.com (208.79.95.2): 56 data bytes 64 bytes from 208.79.95.2: icmp_seq=0 ttl=53 time=41.408 ms blue = sprocket red = ARP been trying it on various local nets... red is always the winer winner too red and blue, sounds political re-install over serial console, I prefer that one, much more secure *grin* vs re-install over vnc hmm, openbsd.. not good heh red @ okc cox cablemodem: round-trip min/avg/max/std-dev = 49.747/54.221/59.090/2.536 ms blue @ okc cox cablemodem: round-trip min/avg/max/std-dev = 15.614/19.917/26.320/2.976 ms not just a small difference, its nearly double the latency sorry, more than oh weird. so it depends on where you're coming from then though I'm sure jeev will cringe when he realizes this was generated using openbsd through an OpenBSD/hppa firewall red=75ms, blue=84ms from here but of course, each fabric of the net has its own resonance frequency that changes the song slightly depending on where you come from and where you go to yup toddf, stop pretending you know anything about how computers, openbsd and networking works! but anyway, let's see if sprocket wants my business bad enough I don't think they'll have VNC console anytime soon ssh serial consolez is where it's at no root password sniffable sounds good to me and now that I've booted and rebooted my virtual DVD enough, I'll really miss that if I don't get it note vnc is not encrypted just yet unless you ssh port redirect it well - yeah, that next still haven't set that up cat $HOME/.ssh/id_rsa.pub | mail -s "randalschwartz.pub" support@arpnetworks.com there, no excuses, cut and paste or if you're paranoid generate a unique key just for arp if you think you like vnc wait till you see serial ;-) RandalSchwartz: did you come to love VNC consoles wrt your VPS VNC console, or do you mean VNC in general? if I had mail on my laptop, sure which I don't :) echo "The problem is undoubtedly`nc bofh.jeffballard.us 666 | grep excuse | sed 's/Your\ excuse\ is://g'`" | mail $customer so I stil have to to transfer the file pbcopy ". done and done and theres a decent amount of approval depending on how much the order is yea up_the_irons, lilo rules but not with xen this is a non-vt capable box i'm setting xen up just for fun oh, that reminds me. i gotta order a 2821. thanks ballen =) heh np order me one while your at it could use it for example, i set up lame software raid and it's resyncing even though i've formatted it although I did find a Pix 501 laying around today we have some retards at one site bitching because their fiber and ds-1's (for backup) all run into a 3845 "but what if the router goes down!?" lmao heh, i think i have a 501 in the bottom of my desk then your SOL, next question i'm ordering the 2821 and they're paying for it, they just don't know it yet we'll move the ds-1's over to it to shut 'em up, but it'll cost 'em =) jlgaddis: that's a 2800 on your blog, no? yea not much use for one, just surprised to find Cisco gear, we were all 3COM (eww I know) now all nortel up_the_irons: the header? yeah yeah, sprocket couldn't even come close they still insist I should have been able to tell they'd run a FreeBSD VPS for me, by looking at a page that talks only about windows and linux. :) but no worries jlgaddis: i'd have to agree w/ the client; if even their redundant links go into the same equipment, it's not very redundant.. I gave them a chance. :) they completely blew it off. :) I mean seriously - http://www.sprocketnetworks.com/virtual.htm - does that imply FreeBSD anywhere? jeev: my bro filled out a credit card app and put "W. Axl Rose" on it, and sure enough, he got a credit card that read "W. AXL ROSE" ;) damn jeev: so go get your jack bauer card you can get paypal to send you a jack bauer card honestly up_the_irons: oh they have a valid point, sure. but they have a spare sfp and card on site, so it's not a huge deal. paypal even sent me a card with my name mispelled they must retype it somewhere and not just use electronic data besides, we're running gigabit over the fiber, and their "backup" is only 3xT1 RandalSchwartz: search for "freebsd" turns up nothing on that page and those will be at 100% if/when it fails over anyway because of all the students http://www.google.com/search?q=freebsd+site%3Asprocketnetworks.com&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:unofficial&client=firefox-a RandalSchwartz: ^^ looks to me FreeBSD is only an option on a dedicated server damn, gotta run to work and scp the snow leopard disc image to my macbook. bbl. later w00t and i did have a 501 in the bottom of a my desk. and a 5505 on top of it too, apparently. http://www.flickr.com/photos/jlgaddis/4249935434/ hah nice hey, im trying this slack lvm, raid1 md0 xen set up one more time.. if it doesn't work, if anyone wants my liver.. i dont drink i could use a new one ok i'll put you on the waiting list hmm. should serial console show me a login prompt? or do I have to enable soemthign special? it accepts my serial password and says [Enter `^Ec?' for help] but no login prompt and yet, the box is up and running. press enter ? yeah, press enter I have, no response RandalSchwartz: /etc/ttys is your friend and yet, red is up and running just fine uncomment the /dev/ttyu0 line getty on serial line is not enabled by default on any FreeBSD that I know of it's uncommented nor OpenBSD RandalSchwartz: err, i mean.. ttyu0 "/usr/libexec/getty std.9600" dialup off secure RandalSchwartz: turn it from "off" to "on" and set the terminal from "dialup" to "vt100" serial console is enabled by default on sparc64 jlgaddis: nice but i imagine most users don't run into that often =) right :) and then signal something, right? sighup init, at a guess RandalSchwartz: yep, but what you signal, i'm not sure kill -HUP 1 well - something like that shuts the system down at that point, i always had to reboot the VM anyway for some other reason, so never worried about it kill -HUP 1 says init(8) here goes \o/ \m/ aha! prompt! neato and there I am, logged in now how does this set me up for SSH VNC? oh - I can tunnel on this duh yup hit 't' for instructions yup, "t" in the menu, that is where... at teh menyu? one of those "obvious" things "t" for instructions "j" for tar w/ bzip2 RandalSchwartz: the "c" option is to interact with your VM's serial line, most often used to run a login getty; the "t" option is to tunnel VNC over SSH and it simply displays instructions on how to do it ugh - 5555 - 5900 is... :) ok - crap, can't type a minus sign can't use this tunnel PLEASE realize that VNC is traditionally 5900-5999 ports outside this range cause some UIs no end of problems 5555 for example is WRONG. flat out WRONG RandalSchwartz: you can pick any port you want RandalSchwartz: port 5555 was just the example yeah, but its' the cut-n-paste example put 5955 in there instead then I can type "55" for my VNC port which in fact, did work that's fine, changed but your choice of 61xx for VNC otherwise as also confusing. :) for my unencoded VNC anyway, nice to know that I now have a tunneled console, thanks all is good RandalSchwartz: 5900 - 5999 is not enough, that's only 100 users you have more than 100 users on this box? or is it all choked through the same IP? for *all* customers? RandalSchwartz: no, but if I think in terms of individual boxes, I can't migrate VMs very easily, because their VNC port numbers will clash I have a "pool" of boxes and within that pool, the VNC ports do not overlap different pools can have overlapping ports understood LOL so far, very happy glad to hear I might exceeed inbound bandwidth for first month transferring crap from old box after that, it'll settle down roger I have a very carefully crafted rsync script which *should* be getting all items of interest from old box rsync to off-site box, doing a backup there using plain ol' tar, and a backup to s3 using tarsnap it's 44GB of data from the old box, but I'll need to refresh any changes once or twice more before turning old box off jlgaddis: i've found sending tarballs to S3 quite unreliable; it's like some get there and some just don't. i split them up at the 5G mark of course, also, but to no avail jlgaddis: so if you do that, just keep on eye on it every now and then; make sure what you think is on there is really on there i'm using colin's tarsnap. it actually goes to his servers first, once it's safely there it's transferred (by his software) to s3 and, yeah, i checked it from another box after the first full backup =) <-- learned a looooong time ago to test backups =) sending ZFS snapshots is my new preferred offsite backup someone should set up a disk farm to receive those there's a business model :) up_the_irons? :) seriously... if you could give me a place to send ZFS incrementals that was across the country from you, I'd pay because a ZFS snapshot is atomic and a ZFS incremental is the minimal change I have very limited experience with ZFS. What exactly does a snapshot consist of? Is there a reason you can't just sent it to S3? when I think of business models, I try to see who can crush it. And right now, with storage, Amazon is crushing a lot of models a zfs snapshot is an exact moment-in-time grab it costs NOTHING to make very cool and then you can "zfs send | ssh remote zfs recv" it but the next step is... make a new snapshot then send only the diffs from old to new to remote and it understands the diffs so you have now moved from atomic point 1 to atomic point 2 ah I see so zfs needs a way to see the old diffs also and because it's at the FS level, it's far more efficient than rsync yeah yeah interesting ... http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide RandalSchwartz: so what would be the requirements? disk space (obviously), ssh shell ?, zfs on destination ?, ... ... http://www.markround.com/archives/38-ZFS-Replication.html yes - the disk sadly woudl be the same you're making a clone, and keeping it up to date but then you can also sell that service "we have hot machine spares offsite" or you could just say "your disk is cloned here, we can move your entire data wherever you want on a fault" RandalSchwartz: so you would need a ZFS formatted fs of the same size at the backup site, along with SSH shell access? yeah anything else? well - depends on if it's failover or just access if it's failover, same CPUs if it's just access, a way to push that data somewhere you could sell two levels I mean, I'd pay for just access but for some business, they'd want COB level so they'd pay for a hot spare machine failover could be problematic because the IPs in Los Angeles would be different than, for example, New York, unless I get a long haul, which would get *quite* expensive. or even with newer MLPS "virtual" links between metro areas, it still is kinda expensive; although something worth looking into well - you failover at DNS level so you require a DNS change for switch ah gotcha not a big issue this is catastrophe planning not hot switching roger I want my data in Los Angeles and "somewhere else" or I want my data and CPU in Los Angeles and "somewhere else" in the former, I can accept a few hours downtime in the latter, I can accept TTL downtime :) yeah hah make sense? yup this would be a great service for your customers and great for the SOHOs who need net presence, and some sense of COB ... http://www.markround.com/archives/38-ZFS-Replication.html right yeah i'm looking at that one oh - did I paste that already :) damn - did failover using dns == fail jlgaddis - suggestion? by using DNS, I say "www.stonehenge.com = 1.2.3.4 not 4.5.6.7" and that'll take TTL to clean up yes, but not that aren't costly so for TTL, I lose hits oh well :) i guess if you're not losing revenue, it's not a big deal well - that's why my TTL is 15 minutes :) yeah - it's a trade i've never really liked DNS failover, but if you can take the TTL downtime, i suppose it works my site is a brochsure site not a live site and twitter was down far more often than me :) and I can never trust other people's resolvers to respect my TTL you have to trust things at some level :) if you really want protection, buy an offsite traffic director RandalSchwartz: indeed, twitter being down often and facebook getting API DoS'd has lowered everyone's expectation of uptime ;) so traffic goes to minnesota, then LA or minnesota, then "other place" so yes, there are solutions we have a hot site several hundred miles away or even better... yes, www.stonehenge.com points to second site, which mostly proxies to first site until first site goes dow then it takes over primary the nice thing about zfs snaps is they're very very cheap to make and send RandalSchwartz: i presume that something like MySQL, on top of ZFS, would not necessarily "snapshot" well since it could be in the middle of a transaction; in that case, could a better practice be to replicate MySQL some other way, or possibly not even use MySQL at all (ala the popularity of distributed KV data stores now) I've heard people sending them in tight loops "make snapshot" "send it" "make snapshot" "send it" up_the_irons - NO that's precisely the cool point of zfs snaps it's ATOMIC it can reproduce the disk PRECISELY midstream so if mysql can handle a random reboot, then it can handle the snapshot restart if it can't, stop using mysql I know postgresql uses WALs, and does this just fine as in, the current transaction will likely fail but everythign committed up to that is done RandalSchwartz: yeah, i don't know what mysql does if it is in the middle of a transaction and it gets rebooted; i assume the transaction simply is never committed ZFS snapshots are *atomic* and this makes them far better than rsync replication bytes are either THIS WAY or THAT WAY nothing in between right zfs create -r zpool@now that makes snapshots in *all* filesystems at NOW man, i'm gonna have to play around with some of this stuff, sounds exciting completely atomic zfs rename zpool@now then zfs create -r zpool@now now I have two snapshots that I can increimtanlly send zfs send -i then zpool@now | ssh remote zpool recv and now I've send the differences over the wire and atomic updates all around repeat as needed even in a tight loop it's pretty exciting wow, interesting yeah - so the ability to *cheaply* clone a disk over the wire has gotten far better thanks to zfs so for people wanting geography backups, this is *the* solution now presuming they're willing to use ZFS can you see why I wanted to boot from ZFS now ? yeah, ZFS is the only dependency (albeit a big one for some) i do! Filesystem Size Used Avail Capacity Mounted on zroot 2.3T 529M 2.3T 0% / :D nice i should make a ZFS-enabled default VM and people can order it. Basically it would be the default install w/ ZFS instead of UFS I cheated a little, USB key for /boot Yeah - the instructions on that page worked once I followed them to the letter I'd be available to talk to your clients I'm happy to share what I know in fact, I snapshotted @INSTALL right after building my machine so I could zfs-send you my install :) but it's only really about a half hour to follow the instructions just set the root password to "changeme" and the timezone to PST :) then snapshot that the rest is completely generic LOL RandalSchwartz: which instructions page? I remember a few being pasted here uno momento, por favor ... http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot that's the instructions worked perfectly once I followed them exactl' y my failures were because I skipped a step, or tried to add "zfs export" to the end so I'd say follow those instructions, setting "changeme" for root pw and "PST" for timezone why was zfs export of any importance ? and then snapshot that for new installs ballen - I thought "zfs export" was like "unmount" turns out ists more than that hah, ah yea not quite unmount at all so when I stopped doing that, then reboot saw the disks as "attached here" which is what it is neede ah nice export means "the disks are going somewhere else" ah which is why boot said "no zfs to do here" so follow those instructions EXACTLY unfortunately, you'll need to do those nce per disk size since the FS is built each time as in - you'll have a 120GB-ZFS and a 60GB-ZFS etc gotcha cd $HOME bye RandalSchwartz: thanks for all the info, I'm going to play around with it "cd" => "cd $HOME" cd ~ again redundant :) "cd" true but just saying true => : :) cd is a little confusing if :; then echo yes; else echo no; fi interesting go home sir cd I'm in LA next week we should meet :) let me know about where you'll be; we can grab lunch around Burbank Wokacno! Wokcano LOL sure across the street from my $DAYJOB green dragon roll, mmmmMMMm spicy plate :) I'm at the holiday inn kittycorner from there i'll be on later, cd $HOME, for real my walk to work is half block bye! zfs receive piped over ssh zfs send -> ssh -> zfs receive up_the_irons: long haul? just do the same trick dns servers do, advertise the same address multiple locations... anycast? anyone in Canada? cd'ing to a variable is pretty funny toddf: anycast? oh please no can I get some traceroutes on 174.136.96.2 pastie them new block I set up a week ago, not completely reachable yet, and I need to know who it fails for can't reach it from Time Warner / RR (big surprise), but works from Slicehost goes hrough through from krypt jeev: pastie.org it :) 2 VLAN3810.BR1.LAX3.VPLS.NET (67.198.200.61) 5.200 ms 0.698 ms 0.557 ms 3 VLAN2099.BR2.LAX2.VPLS.NET (67.198.200.17) 0.384 ms 0.316 ms 0.314 ms 4 mzima.net.any2ix.coresite.com (206.223.143.2) 4.559 ms 5.067 ms 0.452 ms 5 ge0-arpnet.cust.lax07.mzima.net (67.199.135.102) 1.551 ms 1.604 ms 1.817 ms 6 174.136.96.2 (174.136.96.2) 0.697 ms 0.752 ms 0.587 ms :> oh u whore too lazy to open browser i have 40 open already thanks jesus man i cant get this system going i'm gonna try it with crypt, whatever gonna try two more damn things trace through att uverse is gay as usual i c