[01:20] *** schmir has joined #arpnetworks [01:25] rawr [02:45] *** visinin has quit IRC ("sleep") [03:11] *** dbgi3 is now known as dbgi [03:49] *** ziyourenxiang has joined #arpnetworks [06:09] *** heavysixer has joined #arpnetworks [06:09] *** ChanServ sets mode: +o heavysixer [06:45] *** ziyourenxiang has quit IRC () [09:10] *** jlgaddis has quit IRC (Read error: 113 (No route to host)) [09:16] *** jlgaddis has joined #arpnetworks [10:08] *** schmir has quit IRC (Remote closed the connection) [12:39] *** nakano_ is now known as nakano [12:45] *** coil_ has joined #arpnetworks [13:12] *** sroute has quit IRC ("WeeChat 0.3.0") [13:12] *** sroute has joined #arpnetworks [13:12] *** ChanServ sets mode: +o sroute [14:25] *** coil_ has quit IRC (Read error: 104 (Connection reset by peer)) [14:32] *** coil_ has joined #arpnetworks [14:36] *** schmir has joined #arpnetworks [14:46] *** coil_ has quit IRC (Read error: 104 (Connection reset by peer)) [15:09] *** dbgi3 has joined #arpnetworks [15:19] *** baklava has quit IRC (Read error: 104 (Connection reset by peer)) [15:20] *** baklava has joined #arpnetworks [15:24] *** schmir has quit IRC (Read error: 113 (No route to host)) [15:26] *** dbgi has quit IRC (Read error: 110 (Connection timed out)) [16:39] for the record, yes, I changed default partition between 7.2 and 8.0. 7.2 had slices for /usr /var, etc.. 8.0 just has one whopping / slice (ad0s1a) and swap (b) [16:42] even if someone puts custom partitioning in the order form, I can't do it. Default installs come from a template. Custom partitioning would require me to actually do the install from scratch, and given there is no setup fee... not gonna do it ;) Everyone can do their own custom partitioning by performing a re-install over VNC [16:42] up_the_irons, is it possible if i have a 400 gig /dev/md0 raid1 partition to create a LVM using some of it's space? [16:43] jeev: software raid? [16:43] sure [16:43] i've never used software raid [16:43] other than dismantling it [16:43] anyway, that's not the point [16:43] the point is it possible to create a lvm using free space in an existing Linux partition? not LVM [16:43] not a lvm partition [16:44] jeev: not really [16:44] ok, if i have to reformat this box one more time [16:44] im gonna be pissed [16:44] jeev: you want part of the partition to go to lvm and not the rest? are you high? ;) [16:45] maybe rather than installing with kvm over ip that's slow as shit [16:45] with pxeboot that takes forever [16:45] i should dismantle hte raid, set up lvm and copy the data over? [16:45] given i don't know your requirements, i'm not going to comment [16:46] its a mess around box at the datcenter [16:46] datacenter [16:47] heh... sprocketnetworks just got wind of my move [16:47] now they say "oh, we offer FreeBSD VPS" [16:47] idiots... why aren't you listing that on your website? :) [16:48] do you see anythign but Linux or Windows on http://www.sprocketnetworks.com/virtual.htm [16:48] no... neither do I. :) [16:48] is that mr spacely's space sprockets? [16:48] and their 60GB disk plan is $149/month [16:48] and they don't have VNC consoles [16:48] which I have now come to love [16:48] RandalSchwartz, that's a lot of adult videos [16:49] I don't think their disk is raided either [16:49] so - I pointed them at the ARP page... let's see if they want to compete [16:50] here's the other thing - PING blue.stonehenge.com (209.223.236.162): 56 data bytes [16:50] 64 bytes from 209.223.236.162: icmp_seq=0 ttl=241 time=73.464 ms [16:50] PING red.stonehenge.com (208.79.95.2): 56 data bytes [16:50] 64 bytes from 208.79.95.2: icmp_seq=0 ttl=53 time=41.408 ms [16:50] blue = sprocket [16:50] red = ARP [16:51] been trying it on various local nets... red is always the winer [16:51] winner too [16:52] red and blue, sounds political [16:54] re-install over serial console, I prefer that one, much more secure *grin* vs re-install over vnc [16:55] hmm, openbsd.. not good [16:55] * jeev hides [16:56] * toddf fires the cruze missle that was not built due to darpa funds redirected to OpenBSD [16:56] heh [16:57] red @ okc cox cablemodem: round-trip min/avg/max/std-dev = 49.747/54.221/59.090/2.536 ms [16:57] blue @ okc cox cablemodem: round-trip min/avg/max/std-dev = 15.614/19.917/26.320/2.976 ms [16:57] not just a small difference, its nearly double the latency [16:58] sorry, more than [16:58] oh weird. [16:58] so it depends on where you're coming from then [16:58] though I'm sure jeev will cringe when he realizes this was generated using openbsd through an OpenBSD/hppa firewall [16:58] red=75ms, blue=84ms from here [16:59] but of course, each fabric of the net has its own resonance frequency that changes the song slightly depending on where you come from and where you go to [16:59] yup [16:59] toddf, stop pretending you know anything about how computers, openbsd and networking works! [16:59] * jeev looks away [16:59] but anyway, let's see if sprocket wants my business bad enough [16:59] I don't think they'll have VNC console anytime soon [16:59] ssh serial consolez is where it's at [17:00] no root password sniffable sounds good to me [17:00] and now that I've booted and rebooted my virtual DVD enough, I'll really miss that if I don't get it [17:00] note vnc is not encrypted just yet unless you ssh port redirect it [17:00] well - yeah, that next [17:00] still haven't set that up [17:00] cat $HOME/.ssh/id_rsa.pub | mail -s "randalschwartz.pub" support@arpnetworks.com [17:00] there, no excuses, cut and paste [17:01] or if you're paranoid generate a unique key just for arp [17:01] if you think you like vnc wait till you see serial [17:01] ;-) [17:01] RandalSchwartz: did you come to love VNC consoles wrt your VPS VNC console, or do you mean VNC in general? [17:01] if I had mail on my laptop, sure [17:01] which I don't :) [17:01] echo "The problem is undoubtedly`nc bofh.jeffballard.us 666 | grep excuse | sed 's/Your\ excuse\ is://g'`" | mail $customer [17:02] so I stil have to to transfer the file [17:02] pbcopy ... [17:02] toddf: you should pastie the traceroutes of red vs blue [17:03] there - done [17:03] "support - You've Got Mail!" [17:03] some of us setup our laptops to have mta's that use port 587 or 465 to send mail out securely through a relay regardless of where we're at ;-) [17:04] heh, my traceroute to his .162 box goes from here to indy to atlanta to dc to ny (all over internet2) to san jose to dallas [17:05] to arp it's here to indy to atlanta to chicago to dallas to phoenix to la [17:05] fun stuff [17:06] RandalSchwartz: you should be able to login now [17:07] http://pastebin.ca/1739579 <-- red vs blue from okc [17:07] drats, no v6, can't traceroute6 [17:09] route through dallas is faster than through phoenix [17:09] i hate any presence in dallas [17:10] oversold trash network [17:10] that's actually a generalization, the reverse path will be different [17:11] i should get some pricing from Savvis though... [17:11] might be worth adding them [17:11] just avoid cogent whatever you do [17:11] they're cheap as hell, but there's a reason why [17:14] if cogent offered me a decent fiber to my business office I'd take it, but alas nothing reasonable in okc [17:15] they're not that bad, as long as you're multihomed [17:15] (to someone else, not to them) [17:15] all you techies move out here, its real cheap cost of living.. maybe enough IT mass might figure out a way to tap into some fiber to make some reasonable symetric data services available [17:15] "here"? [17:15] if I want 10mbit symettric its near $3k/mo [17:15] here = oklahoma city, ok [17:16] we lease assloads of dark fiber for 500/mo (per 2 strands) [17:16] no exchange point that I've found, which doesn't help [17:17] * toddf holds his breath for randalschwartz to proclaim 'eureka!' wrt ssh console stuffz [17:17] yeah i'm not about to jump on cogent [17:19] jlgaddis: where are you? what distance are you getting on the dark fiber? [17:19] up_the_irons: bloomington, indiana. most of it is no more than 10km, longest is 81km. [17:19] jlgaddis: nice [17:21] *** ziyourenxiang has joined #arpnetworks [17:22] we connect into i-light (http://atlas.grnoc.iu.edu/atlas.cgi?map_name=I-Light) and go out from there [17:23] "indiana gigapop" [17:26] *** ballen|away is now known as ballen [17:26] interesting [17:29] oh that's awesome [17:29] very cool [17:30] there's others as well, for internet2, i2 v6, the gigapop, and some other .edu networks [17:30] and you can overlay radar [17:35] that's a pretty damn cool visualization of bandwidth usage [17:36] there's some pretty neat toys [17:36] newest one (not yet public) allows customer to provision their own vlan's through their internet2 network [17:37] say you have gear in ny and seattle and need a vlan... click click click, done. [17:40] crazy crazy crazy, done. [17:40] jlgaddis: nice [17:40] i'm on my 17th pxe install of slackware cause i can't get this raid1 lvm and stuff with diff boot partition then install grub done [17:41] jeev: one of my admins just had that problem, standby.. [17:41] jeev: clear the disklabel in fdisk first [17:42] just hit "o" (create dos disklabel), then "w" (write changes) [17:42] yea actually i've done it but i've never done it with lvm [17:42] that'll clear any software raid crap on there [17:42] but i think i get the point of it now [17:44] honestly it's just grub being gay [17:44] Got a shipment of 13 SAS2 drives and the needed Dell drive caddies... and not fucking screws today [17:44] no* [17:44] i need grub cause i want to use xen [17:44] damn [17:44] yea [17:44] rather annoyed [17:45] ordered a 100qty box from Amazon [17:45] waste of government time and money [17:45] 4 bucks [17:45] we need a senate hearing [17:45] hah [17:46] oh and I came back to one of my chillers outside cooling tower being frozen solid [17:46] thankfully its the only room that has two chillers [17:47] so tip of the day... if you turn off all your machines for the holiday... the chiller doesn't need to turn on very often [17:47] if you have a water cooling tower [17:47] and its below freezing outside [17:47] you may have issues [17:48] heh [17:48] when you bu y from amazon, do you use a government cc? [17:48] I pass it off to my group office admin [17:48] ah [17:48] she has the cc [17:48] crazy a credit card billable to CTU LA [17:48] sick [17:48] can you order me a card with my name Jack Bauer ? [17:49] yea thats why not many people get one [17:49] jeev: it's amazing how much harder / disconnected / non-coherent some things are in Linux. I mean, installing a boot loader, why does it have to be so hard? On most *BSD's it's like "disklabel -B ". done and done [17:49] and theres a decent amount of approval depending on how much the order is [17:49] yea up_the_irons, lilo rules but not with xen [17:49] this is a non-vt capable box i'm setting xen up [17:49] just for fun [17:49] oh, that reminds me. i gotta order a 2821. [17:49] thanks ballen =) [17:49] heh np [17:50] order me one while your at it [17:50] could use it [17:50] for example, i set up lame software raid and it's resyncing even though i've formatted it [17:50] although I did find a Pix 501 laying around today [17:51] we have some retards at one site bitching because their fiber and ds-1's (for backup) all run into a 3845 [17:51] "but what if the router goes down!?" [17:51] lmao [17:51] heh, i think i have a 501 in the bottom of my desk [17:51] then your SOL, next question [17:51] i'm ordering the 2821 and they're paying for it, they just don't know it yet [17:52] we'll move the ds-1's over to it to shut 'em up, but it'll cost 'em =) [17:52] jlgaddis: that's a 2800 on your blog, no? [17:52] yea not much use for one, just surprised to find Cisco gear, we were all 3COM (eww I know) now all nortel [17:52] up_the_irons: the header? yeah [17:52] yeah, sprocket couldn't even come close [17:53] they still insist I should have been able to tell they'd run a FreeBSD VPS for me, by looking at a page that talks only about windows and linux. :) [17:53] but no worries [17:53] jlgaddis: i'd have to agree w/ the client; if even their redundant links go into the same equipment, it's not very redundant.. [17:53] I gave them a chance. :) [17:53] they completely blew it off. :) [17:54] I mean seriously - http://www.sprocketnetworks.com/virtual.htm - does that imply FreeBSD anywhere? [17:54] jeev: my bro filled out a credit card app and put "W. Axl Rose" on it, and sure enough, he got a credit card that read "W. AXL ROSE" ;) [17:54] damn [17:54] jeev: so go get your jack bauer card [17:54] you can get paypal to send you a jack bauer card honestly [17:54] up_the_irons: oh they have a valid point, sure. but they have a spare sfp and card on site, so it's not a huge deal. [17:54] paypal even sent me a card with my name mispelled [17:54] they must retype it somewhere [17:54] and not just use electronic data [17:55] besides, we're running gigabit over the fiber, and their "backup" is only 3xT1 [17:55] RandalSchwartz: search for "freebsd" turns up nothing on that page [17:55] and those will be at 100% if/when it fails over anyway because of all the students [17:57] http://www.google.com/search?q=freebsd+site%3Asprocketnetworks.com&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:unofficial&client=firefox-a [17:57] RandalSchwartz: ^^ looks to me FreeBSD is only an option on a dedicated server [17:57] damn, gotta run to work and scp the snow leopard disc image to my macbook. bbl. [17:58] later [17:59] *** ballen_ has joined #arpnetworks [17:59] *** ChanServ sets mode: +o ballen_ [17:59] *** ballen has quit IRC (Nick collision from services.) [17:59] *** ballen_ is now known as ballen [18:18] w00t [18:19] and i did have a 501 in the bottom of a my desk. and a 5505 on top of it too, apparently. [18:20] http://www.flickr.com/photos/jlgaddis/4249935434/ [18:20] hah nice [18:21] hey, im trying this slack lvm, raid1 md0 xen set up one more time.. if it doesn't work, if anyone wants my liver.. i dont drink [18:25] i could use a new one [18:26] ok i'll put you on the waiting list [18:29] *** dbgi3 is now known as dbgi [18:32] hmm. should serial console show me a login prompt? [18:32] or do I have to enable soemthign special? [18:33] it accepts my serial password [18:33] and says [Enter `^Ec?' for help] [18:33] but no login prompt [18:33] and yet, the box is up and running. [18:35] press enter [18:35] ? [18:35] yeah, press enter [18:35] I have, no response [18:35] RandalSchwartz: /etc/ttys is your friend [18:36] and yet, red is up and running just fine [18:36] uncomment the /dev/ttyu0 line [18:36] getty on serial line is not enabled by default on any FreeBSD that I know of [18:36] it's uncommented [18:36] nor OpenBSD [18:36] RandalSchwartz: err, i mean.. [18:36] ttyu0 "/usr/libexec/getty std.9600" dialup off secure [18:36] RandalSchwartz: turn it from "off" to "on" and set the terminal from "dialup" to "vt100" [18:36] serial console is enabled by default on sparc64 [18:36] jlgaddis: nice [18:37] but i imagine most users don't run into that often =) [18:37] right :) [18:37] and then signal something, right? [18:37] sighup init, at a guess [18:37] RandalSchwartz: yep, but what you signal, i'm not sure [18:37] kill -HUP 1 [18:37] well - something like that shuts the system down [18:37] at that point, i always had to reboot the VM anyway for some other reason, so never worried about it [18:38] kill -HUP 1 says init(8) [18:38] here goes [18:38] \o/ [18:38] \m/ [18:38] aha! [18:38] prompt! [18:38] neato [18:38] and there I am, logged in [18:38] * up_the_irons puts "kill -HUP 1" into his notes [18:39] now how does this set me up for SSH VNC? [18:39] oh - I can tunnel on this [18:39] duh [18:39] yup [18:39] hit 't' for instructions [18:39] * up_the_irons thinks it's "t" [18:39] yup, "t" [18:39] in the menu, that is [18:39] where... at teh menyu? [18:39] one of those "obvious" things [18:40] "t" for instructions [18:40] "j" for tar w/ bzip2 [18:41] RandalSchwartz: the "c" option is to interact with your VM's serial line, most often used to run a login getty; the "t" option is to tunnel VNC over SSH and it simply displays instructions on how to do it [18:42] ugh - 5555 - 5900 is... :) [18:43] ok - crap, can't type a minus sign [18:43] can't use this tunnel [18:43] PLEASE realize that VNC is traditionally 5900-5999 [18:43] ports outside this range cause some UIs no end of problems [18:44] 5555 for example is WRONG. flat out WRONG [18:44] RandalSchwartz: you can pick any port you want [18:44] RandalSchwartz: port 5555 was just the example [18:46] yeah, but its' the cut-n-paste example [18:46] put 5955 in there instead [18:46] then I can type "55" for my VNC port [18:46] which in fact, did work [18:47] that's fine, changed [18:47] but your choice of 61xx for VNC otherwise as also confusing. :) [18:47] for my unencoded VNC [18:47] anyway, nice to know that I now have a tunneled console, thanks [18:47] all is good [18:47] RandalSchwartz: 5900 - 5999 is not enough, that's only 100 users [18:47] you have more than 100 users on this box? [18:47] or is it all choked through the same IP? [18:48] for *all* customers? [18:48] RandalSchwartz: no, but if I think in terms of individual boxes, I can't migrate VMs very easily, because their VNC port numbers will clash [18:48] I have a "pool" of boxes and within that pool, the VNC ports do not overlap [18:49] different pools can have overlapping ports [18:49] understood [18:49] * RandalSchwartz dials down the bitching [18:49] LOL [18:49] so far, very happy [18:49] glad to hear [18:50] I might exceeed inbound bandwidth for first month [18:50] transferring crap from old box [18:50] after that, it'll settle down [18:50] roger [18:51] I have a very carefully crafted rsync script which *should* be getting all items of interest from old box [18:51] * jlgaddis worked out a nice backup solution [18:51] rsync to off-site box, doing a backup there using plain ol' tar, and a backup to s3 using tarsnap [18:52] it's 44GB of data from the old box, but I'll need to refresh any changes once or twice more before turning old box off [18:52] jlgaddis: i've found sending tarballs to S3 quite unreliable; it's like some get there and some just don't. i split them up at the 5G mark of course, also, but to no avail [18:53] jlgaddis: so if you do that, just keep on eye on it every now and then; make sure what you think is on there is really on there [18:55] i'm using colin's tarsnap. it actually goes to his servers first, once it's safely there it's transferred (by his software) to s3 [18:55] and, yeah, i checked it from another box after the first full backup =) [18:55] <-- learned a looooong time ago to test backups =) [18:56] sending ZFS snapshots is my new preferred offsite backup [18:57] someone should set up a disk farm to receive those [18:57] there's a business model :) [18:57] up_the_irons? :) [18:57] * up_the_irons is on it [18:58] seriously... if you could give me a place to send ZFS incrementals that was across the country from you, I'd pay [18:58] because a ZFS snapshot is atomic [18:58] and a ZFS incremental is the minimal change [18:59] I have very limited experience with ZFS. What exactly does a snapshot consist of? Is there a reason you can't just sent it to S3? [18:59] when I think of business models, I try to see who can crush it. And right now, with storage, Amazon is crushing a lot of models [18:59] a zfs snapshot is an exact moment-in-time grab [18:59] it costs NOTHING to make [19:00] very cool [19:00] and then you can "zfs send | ssh remote zfs recv" it [19:00] but the next step is... [19:00] make a new snapshot [19:00] then send only the diffs from old to new to remote [19:00] and it understands the diffs [19:00] so you have now moved from atomic point 1 to atomic point 2 [19:00] ah I see [19:01] so zfs needs a way to see the old diffs also [19:01] and because it's at the FS level, it's far more efficient than rsync [19:01] yeah [19:01] yeah [19:01] interesting [19:02] ... http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide [19:02] RandalSchwartz: so what would be the requirements? disk space (obviously), ssh shell ?, zfs on destination ?, ... [19:02] * up_the_irons puts that on the reading list [19:02] ... http://www.markround.com/archives/38-ZFS-Replication.html [19:03] yes - the disk sadly woudl be the same [19:03] you're making a clone, and keeping it up to date [19:03] but then you can also sell that service [19:03] "we have hot machine spares offsite" [19:03] or you could just say "your disk is cloned here, we can move your entire data wherever you want on a fault" [19:04] RandalSchwartz: so you would need a ZFS formatted fs of the same size at the backup site, along with SSH shell access? [19:04] yeah [19:04] anything else? [19:04] well - depends on if it's failover or just access [19:04] if it's failover, same CPUs [19:04] if it's just access, a way to push that data somewhere [19:04] you could sell two levels [19:05] I mean, I'd pay for just access [19:05] but for some business, they'd want COB level [19:05] so they'd pay for a hot spare machine [19:06] failover could be problematic because the IPs in Los Angeles would be different than, for example, New York, unless I get a long haul, which would get *quite* expensive. or even with newer MLPS "virtual" links between metro areas, it still is kinda expensive; although something worth looking into [19:06] well - you failover at DNS level [19:06] so you require a DNS change for switch [19:06] ah gotcha [19:06] not a big issue [19:06] this is catastrophe planning [19:06] not hot switching [19:07] roger [19:07] I want my data in Los Angeles and "somewhere else" [19:07] or [19:07] I want my data and CPU in Los Angeles and "somewhere else" [19:07] in the former, I can accept a few hours downtime [19:07] in the latter, I can accept TTL downtime :) [19:07] yeah [19:07] hah [19:08] make sense? [19:08] yup [19:08] this would be a great service for your customers [19:08] and great for the SOHOs [19:08] who need net presence, and some sense of COB [19:09] ... http://www.markround.com/archives/38-ZFS-Replication.html [19:09] right [19:09] yeah i'm looking at that one [19:10] oh - did I paste that already :) [19:10] damn - did [19:10] * RandalSchwartz pushes the drink away [19:10] failover using dns == fail [19:10] jlgaddis - suggestion? [19:11] by using DNS, I say "www.stonehenge.com = 1.2.3.4 not 4.5.6.7" [19:11] and that'll take TTL to clean up [19:11] yes, but not that aren't costly [19:11] so for TTL, I lose hits [19:11] oh well :) [19:11] * jlgaddis nods [19:11] i guess if you're not losing revenue, it's not a big deal [19:12] well - that's why my TTL is 15 minutes :) [19:12] yeah - it's a trade [19:12] i've never really liked DNS failover, but if you can take the TTL downtime, i suppose it works [19:12] my site is a brochsure site [19:12] not a live site [19:12] and twitter was down far more often than me :) [19:12] and I can never trust other people's resolvers to respect my TTL [19:12] you have to trust things at some level :) [19:13] if you really want protection, buy an offsite traffic director [19:13] RandalSchwartz: indeed, twitter being down often and facebook getting API DoS'd has lowered everyone's expectation of uptime ;) [19:13] so traffic goes to minnesota, then LA [19:13] or minnesota, then "other place" [19:13] so yes, there are solutions [19:13] we have a hot site several hundred miles away [19:14] or even better... yes, www.stonehenge.com points to second site, which mostly proxies to first site [19:14] until first site goes dow [19:14] then it takes over primary [19:14] the nice thing about zfs snaps is they're very very cheap to make and send [19:15] RandalSchwartz: i presume that something like MySQL, on top of ZFS, would not necessarily "snapshot" well since it could be in the middle of a transaction; in that case, could a better practice be to replicate MySQL some other way, or possibly not even use MySQL at all (ala the popularity of distributed KV data stores now) [19:15] I've heard people sending them in tight loops [19:15] "make snapshot" "send it" "make snapshot" "send it" [19:15] up_the_irons - NO [19:15] that's precisely the cool point of zfs snaps [19:15] it's ATOMIC [19:15] it can reproduce the disk PRECISELY midstream [19:16] so if mysql can handle a random reboot, then it can handle the snapshot restart [19:16] if it can't, stop using mysql [19:16] I know postgresql uses WALs, and does this just fine [19:16] as in, the current transaction will likely fail [19:17] but everythign committed up to that is done [19:17]