| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| RandalSchwartz | nope - no grok or grope
it was there, but not being used at all darn it - some mailing list I was on was looking for alternatives to easydns beacause they thought that group had pulled plug on wikileaks now I know it's not true, so I want to fix them one of my million mailing lists :0 | [00:42] |
| found it | [00:49] | |
| .... (idle for 17mn) | ||
| *** | LT has joined #arpnetworks | [01:06] |
| nakano is now known as nakano_ | [01:15] | |
| ............... (idle for 1h14mn) | ||
| schmir has joined #arpnetworks | [02:29] | |
| ......... (idle for 44mn) | ||
| _Ehtyar has joined #arpnetworks | [03:13] | |
| .... (idle for 18mn) | ||
| Ehtyar has quit IRC (Remote host closed the connection) | [03:31] | |
| ................... (idle for 1h30mn) | ||
| ziyourenxiang has joined #arpnetworks | [05:01] | |
| ................ (idle for 1h16mn) | ||
| schmir has quit IRC (Remote host closed the connection) | [06:17] | |
| ...... (idle for 28mn) | ||
| fink has joined #arpnetworks
fink has quit IRC (Client Quit) fink has joined #arpnetworks | [06:45] | |
| ...... (idle for 29mn) | ||
| DDevine has quit IRC (Remote host closed the connection) | [07:17] | |
| ........ (idle for 39mn) | ||
| jpalmer | man I love these proliant dl360's. 4 of them sitting in a rack next to my desk. when they are idle (which they are when I'm configuring them) they are absolutely silent. | [07:56] |
| *** | fink has quit IRC (Quit: fink) | [07:58] |
| vcs | wow jpalmer
usually when configuring they are on full blast i freaking hate that | [08:02] |
| jpalmer | the dells are like that. when you first turn the R410's on, for like 2 minutes, you won't hear anything else. | [08:03] |
| .... (idle for 15mn) | ||
| *** | fink has joined #arpnetworks | [08:18] |
| vcs | yeah
its annoying | [08:20] |
| whitefang | wow
those dl360's can have up to 192GB of RAM huh that's kinda insane | [08:29] |
| .... (idle for 17mn) | ||
| jpalmer | the only part I don't like about them, is the 2.5" disk. | [08:46] |
| whitefang | but you can fit more gigabytes with a smaller disk :P | [08:49] |
| *** | cubelogic has joined #arpnetworks
Sheath is now known as MuskyHusky | [08:52] |
| .... (idle for 17mn) | ||
| richardquisumbin has quit IRC (Ping timeout: 265 seconds) | [09:13] | |
| MuskyHusky | is there anything wrong with mercury
i cant ssh or vnc into it | [09:17] |
| *** | LT has quit IRC (Quit: Leaving)
d^_^b has quit IRC (Ping timeout: 265 seconds) ziyourenxiang has quit IRC (Quit: ziyourenxiang) cubelogic has quit IRC (Remote host closed the connection) | [09:19] |
| whitefang | what's mercury? | [09:32] |
| MuskyHusky | a server at arp | [09:35] |
| toddf | looks like at about 20 after my vps stopped responding
it is also on mercury | [09:39] |
| MuskyHusky | yep thats the time i see when my bot pinged out | [09:39] |
| toddf | did you email support@ at that time? | [09:40] |
| MuskyHusky | no | [09:40] |
| toddf | I just sent an email incase I'm the first | [09:40] |
| MuskyHusky | my friend just told me his bot pinged out so i went to investigate | [09:41] |
| ......... (idle for 43mn) | ||
| up_the_irons | hrm
i can't ping mercury either | [10:24] |
| jdoe | jdoe blames up_the_irons
jpalmer: yeah, I love HP gear. | [10:25] |
| up_the_irons | yup, kernel panic | [10:26] |
| dxtr | Why do this shirt have a pleasant smell? It should smell like shit
Oh well | [10:27] |
| *** | cubelogic has joined #arpnetworks | [10:37] |
| nerdd_ has joined #arpnetworks
nerdd_ is now known as nerdd | [10:45] | |
| up_the_irons | well, this is great, grub just hangs at "Loading GRUB..." | [10:52] |
| mike-burns | Can you get into the GRUB console? | [10:55] |
| jdoe | worst case, presumably it's just grub that got trashed somehow...
though I doubt you want to *go* anywhere ;) | [10:55] |
| up_the_irons | mike-burns: nope
sometimes "Error 15" pops up looks like it can't find a file will keep working... | [10:58] |
| mike-burns | Did you change partitions, kernel, mountpoints, etc? | [11:00] |
| toddf | at that stage of the game, the mbr is accessable but the rest of the disk not so much, check that the partition table didn't get munged somehow (I've had this happen to me in the past)
aka I've `recovered' by `from memory' recalling the sizes of the partitions. may not be your problem. yymv. just a thought. ;-) | [11:07] |
| .... (idle for 19mn) | ||
| *** | nerdd_ has joined #arpnetworks
nerdd has quit IRC (Ping timeout: 255 seconds) | [11:27] |
| nerdd_ is now known as nerdd | [11:38] | |
| jpalmer | jdoe: agreed. everyone complains about the price of HP gear, until they actually use it.
it's hard to beat the proliant line. "well, this similar spec dell is 1/3rd the price!" | [11:43] |
| whitefang | I've never owned Dell anything ^_^
we use dells at work though | [11:46] |
| jdoe | meh
anyone complaining about the price of HP gear has never bought from a non-whitebox vendor heh | [11:47] |
| jpalmer | I dunno, a Dell R410, is about half the price of a DL360 G6. configured similarly. and the main difference is only that the HP would use a 2.5" disk. the dell would use a 3.5"
well, t hat and the R410 is a bit limited when it comes to expansion. | [11:50] |
| ........ (idle for 36mn) | ||
| *** | cubelogic has quit IRC (Remote host closed the connection) | [12:28] |
| G | jpalmer: my experience w/ HP gear has been rather painful
jpalmer: give me IBM gear anyday | [12:38] |
| *** | vcs has quit IRC (Read error: Operation timed out)
whitefang has quit IRC (Read error: Operation timed out) | [12:49] |
| up_the_irons | Looks like there is some corruption on mercury's root fs :( The installer USB key I have in there does not provide enough rescue tools :( Will need to make a data center trip | [12:50] |
| *** | vcs has joined #arpnetworks
schmir has joined #arpnetworks | [12:52] |
| .... (idle for 19mn) | ||
| schmir has quit IRC (Ping timeout: 240 seconds) | [13:12] | |
| .......... (idle for 45mn) | ||
| jdoe | shitty :/ | [13:57] |
| *** | cubelogic has joined #arpnetworks
richardquisumbin has joined #arpnetworks | [13:58] |
| cubelogic has quit IRC (Remote host closed the connection) | [14:07] | |
| vcs | lol | [14:09] |
| ...... (idle for 25mn) | ||
| *** | whitefang has joined #arpnetworks
amdprophet has quit IRC (Quit: amdprophet) amdprophet has joined #arpnetworks | [14:34] |
| ............. (idle for 1h0mn) | ||
| up_the_irons | Host mercury has been restored and boots properly. We're doing last minute checks + kernel upgrade before starting VMs. | [15:34] |
| fink | what was the problem? | [15:35] |
| up_the_irons | fucking hell
corruption again hardware must be bad :( | [15:45] |
| toddf | memtest? | [15:57] |
| up_the_irons | memtest will take forever | [16:03] |
| toddf | you can let it run for x mins (1 usually spots really bad memory quickly)
but I'm sure you'll handle it in any event. bugging out for the evening. enjoy! | [16:04] |
| ...... (idle for 26mn) | ||
| jdoe | could be a controller too
could be almost anything :/ | [16:30] |
| *** | robotarmy has quit IRC (Remote host closed the connection)
Tranceo has joined #arpnetworks Tranceo has left | [16:42] |
| .... (idle for 17mn) | ||
| nuke- has quit IRC (Ping timeout: 265 seconds) | [16:59] | |
| .... (idle for 17mn) | ||
| DDevine has joined #arpnetworks | [17:16] | |
| DDevine | I'm guessing the new host that I am on is called Mercury. | [17:17] |
| jpalmer | DDevine: let me guess, outages? | [17:18] |
| DDevine | Actually, no!
Sometimes my SSH takes a long time to prompt for password like 30 seconds + | [17:18] |
| jpalmer | then, I doubt it. mercury is having some hardware issues it seems.
oh, slow ssh logins is usually do to bad rDNS | [17:18] |
| DDevine | Probably. I'll get around to fixing that soon.
I have already requested the transfer. | [17:19] |
| ..... (idle for 22mn) | ||
| *** | robotarmy has joined #arpnetworks | [17:41] |
| up_the_irons | DDevine: you're not on mercury | [17:45] |
| DDevine | up_the_irons: Yeah, just the delay on SSH made it look so. I'll fix up my rdns today probably. | [17:52] |
| *** | wwrl has joined #arpnetworks | [17:56] |
| .... (idle for 16mn) | ||
| wwrl | any updates on mercury? | [18:12] |
| up_the_irons | wwrl: still looks bad. I think the raid controller is bad (although it still appears to work), or something severely corrupted the disks (even the LVM volumes on which VPS' reside :(
some of my own VMs are on there, and if i try to mount the LVM volumes, i get bad blocks | [18:22] |
| fink | up_the_irons: backups? | [18:23] |
| up_the_irons | fink: i have backups of the host root fs, but individual VMs are not backed up (responsibility of the customer to backup their stuff) | [18:29] |
| fink | heh
am i on this machine? | [18:30] |
| up_the_irons | fink: don't know, PM your uuid | [18:32] |
| wwrl | if my VPS was on mercury, there's no chance that I could just have it moved to a different host machine, correct? | [18:35] |
| up_the_irons | wwrl: well, i'm trying to do that now as it is, but it is proving difficult | [18:39] |
| ..... (idle for 21mn) | ||
| wwrl: what is your vps uuid? (pm it to me) | [19:00] | |
| *** | fink has quit IRC (Quit: fink) | [19:04] |
| dxtr | up_the_irons: My VPS probably isn't on Mercury. There's no chance that I could have it moved there now that it's broken and stuff? | [19:07] |
| *** | vapor has quit IRC (Ping timeout: 272 seconds) | [19:18] |
| up_the_irons | dxtr: i hope you're joking ;) | [19:21] |
| whitefang | huh, looks like FreeBSD 8.2-BETA1 was released recently fwiw to anyone that follows the -STABLE branch. | [19:30] |
| up_the_irons: can you move me to mercury too plox?
what's special about mercury anyways other than its broken right now. | [19:36] | |
| up_the_irons | nothing, it's broken | [19:36] |
| whitefang | and why is it called mercury and not like...kvm14?
or is there a special name for all kvms and I just don't know the name of mine. | [19:37] |
| up_the_irons | whitefang: it was created before i had a naming convention | [19:39] |
| whitefang | you could call it kvm0 :P
or you could let me call kvm14 "venus" because it rhymes with a naughty word. | [19:40] |
| G | personally I call all the machines on my home network + VPS' after NZ Native Birds | [19:45] |
| whitefang | wow, that's a lot of machines named kiwi
whitefang ducks. | [19:47] |
| G | the machinethat I called Kiwi is now 'retired' | [19:48] |
| .......... (idle for 48mn) | ||
| DDevine | Silly flightless birds... | [20:36] |
| ......... (idle for 40mn) | ||
| whitefang | hmmm, I see the amount of bandwidth for my package under My Services, but I don't see my current usage.
am I blind? | [21:16] |
| ........ (idle for 35mn) | ||
| *** | _Ehtyar has quit IRC (Remote host closed the connection)
vcs has quit IRC (*.net *.split) toddf has quit IRC (*.net *.split) RandalSchwartz has quit IRC (*.net *.split) islandfox has quit IRC (*.net *.split) nuke^ has quit IRC (*.net *.split) richardquisumbin has quit IRC (*.net *.split) amdprophet has quit IRC (*.net *.split) koan has quit IRC (*.net *.split) azmarco has quit IRC (*.net *.split) awyeah has quit IRC (*.net *.split) cmeiklejohn has quit IRC (*.net *.split) Yamazaki-kun has quit IRC (*.net *.split) mike-burns has quit IRC (*.net *.split) phlux has quit IRC (*.net *.split) up_the_irons has quit IRC (*.net *.split) tooth has quit IRC (*.net *.split) whitefang has quit IRC (*.net *.split) mattx86 has quit IRC (*.net *.split) tuv has quit IRC (*.net *.split) infrared has quit IRC (*.net *.split) cedwards has quit IRC (*.net *.split) baklava has quit IRC (*.net *.split) blovett has quit IRC (*.net *.split) Dougy has quit IRC (*.net *.split) mtve has quit IRC (*.net *.split) Lefty has quit IRC (*.net *.split) N1JER has quit IRC (*.net *.split) MuskyHusky has quit IRC (*.net *.split) bitslip has quit IRC (*.net *.split) pete__c_ has quit IRC (*.net *.split) nipuL has quit IRC (*.net *.split) robotarmy has quit IRC (*.net *.split) hiro_dSn_ has quit IRC (*.net *.split) jpalmer has quit IRC (*.net *.split) mjp has quit IRC (*.net *.split) ww has quit IRC (*.net *.split) bsdguy has quit IRC (*.net *.split) dxtr has quit IRC (*.net *.split) bob^^ has quit IRC (*.net *.split) G has quit IRC (*.net *.split) jdoe has quit IRC (*.net *.split) DDevine has quit IRC (*.net *.split) mhoran has quit IRC (*.net *.split) BarberRonny has quit IRC (*.net *.split) heidar has quit IRC (*.net *.split) nerdd has quit IRC (*.net *.split) boogeyman has quit IRC (*.net *.split) wwrl has quit IRC (*.net *.split) tuvwx has quit IRC (*.net *.split) wwrl has joined #arpnetworks robotarmy has joined #arpnetworks DDevine has joined #arpnetworks amdprophet has joined #arpnetworks whitefang has joined #arpnetworks richardquisumbin has joined #arpnetworks vcs has joined #arpnetworks nerdd has joined #arpnetworks dxtr has joined #arpnetworks mattx86 has joined #arpnetworks cedwards has joined #arpnetworks bob^^ has joined #arpnetworks toddf has joined #arpnetworks blovett has joined #arpnetworks mhoran has joined #arpnetworks awyeah has joined #arpnetworks RandalSchwartz has joined #arpnetworks Dougy has joined #arpnetworks hiro_dSn_ has joined #arpnetworks ww has joined #arpnetworks baklava has joined #arpnetworks G has joined #arpnetworks BarberRonny has joined #arpnetworks mtve has joined #arpnetworks cmeiklejohn has joined #arpnetworks boogeyman has joined #arpnetworks jpalmer has joined #arpnetworks Yamazaki-kun has joined #arpnetworks mjp has joined #arpnetworks islandfox has joined #arpnetworks koan has joined #arpnetworks MuskyHusky has joined #arpnetworks jdoe has joined #arpnetworks Lefty has joined #arpnetworks bitslip has joined #arpnetworks pete__c_ has joined #arpnetworks heidar has joined #arpnetworks bsdguy has joined #arpnetworks mike-burns has joined #arpnetworks azmarco has joined #arpnetworks nuke^ has joined #arpnetworks phlux has joined #arpnetworks up_the_irons has joined #arpnetworks holmes.freenode.net sets mode: +oooo toddf mhoran mike-burns up_the_irons tuv has joined #arpnetworks N1JER has joined #arpnetworks infrared has joined #arpnetworks tooth has joined #arpnetworks tuvwx has joined #arpnetworks nipuL has joined #arpnetworks | [21:51] |
| ......... (idle for 42mn) | ||
| whitefang has quit IRC (Quit: http://lice.muppetz.com) | [22:40] | |
| whitefang has joined #arpnetworks
vapor has joined #arpnetworks | [22:50] | |
| ........... (idle for 51mn) | ||
| up_the_irons | <sigh...>
all data on mercury has been catastrophically lost emails sent to customers offering a fresh VPS (with two months of free service) fuck I'm going to have a long night... and day tomorrow i've never lost a box before, sigh... I guess this is the cost of doing business does ANYONE know an effective way of backing up LVM volumes that doesn't tax the fuck out of the machine? | [23:43] |
| whitefang | was mercury one of the first machines? | [23:45] |
| up_the_irons | whitefang: it was the very first | [23:45] |
| whitefang | that's unfortunate :/ | [23:45] |
| up_the_irons | it is the only machine that does not have the same hardware config as the others | [23:45] |
| vapor | up_the_irons, testdisk was no good?
http://www.cgsecurity.org/wiki/TestDisk did you use this? | [23:45] |
| whitefang | I thought a RAID array was supposed to stop things like complete data loss from happening. | [23:48] |
| up_the_irons | whitefang: if a single disk fails, yes, it is great for that. but what if the controller goes bad? that's what I think happened here
vapor: I'm passed that at this point. I *can* get a lot of data from the lost volumes, but it is very hit or miss. running an fsck produces so many errors it is unbelievable. one may be able to find individual files they need, but the filesystems will never boot | [23:49] |
| whitefang | is there any way to give the customers on that machine access to their specific filesystems do recover what they can? | [23:51] |
| up_the_irons | whitefang: it's possible, but not without some work; right now, i wouldn't have time for that, just need to focus on getting the base VMs up | [23:52] |
| whitefang | up_the_irons: yeah, of course...but it might soften the wrath if you can offer them said access to their possibly corrupted filesystems "soon".
and by "soon" I mean Blizzard(tm)'s sense of the word "soon". | [23:53] |
| tooth | whitefang, RAID is "high availability", not a "backup". :-) | [23:56] |
| up_the_irons | whitefang: yeah, you're right | [23:56] |
| whitefang | up_the_irons: just tell them jpalmer is personally responsible for getting them their old filesystems, he was just telling me how bored he was the other night. :P | [23:57] |
| up_the_irons | LOL | [23:59] |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |