| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | LT has joined #arpnetworks | [01:16] |
| .................................. (idle for 2h49mn) | ||
| ziyourenxiang has joined #arpnetworks | [04:05] | |
| ..... (idle for 24mn) | ||
| ziyourenxiang has quit IRC (Quit: ziyourenxiang) | [04:29] | |
| ..... (idle for 21mn) | ||
| RandalSchwartz has quit IRC (Ping timeout: 272 seconds) | [04:50] | |
| ..... (idle for 23mn) | ||
| heavysixer has joined #arpnetworks
ChanServ sets mode: +o heavysixer | [05:13] | |
| .............................. (idle for 2h29mn) | ||
| RandalSchwartz has joined #arpnetworks
RandalSchwartz has quit IRC (Changing host) RandalSchwartz has joined #arpnetworks | [07:42] | |
| ..... (idle for 24mn) | ||
| RandalSchwartz | up_the_irons - having problems with my box # 3 for neil
it comes up, then goes off the net kvr13 and I can't get to the console from there either anyone else on kvr13 with problems? | [08:06] |
| toddf | I presume the 'vnc server' is also the host system name the vps is run off of? | [08:09] |
| RandalSchwartz | ok, I'm sitting on the console now, after rebooting the vm
I'm gonna watch to see what goes wrong vnc logged in as root on console | [08:10] |
| toddf | zfs: malloc failed! | [08:12] |
| RandalSchwartz | looping with date; sleep 1
whoa. it just went bonkers it's still running internally, but all network activity stop | [08:12] |
| toddf | ping6 -n -w ff02::1%em0 ?
40 bytes from fe80::5054:ff:fe27:2122%em0: 0.v.freedaemon.com. 44 bytes from fe80::5054:ff:fe27:9007%em0: s3.lax.arpnetworks.com. thats what I get on mine, just to see if `anything else' is on the same net, though I'm sure pinging the default gateway (v6/v4) would tell you that as well | [08:12] |
| RandalSchwartz | even the vnc goes away at the same time
this isn't good this is very bad this is the kind of thing for which I need urgent support. :) ironic, since we talked about that yesterday. | [08:14] |
| toddf | vnc going away means the kvm process itself dies | [08:15] |
| RandalSchwartz | NO. already confirmed
the box is still *running* it's just network stack my cron jobs continue to fire off | [08:15] |
| toddf | I suspect serial console might shed some light if you can force it to spew console messages and the like to serial console
since serial console can't `go away' like vnc | [08:15] |
| RandalSchwartz | doesn't the serial console still use the network stack to talk to me? | [08:16] |
| toddf | if vnc goes away and your vps looses net .. almost sounds like the host nic has connectivity issues .. | [08:16] |
| RandalSchwartz | Yeah... that's why I think this is an up_the_irons issue, not mine | [08:16] |
| mattx86 | I could see if I have problems.. I'm on kvr13 as well | [08:17] |
| toddf | well, true, serial console uses a tcp connection from a local vps to your kvm instance's serial port on a tcp port
however, if it did disconnect, at least you'd see the last message as opposed to vnc going 'blip' and disappearing again, kvr13 .. is that your vnc server hostname? if so, I'm on `mercury' thus can't help ;-( | [08:17] |
| RandalSchwartz | yes
it's not like there's a lot of web activity or anything either | [08:18] |
| up_the_irons | ugh
See my latest tweet on @bsdvps | [08:18] |
| *** | Hien has joined #arpnetworks | [08:19] |
| toddf | bsdvps: Host "kvr13" experienced kernel memory corruption earlier this morning. We performed emergency maintenance: kernel upgrade and reboot | [08:19] |
| RandalSchwartz | aha.
that's me and that's why I'm having troubles is it all good now? neil's website was down for two hours he hates that | [08:19] |
| up_the_irons | There were a bunch of conntrack over limit errors on the serial port, yet I saw almost no traffic go to the box. Could not get anything on the monitor. Box still appeared alive b/c it was logging to serial. After working with it for about 10 minutes, I deemed the only thing left to do was a reboot | [08:20] |
| RandalSchwartz | already ragging on me to "move chris somewhere else" | [08:20] |
| toddf | bah, I even have a window open from 'microblog-purple' in pidgin that said so at 13 after, 10 mins later, I still hadn't noticed. bah! | [08:20] |
| RandalSchwartz | is there anything I can tell neil about how rare this is?
I really don't want to spend a lot more time planning for something that has enough 9's normally | [08:21] |
| mattx86 | yeah, i cant connect on kvr13 either | [08:22] |
| up_the_irons | RandalSchwartz: this is the first time i've seen something like this in 18 months (basically, since i started with this product line) | [08:22] |
| RandalSchwartz | ok - I passed that along to him
I don't know if he knows how much more expensive 5 9's is over 4 9's :) | [08:23] |
| toddf | randal: heh, like load balanced vps's on separate kvm hosts? ;-) | [08:24] |
| RandalSchwartz | what we'll probably do is set up some sort of alternate hosting for the brochure web site and loadbal it, yeah | [08:24] |
| up_the_irons | mattx86: your VPS should be coming up now | [08:27] |
| mattx86 | up_the_irons: yep, there it is | [08:27] |
| RandalSchwartz | if kvr13 keeps having problems, can you migrate us fairly rapidly?
or do you have to copy disk around to do that? not sure how your disk-to-cpu mapping works | [08:31] |
| mattx86 | would it be possible that the bug I discovered in mikrotik's routeros product would have affected the host in any way? | [08:33] |
| up_the_irons | RandalSchwartz: i have to copy the disk to another host; doesn't take all that long | [08:33] |
| toddf | disks are local to each kvm host | [08:33] |
| up_the_irons | mattx86: i would think not, but ya never know | [08:33] |
| RandalSchwartz | yeah, it's only about 30G for that box
my settings wouldn't change though, right? except for my virtual console access | [08:33] |
| mattx86 | true | [08:34] |
| RandalSchwartz | Neil's saying "oof" about the two hour downtime | [08:34] |
| up_the_irons | RandalSchwartz: vnc host would change, that is about it | [08:35] |
| RandalSchwartz | well - please help me out by keeping on top of this, and feel free to migrate me if needed, even if it means a few minutes downtime
gotta keep neil happy freebsd responds nicely to graceful down. takes about 30 seconds | [08:35] |
| up_the_irons | RandalSchwartz: definitely going to stay on top of this | [08:40] |
| RandalSchwartz | I need to figure out why nagios didn't tell me either
I must have something configured wrong it notified me about ssh down and up, but not ping or website and ssh is apparently "under" ping, so if ping is down, that is supposed to control all other notifications but I still should have got the ping-down notice | [08:41] |
| up_the_irons | if you get a HOST DOWN message, no services on that host are checked further | [08:43] |
| up_the_irons wanders off | [08:48] | |
| ...... (idle for 28mn) | ||
| *** | LT has quit IRC (Quit: Leaving)
cedwards has quit IRC (Changing host) cedwards has joined #arpnetworks | [09:16] |
| ....... (idle for 33mn) | ||
| Hien | it was raining outside...
a little cold.... | [09:52] |
| ....... (idle for 30mn) | ||
| *** | Hien has quit IRC (Quit: Page closed) | [10:23] |
| RandalSchwartz | oh bah. someone thought it clever that freebsd box admins should only be notified during normal working hours in the default nagios config
everything else was 24x7. Dumb. I erased that puppy right away, reloaded nagios | [10:29] |
| IPv6Freely | morning doods
<3 nagios | [10:29] |
| RandalSchwartz | but that explains why I didn't get my down notice for the 6am fault | [10:29] |
| IPv6Freely | yeah, i wish it would queue notifications or something, so if youre set to 9-5 and a fault is at 6am, it will queue it and notify you at 9am about it
maybe there is a way to do that, but i havent found it | [10:30] |
| RandalSchwartz | if it's still a problem at 9am, don't you get the notice anyway? | [10:32] |
| jpalmer | you'd get teh next 'reminder" notification, yes.
but, it may not be immediately at 9. for instance, you have 9-5 notifications. you have a failure at 8:55, and a 2 hour reminder.. you may not get the reminder until 10:55ish | [10:46] |
| ......... (idle for 43mn) | ||
| IPv6Freely | yea
itd be nice if the 6am event would go out immediately when the notification timeperiod starts for that reason i just had everything 24x7 but page/sms only went out 9-5 but email worked 24x7 so id at least see it immediately when i woke up | [11:30] |
| jpalmer | IPv6Freely: thats generally what I do, as well. | [11:33] |
| IPv6Freely | this job is a fucking waste of time
i show up to install a new juniper network... and they havent even decided if they are ACTUALLY going juniper, going cisco, or going some other noname garbage ive never heard of... not to mention the lack of an MPLS circuit being ordered yet so im sitting here... | [11:34] |
| mattx86 | wow
that sucks | [11:46] |
| ......... (idle for 42mn) | ||
| *** | hsien has joined #arpnetworks | [12:28] |
| hsien | yo | [12:28] |
| jpalmer | yo back | [12:28] |
| hsien | cool | [12:28] |
| toddf | IPv6Freely: so install OpenBSD on a pc of theirs and setup MPLS for them while you wait.. | [12:39] |
| IPv6Freely | toddf: hahahahahhahaa
oh man thatd be funny | [12:49] |
| toddf | toddf grins | [12:55] |
| IPv6Freely | LOL the cisco quote came back
303k for one router. | [12:55] |
| mattx86 | that's rich :> | [12:56] |
| jpalmer | yikes, I'm hoping they are doing some SERIOUS traffic. | [12:56] |
| IPv6Freely | very little actually... we're just tryng to quote hardware comparable to the juniper equivalent
303k for one cisco router. They need four of them. 103k for four juniper SRX650s, and four EX4200s. ...yet theyre leaning cisco IPv6Freely facepalm | [12:56] |
| toddf | wow, and I thought comparing openbsd to an $18k ASA for IPSec was a rather large price difference, maybe I should learn more about MPLS | [12:57] |
| jpalmer | very little traffic? I have a 2611, and a 2621xm I'll happily ship you (overnight, morning delivery) for $250k. look, a discount! I'll even buy you a smartnet contract. | [12:57] |
| IPv6Freely | heh
hell, one single 10x1gigE module for the Cisco ASR1000 is $17,700... and this router needs 5 of those | [12:57] |
| jpalmer | interesting. my VPS rebooted a couple hours ago. | [13:07] |
| RandalSchwartz | jpalmer - you on kvr13? | [13:07] |
| jpalmer | RandalSchwartz: yes | [13:07] |
| RandalSchwartz | ... http://twitter.com/#!/bsdvps/status/25988394637
memory corruption | [13:07] |
| jpalmer | I was wondering why my mysql DB had corrupted itself. now I know ;) | [13:08] |
| RandalSchwartz | the WTF there is "mysql - why?" | [13:08] |
| IPv6Freely | heh | [13:08] |
| RandalSchwartz | something legacy that demands mysql instead of a sane database? | [13:08] |
| jpalmer | yes | [13:08] |
| IPv6Freely | i should probably follow that account | [13:08] |
| RandalSchwartz | sad to hear that | [13:08] |
| jpalmer | I'd much rather be using postrge. but, eh. it is what it is. | [13:09] |
| RandalSchwartz | postrge? :) | [13:09] |
| *** | schmir has joined #arpnetworks | [13:09] |
| RandalSchwartz | maybe you mean postgres | [13:10] |
| jpalmer | ugg, I hate when i see others type that. why the hell did I just do it :P | [13:10] |
| RandalSchwartz | postgres aka postgresql and never postgre
and sometimes pg :) | [13:10] |
| toddf | and definately not postrge either | [13:10] |
| RandalSchwartz | or mysql :) | [13:11] |
| jpalmer | pg is acceptable. postgresql is acceptable. the rest, not acceptable. | [13:11] |
| RandalSchwartz | postgres is also typical
just not postgre | [13:11] |
| jpalmer | so is "u" but thats not acceptible either. ;) | [13:11] |
| RandalSchwartz | and it's "post gres cue ell"
not "post grey see quel" - which grates my ears if Monty gets it right with "my ess cue ell", the pg people can learn to say it right too | [13:11] |
| jpalmer | I remember when they first put up the mp3/wave of the pronunciation. heh it obviously bothered a few people :P | [13:12] |
| RandalSchwartz | people who say "my see quel" should be shot
it's... like typing Perl in "CAPS" :) | [13:12] |
| jpalmer | PERL!
jpalmer stops | [13:12] |
| RandalSchwartz | a clue of the clueless :)
or as that comedian would say, "here's your sign!" | [13:12] |
| jpalmer | I was joking, but have no problems admitting.. I'm clueless with perl. it makes my eyes bleed. | [13:13] |
| RandalSchwartz | ... http://en.wikipedia.org/wiki/Bill_Engvall | [13:13] |
| toddf | its close to C but without the buffer overflows (just leaks memory if you don't watch it..) | [13:13] |
| IPv6Freely | i love making words out of acronyms. MPLS is MIPPLES
heh :P | [13:13] |
| jpalmer | if I said tha tin the office, I'd get some looks. | [13:14] |
| IPv6Freely | haha
The best one is "e-grep" for EIGRP where the hell do you get "grep" from that? I also love ackels for ACLs by love, i mean it makes me want to punch babies | [13:14] |
| RandalSchwartz | RandalSchwartz wanders to a different location | [13:15] |
| mattx86 | :) | [13:17] |
| ...... (idle for 29mn) | ||
| dxtr | I wonder where up_the_irons have been. started a support ticket a couple of days ago and he answered once :P | [13:46] |
| .... (idle for 15mn) | ||
| toddf | interesting, https://bugzilla.redhat.com/show_bug.cgi?id=508801 suggests mpbios issues in OpenBSD vs kvm can be resolved by the bios being updated inside qemu-kvm .. heh! | [14:01] |
| ........... (idle for 54mn) | ||
| *** | schmir has quit IRC (Remote host closed the connection) | [14:55] |
| .............. (idle for 1h6mn) | ||
| dxtr | up_the_irons! | [16:01] |
| mattx86 | fwiw, they're sold out again with new hardware on the way
that'll keep up_the_irons busy, or to a certain extent anyways | [16:10] |
| ........ (idle for 35mn) | ||
| *** | ficovh has joined #arpnetworks
ficovh has quit IRC (Client Quit) | [16:46] |
| ................................. (idle for 2h41mn) | ||
| up_the_irons | dxtr: i've been resting a lot this week b/c I got sick, so support is a little slow | [19:29] |
| .... (idle for 18mn) | ||
| dxtr: oh and i see you wanted the upgrade; yeah, those take a little time. i've mainly been sticking with the quick requests like rdns, until i feel better | [19:47] | |
| jpalmer | up_the_irons: I know someone asked the other day. in our VPS's, we have 2 cdroms. can we request 2 seperate iso's be permanently mounted, or is thee second one unused? | [19:55] |
| mattx86 | up_the_irons: ah, that makes me more understanding. I'm sorry about giving you a hard time earlier.
hope you get to feeling better | [20:00] |
| up_the_irons | mattx86: :)
jpalmer: the 2nd cd-rom is a kvm bug i think. there's only 1 in the VM config. Technically, I *could* configure two if you wanted up_the_irons wanders of *off | [20:14] |
| jpalmer | up_the_irons: gotcha. I don't personally need it, but I've seen it asked before. good to have an answer
up_the_irons: thanks for the answer. get some chicken soup and rest. | [20:15] |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |