[00:01] *** novae has quit IRC (Remote host closed the connection) [00:06] *** jlgaddis has joined #arpnetworks [00:06] *** jlgaddis has quit IRC (Changing host) [00:06] *** jlgaddis has joined #arpnetworks [00:07] *** novae has joined #arpnetworks [00:29] *** jcv has quit IRC (Ping timeout: 265 seconds) [07:13] *** ben1 has joined #arpnetworks [07:17] *** merc2 has quit IRC (Ping timeout: 276 seconds) [11:11] *** zeshoem has quit IRC (Remote host closed the connection) [11:12] *** zeshoem has joined #arpnetworks [11:52] *** milki has quit IRC (Remote host closed the connection) [12:03] *** milki has joined #arpnetworks [12:03] i hosed my vps. no sudo and no root password -.- [12:09] * milki kicks gettext [12:36] no rsynca nd git too -.- [12:37] my best bet is probably...single user mode to reset the root passwd [13:03] can you get on the console? [13:06] ya i have console [13:06] but i only have my normal user login [13:07] not sure what other options i have that will get me root [13:08] alright, got vnc working [13:09] lets see if this will get me boot screens [13:10] excellent [13:11] su? [13:11] it's good to have sudo and su [13:11] oh no root password :/ [13:12] ya, i have a gpg encrypted file that contains the random password i created [13:12] i dont remember the password for that file either >.> [13:12] but im in single user mode [13:12] i should be able to recover root with this [13:14] fun :) [13:27] yay sudo works again -.- [13:36] *** milki has left [13:36] *** milki has joined #arpnetworks [13:45] how did you break it? [15:16] *** joepie91 has quit IRC (Disconnected by services) [15:18] *** joepie91_ has joined #arpnetworks [15:57] s/break it/break in [15:57] how did you break in? [15:59] i meant break being able to login [17:49] *** dj_goku has quit IRC (Ping timeout: 264 seconds) [17:52] *** dj_goku has joined #arpnetworks [18:00] *** dj_goku has quit IRC (Ping timeout: 245 seconds) [18:02] me? [18:02] That's what she said!! [18:02] mercutio: pkg delete gettext [18:02] ! [18:02] i keep on forgetting that it breaks sudo [18:02] oh what [18:02] and it doesn't warn you? [18:02] isnt' gettext gpl? [18:03] /usr/ports/UPDATING does warn you [18:03] but the command itself doesnt [18:03] needs more [18:04] *** dj_goku has joined #arpnetworks [18:04] *** dj_goku has quit IRC (Changing host) [18:04] *** dj_goku has joined #arpnetworks [18:04] hshs [18:05] [18:05] the exact text is [18:05] You must first delete the existing installation of gettext and then [18:05] reinstall it. This will break sudo, so you *must* do this in a root [18:05] shell (sudo -i) if you use sudo. [18:09] but it doesn't tell you to type YES to conitnue or such? [18:09] nope [18:10] i dont recall there ever being a prompt for delete [18:10] only install [18:10] :P [18:36] FreeBSD is silly in that way. As important as UPDATING is, there is absolutely no beating you over the head with it before you break things. [19:14] sounds scary [19:30] is it possible to check if someone is in sudo rather than su? [19:32] What do you mean by "someone is in..."? [19:33] like has used sudo to get to root [19:35] /var/log/auth.log (or /var/log/secure, or... whatever else you have systlog configured for) [19:35] as the update script i meant [19:36] What update script? Did I miss something? [19:37] milki updated gettext and it broke sudo [19:39] Technically, a package could look to see if sudo is installed. However, this would need to be run before the package is installed, and it would need to abort the process somehow. I don't think that's possible. (But I'm not a FreeBSD packager...) [19:40] oh hmm [19:40] i should have just checked [19:40] SUDO_GID=1000 [19:41] it has environment variables [19:41] so i blame the update script :/ [19:41] Oh you meant checking to see if the present invocation is within sudo. [19:41] yeh [19:42] If I were logged in on a tty as root and ran pkg upgrade, the system would still be hosed [19:42] "hosed" [19:43] Or for those living more dangerously, the update could have been done by cron and the presence/use of sudo would not have been detected by $SUDO_ variables and he'd be right back where he is now. [19:43] (Except being chastised for running updates blindly from cron) [19:45] My point is that relying on checking that the update is being run from sudo is a poor way to check whether the update is going to hose things up. Heck, I'd bet that the ENV is sanitised before any package scripts are run, so there go the $SUDO_ variables. [19:46] well it's better than nothing [19:47] it could give a warning [19:54] *** bmacs has joined #arpnetworks [19:56] graphs.arpnetworks.com seems to be ill right now. [19:57] hmm can't establish secure connection for me [19:58] up_the_irons: ^ [19:58] The http anf https connections both just hang it seems [19:58] yeah same here [19:59] too much graphing, not enough web serving [20:01] The box returns pings though. weird. [20:01] it accepts tcp connection too [20:02] ah, it accepts the connection then jsut doesn’t say anything back? [20:02] yeah [20:02] curl -v https:// [20:02] weird. [20:02] probably apache got into a bind [20:03] but that's just a guess [20:03] Sounds like a good guess to me. [20:04] But, Apache is a web server. It has no business being a BIND. No wonder it's having issues... [20:05] yay for overloading words [20:09] fwiw, sent up_the_irons an e-mail, in case he's not watching IRC and no alerts were triggered (the host pings, the port is open... which covers 90% of service check configurations out there) [20:12] smart thinking [20:13] * brycec tries to be smart [20:13] * brycec has also setup monitoring before and been stung by situations like this [20:13] Which reminds me... I really need to setup host monitoring for a client :/ [20:14] What does everyone here use for monitoring? I was looking into nagios but it seems overly complicated. [20:14] xymon is lighter weight [20:14] but ugly :/ [20:15] http://xymon.sourceforge.net/ [20:15] I like the green laserbeam down the side of their page. haah [20:15] haha [20:17] i find it less annoying than zabbix [20:17] zabbix is a huge slow resource hog [20:18] xymon with a facelift could be pretty awesome [20:18] Yeah, I want the kind of thing that could be displayed on a large screen 24/7 in the office so that CEO types think we know what we are doing [20:19] pingdom? [20:19] huh. pingom doesn’t look too bad. looks like it could get pricey though. [20:19] sysadmin is one of thoese areas where people don't know what you do unless things go wrong :/ [20:20] but ceo types can pay for it? [20:20] good point. [20:20] you can get free single host trial [20:20] last i looked web interface seemed to get nicer [20:20] and a lot of the free options tend to have pretty clunky interfaces [20:20] well all of them that i know of :/ [20:21] the kind of people who decide to write free open source monitoring don't seem to be the type to be good at design. [20:21] I’ve noticed. it’s weird. the code can be great, but the design often falls short. [20:22] yeh or downright shocking :/ [20:22] i like zabbix because it's fast and cross platform. [20:22] Pingdom seems to only be able to monitor publicly facing sites. [20:22] but i've only ever used it for monitoring load etc rather than for alerts. [20:22] i use nodeping.com for alerts. [20:22] and spong [20:23] ahh yeah that's for extnerla monitoring [20:23] what didn't you like about nagios? [20:23] it's probablythe most popular for medium to large sites using open source monitoring. [20:23] for smaller sites you care more that configuration is confusing/annoying/etc. [20:24] cacti isn't bad too. [20:24] it just means you need to export via ipmi from other hosts. [20:24] ipmi [20:24] uhh i mean snmp [20:25] Yeah, cacti is pretty good. maybe that’s what I should just use and expose server load and such via snmp [20:25] maybe my only problem with nagios is that it looks a little clunky. I should probably play with it and see how I like it. [20:26] it is a little clunky [20:26] but it's functional. [20:27] yeah. it must be if it’s so popular. [20:32] *** toeshred has quit IRC (Read error: Connection reset by peer) [20:42] bmacs: Recently I've used Opsview. They have a free offering, and it's built atop nagios, reasonably easy to setup. [20:42] I wonder if BigBrother is still around... [20:43] I’ll check it out. [20:43] xymon takes big brother plugins [20:43] i know the look is pretty ick, but it is pretty functional :/ [20:44] http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems [20:44] Comparison of network monitoring systems :: The following tables compare general and technical information for a number of network monitoring systems. Please see the individual products' articles for further information. Features Legend Product Name The name of the software, linked to its Wikipedia article. IP SLAs Reports Support of Cisco's IP Service Level Agreement mechanism. Logical Grouping Supports arranging the hosts or [20:44] hmm [20:45] I’m just going to install every one of those. haha [20:45] haha [20:46] trend prediction seems to be uncommon [20:46] distributed monitoring is surprisingly common [20:47] 2015-01-18 20:16:45 mercutio zabbix is a huge slow resource hog2015-01-18 20:21:42 mercutio i like zabbix because it's fast and cross platform. [20:47] I'm confused, mercutio, is it fast or slow? [20:47] oops [20:47] i meant xymon the second time [20:47] sorry i'm overheating [20:47] i know it's hard to understand from where you are :) [20:47] lol [20:47] overheating? [20:48] yeh it's really hot here :/ [20:48] @weather auckland [20:48] Auckland, New Zealand: Clear 81°F (27°C), Humidity: 66%, Wind: Calm -- For more details including the forecast and almanac, see http://www.wunderground.com/cgi-bin/findweather/getForecast?query=-36.973896,174.878021 or re-request this with: @weather -v auckland [20:48] hmm that doesn't /sound/ so hot, but it feels hot :/ [20:49] xymon is c rather than perl [20:49] btw bmacs, afaik Cacti is graphing/logging only, no alerting. [20:49] Hmm. yeah. [20:49] monit is worth considering, depending on your exact scenario [20:49] also it can still be good to use more than one monitoring thing [20:49] It's same-host, but can manage processes, restarting or performing other actions as necessary [20:50] ^++ [20:50] like i use cacti to check load averages, but i never alert on load averages. [20:50] i just use it to see when things get worse etc [20:50] or for trends. [20:50] Ooh and munin is nice too [20:50] but like if web hosting, you can alert based on how quickly a page loads. [20:51] so many options :) [20:51] I use so many of them... [20:51] graphing page load times would be pretty sweet. [20:51] heh brycec get him to use smokeping too :) [20:51] bmacs: you can do graphing of page load times with curl too [20:51] err with smokeping/curl [20:51] truth ^ [20:52] (There's some alerting to it too) [20:52] brycec: oh? [20:52] But I don't use alerting, so I can't speak to it. [20:52] i haven't tried the alerting :) [20:53] Me either :p [20:53] * mercutio makes mental note to look into more monitoring again sometime [20:54] i still want soemthing that will deduce where there are issues [20:54] it's kind of a complex problem though. like a few days ago? arp had some loss on ntt due to ddos, but it seemed that coresite stuff was fine. [20:54] Nagios/opsview does that for network links [20:55] brycec: but does it do it for upstream of upstream etc? [20:55] like if it can be more particular that'd be nifty. [20:55] If host X is down, then it doesn't alert for downstream hosts Y and Z [20:57] yeah but it's more complicated than that [20:57] because sometimes there's partial connectivity from differnet locations etc. [20:57] and host y/z maybe on a different subnet to x and behave differently etc [20:58] If Y/Z are on a different subnet, then you wouldn't configure them as downstream hosts :P [20:58] even if they're vm's off box x? [20:59] it seems i'm mostly thinking of network issues [20:59] It's however you want to configure things... [20:59] it seems network issues are likely to increase more and more [20:59] i wonder if snmp amplification attacks are being done yet [20:59] (speaking of snmp) [22:08] *** toeshred has joined #arpnetworks [22:10] *** mkb has quit IRC (Remote host closed the connection) [22:58] *** mkb has joined #arpnetworks