[01:19] *** ziyourenxiang has quit IRC (Quit: Leaving) [02:30] *** carvite has quit IRC (*.net *.split) [02:30] *** sjackso has quit IRC (*.net *.split) [02:30] *** jpalmer has quit IRC (*.net *.split) [02:30] *** KDE_Perry has quit IRC (*.net *.split) [02:30] *** KILLALLHUMANS01 has quit IRC (*.net *.split) [02:30] *** DaCa has quit IRC (*.net *.split) [02:30] *** milki has quit IRC (*.net *.split) [02:30] *** toddf has quit IRC (*.net *.split) [02:30] *** up_the_irons has quit IRC (*.net *.split) [02:30] *** d^_^b has quit IRC (*.net *.split) [02:30] *** mike-burns has quit IRC (*.net *.split) [02:30] *** mhoran has quit IRC (*.net *.split) [02:36] *** carvite has joined #arpnetworks [02:36] *** sjackso has joined #arpnetworks [02:36] *** jpalmer has joined #arpnetworks [02:36] *** KDE_Perry has joined #arpnetworks [02:36] *** KILLALLHUMANS01 has joined #arpnetworks [02:36] *** DaCa has joined #arpnetworks [02:36] *** milki has joined #arpnetworks [02:36] *** toddf has joined #arpnetworks [02:36] *** up_the_irons has joined #arpnetworks [02:36] *** d^_^b has joined #arpnetworks [02:36] *** mike-burns has joined #arpnetworks [02:36] *** mhoran has joined #arpnetworks [02:36] *** asimov.freenode.net sets mode: +oo toddf up_the_irons [03:01] *** nesta has quit IRC (Ping timeout: 245 seconds) [03:02] *** nesta has joined #arpnetworks [07:41] *** Lucifer333 has quit IRC (Quit: Leaving) [07:42] *** Lucifer333 has joined #arpnetworks [09:04] I couldn't say. I have neither the expertise nor the data to fully form a hypothesis. [09:13] *** BryceBot has quit IRC (Ping timeout: 258 seconds) [09:13] Well cool/fuck. [09:13] Reproduced my issue even though I slowed down the whole process. [09:14] *** brycec has quit IRC (Ping timeout: 248 seconds) [09:15] So I'm hitting this when I copy from one database to another (moving BryceBot and other services off MySQL, finally.) I wrote a Ruby script based on Sequel's (bin/sequel) copy-database mode. It does a big SELECT on the source database (some half-million rows as I recall) and then for each result, INSERTs into the new database. [09:16] I thought the act of slamming so many rows into the database at such a high rate was responsible for bringing my system down, so I put a sleep in - 1 row every 5 seconds. BAM still died. [09:16] What's kindof funny is that neither MySQL nor Postgres (nor my script) are showing in htop (well MySQL is, just not at high load) [09:17] Also weird is that kswapd is responsible for a relatively large chunk of CPU, along with kworker/0:3 (I'll need to check which device this refers to), but there is ZERO swap usage as of yet, and still a decent chunk of free memory for that matter. [09:17] https://www.dropbox.com/s/0o1e1e39we6drcp/screenshot_2016-12-29_09-12-03.png?dl=0 [09:18] I'm guessing that something (MySQL?) tried to allocate a big piece of memory, forcing the system to start trying to figure out what all it can swap out, but it's having trouble... Either deciding, or actually writing to swap. [09:18] Regrettably I don't have an iotop this time. [09:19] Also of note, pretty low load average... Then again, htop hasn't refreshed any time recently. It probably froze when the kernel started looking for volunteers to swap out. [09:20] So... Yeah. There's a bit more data, I guess. At least as much as I can get. Seems like I can reproduce it relatively easily, at least with this current state of things. [09:20] (Or maybe it's the Ruby process trying to allocate?) [09:22] Unrelated question - On consule.cust, the "broadcast message" function ^Ecb, does that go to all users on consule.cust, or just those connected to my VPS? (If the former, then I apologize for my language just now.) [09:27] *** BryceBot has joined #arpnetworks [09:32] *** brycec has joined #arpnetworks [10:41] maybe you need moar RAM brycec [10:44] or maybe it's some kernel bug [10:45] hmm i'm not sure if it broadcasts to all [10:45] we should try it :) [11:08] nathani: You don't even work for ARP, why are you trying to sell me on more RAM? :P (Once I'm off MySQL, I'll actually have an even smaller memory footprint :) ) [11:09] :-) [11:17] linux's memory management used to be really really bad [11:17] and having more ram could make it worse sometimes [11:17] linux aggressively forces memory into being used as disk cache [11:18] so you can get memory pressure even when you have plenty of spare memory if you read a lot of data in [11:18] in 2.2.x kernels it was all over the place [11:19] erk, i just remembered that was like 15 years back [11:19] haha [11:22] Yeah I'm pretty sure it was better by 3.16.0-4-amd64 [11:24] i don't notice much changes in new kernels these days [11:25] i do wonder when BBR will be merged though [11:26] oh it's in 4.9 [11:39] Anyhow, now paginating my input so hopefully no random memory pressure. So far, so good... [11:39] 3+ million rows, 1000 at a time. [11:39] With periodic breaks every 5 seconds to allow disk writes to flush if they want, etc [11:40] * brycec is watching the source MySQL database gradually use more RAM while Postgres just sits back and takes it. [11:41] My current client uses MySQL; it's been fun watching each new dev they hire raise the same question within their first week: "So, when are we switching to Postgres?" [11:49] lol [17:21] nathani: i'm using weechat and screen. non-ascii is no problem.:) [17:22] JC_Denton: i use mutt [17:29] *** nesta has quit IRC (Quit: WeeChat 1.6) [17:59] *** nesta has joined #arpnetworks [18:06] up_the_irons: 1.7.x? [18:06] no 1.5.x [18:06] it's old [18:06] *shrug* [18:06] 1.7 isn't that new ;) [18:07] but it did mainline the sidebar patch, which is why i built from source on my debian login host [18:07] was having a god awful annoying screen corruption bug [18:07] i finally figured it out, though. i needed to compile against wide-ncurses vs. plain [18:18] i thought it was slang, hmm [18:19] mine is curses too [18:41] i c [19:36] *** ziyourenxiang has joined #arpnetworks [20:02] i'm using 1.4? [20:02] it seems i am way behind [20:05] you can compile it with slang