| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | ziyourenxiang has quit IRC (Quit: Leaving) | [01:19] |
| ............... (idle for 1h11mn) | ||
| carvite has quit IRC (*.net *.split)
sjackso has quit IRC (*.net *.split) jpalmer has quit IRC (*.net *.split) KDE_Perry has quit IRC (*.net *.split) KILLALLHUMANS01 has quit IRC (*.net *.split) DaCa has quit IRC (*.net *.split) milki has quit IRC (*.net *.split) toddf has quit IRC (*.net *.split) up_the_irons has quit IRC (*.net *.split) d^_^b has quit IRC (*.net *.split) mike-burns has quit IRC (*.net *.split) mhoran has quit IRC (*.net *.split) | [02:30] | |
| carvite has joined #arpnetworks
sjackso has joined #arpnetworks jpalmer has joined #arpnetworks KDE_Perry has joined #arpnetworks KILLALLHUMANS01 has joined #arpnetworks DaCa has joined #arpnetworks milki has joined #arpnetworks toddf has joined #arpnetworks up_the_irons has joined #arpnetworks d^_^b has joined #arpnetworks mike-burns has joined #arpnetworks mhoran has joined #arpnetworks asimov.freenode.net sets mode: +oo toddf up_the_irons | [02:36] | |
| ...... (idle for 25mn) | ||
| nesta has quit IRC (Ping timeout: 245 seconds)
nesta has joined #arpnetworks | [03:01] | |
| ........................................................ (idle for 4h39mn) | ||
| Lucifer333 has quit IRC (Quit: Leaving)
Lucifer333 has joined #arpnetworks | [07:41] | |
| ................. (idle for 1h22mn) | ||
| brycec | I couldn't say. I have neither the expertise nor the data to fully form a hypothesis. | [09:04] |
| *** | BryceBot has quit IRC (Ping timeout: 258 seconds) | [09:13] |
| KILLALLHUMANS01 | Well cool/fuck.
Reproduced my issue even though I slowed down the whole process. | [09:13] |
| *** | brycec has quit IRC (Ping timeout: 248 seconds) | [09:14] |
| KILLALLHUMANS01 | So I'm hitting this when I copy from one database to another (moving BryceBot and other services off MySQL, finally.) I wrote a Ruby script based on Sequel's (bin/sequel) copy-database mode. It does a big SELECT on the source database (some half-million rows as I recall) and then for each result, INSERTs into the new database.
I thought the act of slamming so many rows into the database at such a high rate was responsible for bringing my system down, so I put a sleep in - 1 row every 5 seconds. BAM still died. What's kindof funny is that neither MySQL nor Postgres (nor my script) are showing in htop (well MySQL is, just not at high load) Also weird is that kswapd is responsible for a relatively large chunk of CPU, along with kworker/0:3 (I'll need to check which device this refers to), but there is ZERO swap usage as of yet, and still a decent chunk of free memory for that matter. https://www.dropbox.com/s/0o1e1e39we6drcp/screenshot_2016-12-29_09-12-03.png?dl=0 I'm guessing that something (MySQL?) tried to allocate a big piece of memory, forcing the system to start trying to figure out what all it can swap out, but it's having trouble... Either deciding, or actually writing to swap. Regrettably I don't have an iotop this time. Also of note, pretty low load average... Then again, htop hasn't refreshed any time recently. It probably froze when the kernel started looking for volunteers to swap out. So... Yeah. There's a bit more data, I guess. At least as much as I can get. Seems like I can reproduce it relatively easily, at least with this current state of things. (Or maybe it's the Ruby process trying to allocate?) Unrelated question - On consule.cust, the "broadcast message" function ^Ecb, does that go to all users on consule.cust, or just those connected to my VPS? (If the former, then I apologize for my language just now.) | [09:15] |
| *** | BryceBot has joined #arpnetworks | [09:27] |
| brycec has joined #arpnetworks | [09:32] | |
| .............. (idle for 1h9mn) | ||
| nathani | maybe you need moar RAM brycec | [10:41] |
| mercutio | or maybe it's some kernel bug
hmm i'm not sure if it broadcasts to all we should try it :) | [10:44] |
| ..... (idle for 23mn) | ||
| brycec | nathani: You don't even work for ARP, why are you trying to sell me on more RAM? :P (Once I'm off MySQL, I'll actually have an even smaller memory footprint :) ) | [11:08] |
| nathani | :-) | [11:09] |
| mercutio | linux's memory management used to be really really bad
and having more ram could make it worse sometimes linux aggressively forces memory into being used as disk cache so you can get memory pressure even when you have plenty of spare memory if you read a lot of data in in 2.2.x kernels it was all over the place erk, i just remembered that was like 15 years back haha | [11:17] |
| brycec | Yeah I'm pretty sure it was better by 3.16.0-4-amd64 | [11:22] |
| mercutio | i don't notice much changes in new kernels these days
i do wonder when BBR will be merged though oh it's in 4.9 | [11:24] |
| brycec | Anyhow, now paginating my input so hopefully no random memory pressure. So far, so good...
3+ million rows, 1000 at a time. With periodic breaks every 5 seconds to allow disk writes to flush if they want, etc brycec is watching the source MySQL database gradually use more RAM while Postgres just sits back and takes it. | [11:39] |
| mike-burns | My current client uses MySQL; it's been fun watching each new dev they hire raise the same question within their first week: "So, when are we switching to Postgres?" | [11:41] |
| brycec | lol | [11:49] |
| ................................................................... (idle for 5h32mn) | ||
| up_the_irons | nathani: i'm using weechat and screen. non-ascii is no problem.:)
JC_Denton: i use mutt | [17:21] |
| *** | nesta has quit IRC (Quit: WeeChat 1.6) | [17:29] |
| ....... (idle for 30mn) | ||
| nesta has joined #arpnetworks | [17:59] | |
| JC_Denton | up_the_irons: 1.7.x? | [18:06] |
| up_the_irons | no 1.5.x
it's old | [18:06] |
| JC_Denton | *shrug*
1.7 isn't that new ;) but it did mainline the sidebar patch, which is why i built from source on my debian login host was having a god awful annoying screen corruption bug i finally figured it out, though. i needed to compile against wide-ncurses vs. plain | [18:06] |
| mercutio | i thought it was slang, hmm
mine is curses too | [18:18] |
| ..... (idle for 22mn) | ||
| up_the_irons | i c | [18:41] |
| ............ (idle for 55mn) | ||
| *** | ziyourenxiang has joined #arpnetworks | [19:36] |
| ...... (idle for 26mn) | ||
| mercutio | i'm using 1.4?
it seems i am way behind | [20:02] |
| JC_Denton | you can compile it with slang | [20:05] |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
