sct04 / ceph cluster have a little bump? Oct 11 22:08:24 vps3 kernel: [12522051.816013] sd 0:0:0:0: [sda] abort
(And my Thunder instance has been tumbling downhill since then, having remounted to read-only, Postgres is unhappy, etc etc etc
KILLALLHUMANS01: had a couple reports of disk timeouts, so i'm looking into it
Thanks up_the_irons
I ended up hard-rebooting things
and am gradually getting back to normal, you should see brycec join shortly
btw this was the SSD-backed pool specifically
KILLALLHUMANS01: but are you still seeing disk issues?
Not currently
OK
According to logs, the issue started at 22:10 and continued for a few minutes at least (when the system dropped to read-only mode and logging was useless) -- And anything logged after read-only is muddy at best since there was a lot of stuff just flapping
PST ?
"ARP Standard Time" :P yeah
LOL
OK
works
what works
brycec: ^
my connection (except actually I was in the middle of typing "/join #arpnetworks" when Weechat rejoined the room on its own, so you just got the last 5 letters :P
I kicked out a particularly noisy and dying drive from our Ceph cluster a few moments ago.  Not sure why it wasn't kicked out on its own, but I suspect those who had some blocks on that particular one may have got some timeouts too.
interesting
I think so too