[06:19] *** [FBI] starts logging #arpnetworks at Tue May 14 06:19:33 2019
[06:19] *** [FBI] has joined #arpnetworks
[06:20] <up_the_irons> mnathani: I invited [FBI] back
[06:20] <up_the_irons> LG should be back up today
[06:45] <mnathani> Thank you!
[08:53] *** ziyourenxiang has quit IRC (Ping timeout: 268 seconds)
[12:31] <brycec> up_the_irons: Just curious, what kind of drives is ARP buying these days? I'm trying to shop and having a helluva time (maybe I'm just out of practice) finding *new* 4-6TB drives. I don't need more than that (a modest initial roll-out, 4 servers @ 4 drives each), under 100TB (raw) is more than enough. But... gotdamn I'm having a time. It seems the brand new drives I can find these days are 10-14TB (wow)
[12:31] <brycec> and they're $300+/ea (breaking the bank); anything <8TB stopped being made at least 3 years ago it seems.
[12:32] <brycec> (I'm obviously looking at enterprice/datacenter "grade" drives, not WD Blacks or whatever)
[12:32] <brycec> (PS I got your sales response and I'll definitely be following-up as soon as I get this initial proposal done... which is currently hung-up on sourcing fucking drives.)
[12:33] <mercutio> huge hard-disks are bad for VM work loads ok for archival
[12:33] <brycec> mercutio: precisely why I'm avoiding them too
[12:33] <mercutio> well 4tb is huge
[12:34] <brycec> You think? It was my upper limit, but it just seemed "large" not "huge"
[12:34] <mercutio> well most people are doing ssd these days
[12:34] <mercutio> and not 4tb ssd :)
[12:34] <brycec> In any case, I'm about to start budgeting SSDs instead
[12:34] <brycec> lol
[12:35] <mercutio> you really want as many drives as you can
[12:35] <mercutio> we have hundreds
[12:35] <brycec> I also feel really out of touch with buying hard drives in general. There was a time I felt comfortable buying from Amazon or Newegg, provided the seller was Amazon/Newegg and not some rando third-party that's offloading "renewed" inventory.
[12:36] <mercutio> 4x4 is the wrong approach..
[12:36] <brycec> I get that, or at least I thought I did (quantity over capacity).
[12:36] <brycec> 4*8?
[12:36] <mercutio> well you also want journal disk usually
[12:36] <mercutio> so you want servers that can take 12+ hard-disks really
[12:36] <brycec> Already accounted for that, dedicated SSD for that, dedicated SSD for OS
[12:36] <mercutio> you want ssd per 3 or 4 disks
[12:37] <mercutio> so like 12 disk server you may have 3 ssd and 9 data disks
[12:37] <mercutio> it's changing a bit with blue store..
[12:37] <brycec> I'm limited by overall rackspace so I'm going with 2U servers, 12 slots (plus 2 for OS & journal)
[12:37] <mercutio> i think with bluestore it can journal rather than full copy data so you may be able to have slightly fewer ssd
[12:38] <brycec> Yeah I've heard Bluestore is quite improved.
[12:39] <mercutio> and if you're not logging shit loads then you should be able to use the same disks for journal and OS
[12:39] <mercutio> although that does complicate things a little
[12:40] <brycec> Yeah I intend to do a bit of benchmarking there - RAID1 the 2 SSDs vs OS/journal split.
[12:41] <mercutio> with mdadm you can do raid 1 with part of the disk and not all of it
[12:41] <mercutio> you can also do things like 3 way mirror for OS
[12:43] * brycec grumbles - Amazon has a limit of 20 drives/customer
[12:44] <mercutio> curious i wonder why that is
[12:44] <mercutio> did you scope out what performance you need?
[12:44] <brycec> They probably don't trust me to not be starting my own reseller
[12:45] <brycec> mercutio: From my users, "better than it is now"
[12:45] <mercutio> damn it costs the same for 5tb and 2tb?
[12:45] <mercutio> what iops can you do now?
[12:45] <brycec> I honestly couldn't say :/
[12:45] <mercutio> have you tried fio?
[12:45] <mercutio> ioping can be handy too
[12:46] <mercutio> like you can do ioping -R /dev/vda
[12:46] <mercutio> and it'll tell yuo your access times
[12:46] <mercutio> it's pretty latency dependant
[12:46] <brycec> Thanks I'll check those out
[12:47] <mercutio> fio is a little more difficult to work
[12:47] <mercutio> but you can do things like simultaneous 4k, or 64k, or whatever requests
[12:47] <mercutio> and you can do read/write, mixed etc to a file
[12:47] <mercutio> it's basically the best tool for in depth benchmarking
[12:47] <mercutio> but you have to figure out what it is that you want to benchmark :)
[12:49] <mercutio> also for ssd for journal you want an ssd that has battery backed write cache
[12:49] <mercutio> or power loss protection
[12:49] <mercutio> as it means you get a lot lower latency...
[12:50] <mercutio> https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
[12:55] <dne> brycec: I got 4TB HGST Ultrastar 7K6000 drives for our office bulk storage array last year - they've been fine (not using Ceph though)
[12:55] *** acf_ has joined #arpnetworks
[12:56] <mercutio> damn they have 14tb disks now
[12:58] <mercutio> i wonder if going for such a small cluster if it is better to just go all ssd from the get go
[12:58] <mercutio> especially when you say density is important
[12:58] <mercutio> you can get servesr that take a lot of 2.5" disks
[16:45] *** ziyourenxiang has joined #arpnetworks