mercutio: apparently they're trying to be stable and recent at the same time
arch linux manages to be pretty stable with rolling release dne: "SFTP has been disabled by default due to security considerations" uhm, ok mercutio: yeah sftp is considered obsolete and insecure by openbsd too
scp is better
there was a bug in sftp recently dne: I think you're mixing them up - scp is an old hack, sftp is not mercutio: oh? ant: It would be very interesting to me to learn why SFTP would be considered obsolete and insecure. Unfortunately, I was unable to find information about that on Google. dne: mercutio: openssh 8.0 release notes: "The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead." mercutio: ah sweet so i was around the wrong way dne: and actually the recent scp issue, CVE-2019-6111, was on the client side ant: Alright :-) ***: ziyourenxiang has quit IRC (Ping timeout: 246 seconds) brycec: Next "poll" on Ceph storage -- Thoughts on SAS 15k drives?
From my research thus far, they provide very low seek rates and higher throughput than conventional rust, thus offering a relatively affordable middleground between disks and SSDs
Not to mention they ought to have a longer longevity, and their lower density (they seem to peak at 2TB) makes it easier to achieve a "Many smaller drives" solution for better server IOPS and lower recovery times.
(Correction: It's 15k drives that peaked at 2TB. There are, of course, 7200 drives in many-TB capacities. But if I'm going SAS, I'm going 15k, or 10k minimum)
It's true that brand-new SAS drives are more expensive than SATA counterparts, but if I've already resigned myself to buying used... Plus I know of at least one off-lease reseller that will ship fully-assembled, meaning less work for people I don't trust to do work right.
(I haven't abandoned the idea of all-flash, which is not too out of reach price-wise. But I still have concerns about SSD longevity.) acf_: is it a problem with number of write cycles? mercutio: 10k sas isn't a bad way to go brycec: acf_: Yeah write cycles are the longevity concern I have.
I think I've arrived at doing both ssd and hdd, either in separate pools or Ceph's recent cache tiering (depends how my benchmarking and testing goes). This should get me something with plenty of performance room.
By splitting it into two pools, like ARP has done, I can spent money and resources better, only where they're needed -- I'm not wasting SSDs capacity on lesser VMs, which leaves the relatively smaller SSD capacity and IO for the more important tasks. And the stuff on the spinning rust should still feel plenty snappy.
And, hopefully someday, I can scale to more OSD servers and care less about IOPS on a budget.
*I can spend money (I swear, I speak English well) mercutio: i'd still suggest you try and determine what your iops load is like
tehre are high write endurance ssds
https://www.anandtech.com/show/13704/enterprise-ssd-roundup-intel-samsung-memblaze
oh this is newer ones
hah the intel optane is 60 daily writes per day, and samsung 860 dct is 0.2 daily writes per day
but yeah it varies quite a lot... brycec: mercutio: "try and determine what your iops load is like" I have! And I have pretty graphs to show for it too. (Spent a chunk of yesterday instrumenting the current VM hosts, pushing metrics into Influxdb, putting a shiny Grafana dashboard on it etc. mercutio: oh sweet
it's always much harder to do things on small scale brycec: Most of the time, we sustain 50-100 write IOPS, reads are lower. Current hardware/setup seems to peak about 400 IOPS both read and write. mercutio: if you could just throw lots of servers at it then iops would keep going up brycec: Based on back-of-the-napkin math/estimates (formulae provided by various Ceph persons), this pool should see around 1 million IOPS for the SSD-backed storage, and about 5000 IOPS for the HDD pool. mercutio: oh wow that's pretty low
you need to write 3x
sata disks doing 5000 iops would require a lot of disks brycec: Oops typo, peak write IOPS is around 1000.
"you need to write 3x" Not sure what you mean here. mercutio: 3 way replication means each write happens to 3 hard-disks brycec: Yeah, I accounted for that mercutio: and you still think 5000 iops?
how much are you giving each disk? brycec: That 5000 IOPS is 200 IOPS * 12 drives * 4 servers * 0.64 (a magic number from one of the Ceph devs) mercutio: 7.2k won't usually do 200 iops brycec: (And 200 IOPS is a conservative estimate) mercutio: it's relaly not conservative brycec: mercutio: When have I ever said 7200? mercutio: oh ok
yeah 15k sas will
10k sas tends to be about twice as fast as 7200 rpm sata
but 15k sas is only 50% faster
err 50% faster than 10k sas brycec: "only 50% faster"
I don't consider "50%" an "only"
eg: The difference between driving 100kph and 150kph is quite noticeable mercutio: i think why 10k sas tends to outperform 7200 rpm sata by so much is both better command queueing, and firmware that is more designed for high iops
true
but 50k vs 100k is bigger difference
i think it's around ~3msec latency on 15k sas, ~6msec, 10k sas, ~12msec on 7200 rpm sata
of course it's like .15msec latency on ssd
hmm wikipedia on iops actualyl has iops figuers for hard-disks
https://en.wikipedia.org/wiki/IOPS BryceBot: IOPS :: Input/output operations per second (IOPS, pronounced eye-ops) is an input/output performance measurement used to characterize computer storage devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks (SAN). Like benchmarks, IOPS numbers published by storage device manufacturers do not directly relate to real-world application performance. Background To meaningfully describe the performance... mercutio: they even confirm my nearly 2x improvement from 7200 rpm to 10k
they're showing less than 50% improvement from 10k sas to 15k sas anisfarhana: I love arpnetworks!
Since 2013!
And keep going now.
Thank you #arpnetworks ***: ziyourenxiang has joined #arpnetworks up_the_irons: anisfarhana: lol you're welcome
BryceBot: my opinion on refurbished drives is they are a complete waste of time. We have droves of refurbs because we actually would send our failed drives back that were still under warranty (many had 5 year warranties, so we made the manufacturers honor that). The drives we got back would all fail frequently, and keep in mind, the drive you get back doesn't have any of the warranty left. Only 90
days. It's really just blind robbery.
Over the years, on SATA, we used Seagate, Hitachi and Western Digital. Seagate was the worst. Hitachi was a lot better. Western Digital has also been pretty good. m0unds: yeah, HGST/Hitachi was always our most reliable too
but they got eaten by WD i think mercutio: is toshiba still around?
i really feel like more hard-disk competition would be good m0unds: umm, i think so mercutio: i dunno what happened to 10k sata hard-drives m0unds: they still do some ssds, afaik mercutio: there was velociraptor... m0unds: yea, they were expensive though mercutio: yeh m0unds: they did 15krpm 2.5" too
but i think the bulk of consumer use is 5400/5900rpm anyway mercutio: affordable 10k 1tb sata disks would be nifty for things like games
yeah for movies that doesn't matter m0unds: yep
i got some WD reds that are nice for bulk storage - replaced some ancient (6 year old or so) 2TB HGST disks that were no longer made and i couldn't find replacements for
but consumer ssds have gotten so damned affordable, i ended up replacing one of my 1tb 7200 rpm disks with a 1tb ssd for like $120 mercutio: real
i know 500gb are getting cheaper now
but thought 1tb was still a bit far out
i got a second nvme ssd.. i have three now m0unds: not at all - the newer sandisk w/marvell chipsets go one sale for $120-179 mercutio: one isn't being used though.. it's only 128gb, and i need another pci-e adapter m0unds: yeah, i have a 540 or some-odd boot SSD on my workstation and a pair of 1TB ones for games and such mercutio: i went intel instead of samsung for my last sd m0unds: and then a 4TB for the remainder of games that i don't want to load fast (my flight sim stuff takes a ton of space) mercutio: a bit lower raw performance, but samsung don't even deliver the speed they say m0unds: yeah, i had bad luck w/sammy stuff in the 540 chipset guys up_the_irons: So USPS was supposed to deliver my X1 Carbon today, but it also says "Arriving Late" on the tracker. Thanks fuckers.. m0unds: had a couple failures under warrant up_the_irons: Hopefully tomorrow m0unds: lame mercutio: what size ssd you getting with it up_the_irons ? m0unds: the worst are when they say it's out for delivery or something, then you don't hear anything, then at like 0200 it says "arriving late" cuz it went out on the wrong truck mercutio: speeds are getting good enough with ssd that reliability is more important than performance up_the_irons: got a 6th gen, WQHD screen, 256GB SSD, 8650U CPU and 16GB of RAM for a bit over $1100 mercutio: and i feel like intel are less likely to fail than samsung m0unds: awesome, great deal mercutio: not bad up_the_irons: yeah! mercutio: wqhd is 2560x1440? m0unds: yeah mercutio: nice up_the_irons: and it was described as mint condition, barely used. The pics look great mercutio: that sounds like an ideal laptop to me
its' pretty recent! m0unds: is that like a 13/14"? up_the_irons: yeah 6th gen isn't old at all
14" m0unds: nice mercutio: not bad
with 1440p that'll look real sharp
you probably need to raise the DPI in X up_the_irons: i wouldn't have minded a 5th gen, but this guy popped up and was a good deal...
there's a 5th gen open box (about same specs) for $999 floating around on ebay m0unds: ah
i think mine's a 5th gen
not an x1, yoga 910 mercutio: intel onboard video has been improvnig a little for 2d workloads up_the_irons: i wanted 5th gen or higher b/c of USB C charging mercutio: so if it's using onboard video at 1440p it may help a little... up_the_irons: 1 charger can do both my MacBook and the X1 now m0unds: oh, nice - new macbooks are usb-c right? up_the_irons: yeah mercutio: usb c sounds awesome up_the_irons: traveling with just 1 charger will reduce weight m0unds: oh, i guess this one's a 7th gen. huh.
i figured 7xxx woulda been older mercutio: damn your cpu is 4 core + ht up_the_irons up_the_irons: yeah it's beefy
i7 i believe mercutio: most laptops cheap out and have i7s with 2 core+ht or something
it's odd m0unds: yea, that's what mine is
7500u
https://ark.intel.com/content/www/us/en/ark/products/95451/intel-core-i7-7500u-processor-4m-cache-up-to-3-50-ghz.html mercutio: yeah so it says i7... up_the_irons: 6th gen went all out on the CPUs:
https://psref.lenovo.com/syspool/Sys/PDF/ThinkPad/ThinkPad%20X1%20Carbon%20(6th%20Gen)/ThinkPad_X1_Carbon_6th_Gen_Platform_Specifications.pdf mercutio: hd 620 vs hd 630
i wonder how much diff that makes m0unds: dunno, i have a 4k display so i wouldn't mind a tick faster gpu ,haha up_the_irons: so now that i've given up waiting on USPS, i'll go step outside.. bbiab mercutio: the macs have onboard memory on some..
but yeah onboard video at high res can struggle a bit with scrolling etc
like normal 2d workloads
memory speed helps a bit m0unds: haven't noticed that much on win10
16gb, 1tb nvme ssd (samsung) mercutio: it's pretty bad on i3 with 1440p
with ddr3
not sure what cpu it is
maybe ivy bridge m0unds: ah
i have a haswell workstation, but beefy gpu (gtx 1080) mercutio: it's better on haswell
it must be ivy bridge
but yeah they're still making improvements to onboard video m0unds: seems like most of the benefit of newer intel cpus have been more mem bandwidth and better power/tdp mercutio: amd is better that said
it's better idle power usage mostly
rahter than load m0unds: yeah mercutio: better power gating etc, partial shut down m0unds: yeah, i turn that off on my workstation cuz it's only used for gaming anymore mercutio: also context switch performance has been improving
which helps virtualisation type applications
and spectre workarounds... m0unds: yeah mercutio: if you don't care too much about power usaage than c1 idle states gives most power reduction but low latency m0unds: ah mercutio: i hav haswell with 1060, and the one after haswell with 1070 m0unds: i wouldn't mind a newer AMD as an "upgrade" mercutio: ryzen? m0unds: yea mercutio: yeh m0unds: can get MANY-A-THREAD
hahaha mercutio: yeah i doubt it helps much still m0unds: modern games are liking the extra threads mercutio: i kind of thought stuff woudl be better threaded years back
ah ok m0unds: but IPC just isn't a big enough jump mercutio: starcraft 2 still is dual core m0unds: i do a lot of flight simming (x-plane 12) and some other crap that likes mercutio: i suppose it's an 8 year old game now m0unds: extra threads * mercutio: yeah depensd what you use m0unds: yup
i got a small-ish xeon workstation w/256gb ssd and 16gb of ram for $100 to use as a home "server" of sorts, runs a vm and all my nas backup crap mercutio: but the vast majority of software doesn't parallelise well
compiling is like the main exception m0unds: yeah mercutio: but linking doesn't parallelise well
llvm has a cool new faster linker though. m0unds: e3-1245v2 guy
yeah, i don't do anything w/linux or unix outside of vms anymore mercutio: did you see windows has ssh now? m0unds: yeah
i still just use kitty, haha
old habits die hard mercutio: it's odd how microsoft is embracing linux m0unds: they're giving up on their browser engine in favor of chromium too mercutio: wtf
that's kind of bad news :( m0unds: yea, i agree - big part of it was that web devs would just add code to say "we only support chrome and firefox" whenever a site was requested w edge
kind of crummy, reminds me of the old IE days mercutio: things kind of work better if multiple people implement standard compliant browsers imo m0unds: yep mercutio: oh m0unds: but on the plus side, they fixed a 4 year old bug in chromium, lol mercutio: heh
is it faster? BryceBot: That's what she said!! mercutio: chromium used tob e faster than chrome m0unds: yea
edge was faster at a lot of stuff than chromium
but it depended
i have a dev build of nu-edge, but haven't played with it much
edge had much, much better text rendering mercutio: i can't find what edge version i have on my windows box m0unds: if you do three dot > settings, it's right at the bottom mercutio: Microsoft Edge 44.18342.1.0
yeah i loked under help -> about m0unds: insider build? mercutio: yeah m0unds: yeah, my stable one is 44.17763.1.0 mercutio: oh you can download it anyway m0unds: the chromium based one is 76.0.159.0
yeah
it's a separate channel for edge mercutio: i'm doing dev channel m0unds: yeah, that's the one i'm on for edge
has all the favorite sync stuff integrated already and they're doing some work for extension qa now mercutio: it's downloading real slow
network interface doing 4 megabit/sec m0unds: lame mercutio: yeah i dunno why it's so slow
most things download fast m0unds: what's the cdn-of-choice in nz? akamai? mercutio: this one is using hwcdn and going to los angeles m0unds: ah, ok mercutio: akamai is terrible generally
but it's still pretty popular
cloudflare has become very popular
most cdns have nodes in nz though
except weird ones like verizon m0unds: gotcha
verizon is weird in general mercutio: no adblock! m0unds: hm? mercutio: and it doesn't support dark mode
in edge
it downloaded m0unds: ah, i installed ublock origin mercutio: heh default search is bing m0unds: i use nano adblock in chrome mercutio: oh you can still install chrome plugins? m0unds: chrome://extensions mercutio: oh wow
maybe they will kill bing soon and use google too m0unds: dunno, i only really use bing tbh
mostly cuz they do a rewards prog in the us
i hear it's crummy in other countries though
the rewards thing pays for my monthly xbox gamepass stuff, haha mercutio: hah opera is saying it's not good for competition.. but they did the same! m0unds: yeah mercutio: it's rather scary m0unds: yep mercutio: apple is our only hope m0unds: yup, hahaha mercutio: they can bring safari to windows
acutally isn't that already a thing m0unds: i hate new firefox - i had been using it for a while, but they kept breaking stuff and screwing other shit up
it was, they stopped mercutio: oh
hopefully they resume
i use firefox a little
it's faster than chrome
but a bit awkward m0unds: yeah, it's bad on android
they're doing that whole fenix thing now which might make it better acf_: up_the_irons: nice
that's basically the upgrade I did a few months ago
X250 to 5th gen x1 carbon, i7, 500GB nvme, 1080p
I'm glad I didn't go for the WQHD screen though
Linux's hidpi support still sucks bad
I'm just running it at a 1:1 scale