Wow, Linux really does have a faster network stack than OpenBSD. Wheezy on this Metal box gets at least 800mbps consistently.
100MB test file downloaded in 1 second :p
brycec: it does bug me a little :)
i find my home mirror pretty fast :)
well proxy cache, with ssd.
i think a lot of the time it's about small file performance.
and so having a low ping, and cached on ssd or ram helps.
saturating gigabit is quite hard for a hard-disk with lots of small files.
I notice it most with Sourceforge mirrors, many of which barely crack 10mbps
and raid can't really fix it.  using 15k sas disks or something would help more than raid probably.
There's no excuse for that in this day and age.
oh sourceforge has it's own issues i think.
(*its)
i get 200 to 300k/sec or something on sourceforge frequuently
actually the worst i find are motherboard wsites.
if you ever download anything from a motherboard site i tend to find it can be like 20 to 30k/sec or less!
Heh I don't run into too many of those typically, but yeah they are usually slow. I don't mind much, they aren't running a professional download service :p
and it's not just because they're in taiwan.
well sourceforge don't charge for downloads.
and don't have much of a revenue stream?
i find mirrors.kernel.org seems to have consistent speeds.
well when it goes to the right mirror hah
Indeed, quite reasonable. It's my goto mirror
but it doesn't tend to get slow at diff times.
whereas archive.ubuntu.com/security.ubuntu.com can crawl
whenever there is a new ubuntu release it shows on my smokeping :)
i actually use arp's mirror a lot :)
plus nz mirror.
the nz mirrors are faster than arp it seems (when similar ping)
probably just better raid config, or more users so more likely to be in cache
haha
I'm using ARP's mirrors quite a bit right now too... on-network.
I'd get much more use out of it if ARP would just mirror Debian too.
(and stop wasting space mirroring architectures they themselves don't support)
yeah i think he's short on space.
yeah that'd be sensible.
Ditto - he's commented on it before.
Why do we mirror powerpc ISOs for FreeBSD? :p
maybe you should volunteer to clean it up.
we/they
I didn't know up_the_irons was looking for volunteers
i have no idea if he is or not :)
the full Debian archive is ~1TB :/
I looked into downloading it once..
I've run company mirrors in the past, I have some experience doing so :)
but i think he's kind of busy with other things.
you probably really want 4x3tb disks or something for a large mirror.
He does indeed seem to be quite busy
if you used zfs and had a ssd cache it could be amazing :)
i was going to setup an openbsd mirror before.
I did that last month :)
(internal only)
how big is it now?
472GB?
hangon this site says debian is 578GB
(my vpn seems to be down, ugh)
oh there's debian CD too which is 327GB
http://mirrors.gigenet.com/
this mirror seems to list sizes
even postfix is 1.3gb haha
https://www.debian.org/mirror/size
oh mirorring just amd64 is only 100gb
amd64+i386+source would be better
yea, that should be fine
I'd be very happy about a Debian mirror at ARP
I have lots of boxen connected to ARP for IPv6 via a tunnel
arch is only 41gb
+7gb for iso, +28GB for source.
mercutio: My OpenBSD mirror is 738GB, only mirroring amd64 and i386, releases 4.0 through 5.6 as well as snapshots, Open{SSH,NTPD,BGPD}, LibreSSL, as well as M:tier's openup
brycec: arp doesn't mirror that many versions
Okay. And?
that's quite large.
twss
Okay! twss! 'that's quite large.'
732GB :(
http://sprunge.us/fIVN
hmm
i wonder how much disk arp's mirroring is using.
http://www.amazon.com/dp/B00LO3KR96
Amazon: "WD Red 6TB NAS Hard Drive: 1 to 8-bay RAID Hard Drive: 3.5-inch SATA 6 Gb/s, IntelliPower, 64MB Cache WD60EFRX"
And yes that's quite a few releases. It spans our development of OpenBSD-based systems, so we kinda need to have access to those old versions. (which is hard to come by, these days)
eww ed :)
wd
yeah i've had that issue before brycec
actually i'm surprised arp doesnt' go back further
it's only got 5.5 and 5.6
ARP is just rsync'ing from upstream, which doesn't carry older than that
i nearly bought those cheap external seagate drives.
oh ok
i think it was $140 for 5tb or something?
http://www.amazon.com/Seagate-Expansion-Desktop-External-STBV5000100/dp/B00JT0EGPW/ref=lp_1254762011_1_2?s=pc&ie=UTF8&qid=1428564996&sr=1-2
Amazon: "Seagate Expansion USB 3.0 5TB Desktop External Hard Drive (STBV5000100)"
woah
so you strip them out of their cases...
yea
yeah it's pretty good value
but seagate have some weird screw stuff with some of their larger drives.
so they won't necessarily work with trays properly.
i've got a dying disk on my personal server :(
is that a 3.5" disk?
yeah 3.5"
i was hoping to upgrade to 6tb when i got disk failure
but i'll probably just stick another 3tb in
actually zfs registered no errors.
so maybe it fixed itself.
[689969.969725] blk_update_request: I/O error, dev sdd, sector 4314598592
you'd /think/ zfs would have an issue there
that's always scary..
using RAID?
yeah raid-z
good :P
it has 2.6tb of disk free.
but it's in nz.
is the ARP mirror a dedicated box?
don't think so
darn
i wish amazon wouldn't combine reviews
so amazon charge $73 US for 1tb, $83 US for 3tb.
oh it's cheaper for normal 1tb, that was retail, some weird link thing
hey Brycec you could run a mirror :))
i wonder if arp would let you provide your own hard-disks
i can't figure out if this hard-disk is the weird screw kind of not though
bit's smr
so it's randomly slow
you'd have to have a dedicated to have your own hard disk
and then what if it fails?
the odd# TB disks tend to have longevity issues
1.5, 3, etc
because they do weird stuff to achieve that density
so i guess not odd numbers as in 1,3,5 etc, but odd numbers as in half numbers or combinations of half numbers
https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/ oddly enough, it seems like backblaze did a study of sorts w/failure rates
I don't find it odd that a company that consumes hard drives by the truckload did a study :P
nothing odd about it
that was the joke
at any rate, not the first time i've seen data saying the weird in-between sizes are unreliable. i just couldn't find the original one i read a while back (had a lot more specific data)
Oh I didn't catch the joke/sarcasm :p
there was an amusing post on their blog from the middle of the disk shortage that happened following the flooding in thailand
where their cost for standalone consumer disks doubled (or more) so they resorted to buying external drives and "shucking" them from their enclosures
https://www.backblaze.com/blog/backblaze_drive_farming-2
Yeah I remember
My company was doing the very same thing
In smaller quantities at least. But we were doing it too. And having friends+family buy-out stores and ship the drives to us
m0unds: sometimes it's hard to know what they're doin ghtough
like 2tb disks can be 2 or 3 paltters with seagate.
Easy to tell by weight and construction, though that usually comes after you've opened store packaging ;P
i thought 1tb were good myself
That's what she said!!
m0unds: you know there's iossues with that study btw?
shit i so i can't type before coffee.
basically they don't have proper vibration protection at blackblaze, and they';re using consumer hard-disks which don't uusually have good vibration tolerance as serve rhard-drives.
I'm pretty sure BB do something about vibrartion stuff
*vibration
oh?
Seen it mentioned in their detailed builds
https://www.backblaze.com/blog/backblaze-storage-pod-4/
""
"Includes case, anti-vibration assemblies, cross supports, etc."
the only recent disk failutres i've had have been with wdc black 1tb and seagate 3tb
but that doesn't say much on it's own :)
(I haven't actually seen their writeup on "4.0" before, but it was the first google result)
our failure rates w/ES2 and 3 seagates was less than 3%
hard-disk tech doesn't seem to have improved recently
dynamic spindle speed stuff is kind of nice for energy use reduction
they're doing some "density improvements" but it seems it coudl be at the expense of reliability/performance.
yea
i'd be happy not buying another platter disk again
i'd like cheap bulk ssd that is slower
and to have fast ssd cache in front of it :)
i'd really like to justify these 2gb/sec ssd's.
but motherboards with m.2 are really expensive.
you can get pci-e adapters, but they require a pci-e slot...
i wish intel's chipsets would have more pcie lanes to share
and shift to pci-e 3
amd provided 48x of bandwidth starting w/their 990fx chipsets
of pci-e 3?
whatever the spec was in 2010 when that chipset came out
you could run a gpu at 16 and a nic or sata controller or whatever, and neither would downgrade
my onboard ethernet is only 2.5GT/s
1 lane
or two gpus and a nic and still have them at native speeds. only downgrade occurred w/4 gpus + anything else
where my 10 gigabit is 8 lanes at 5GT/s
so has 16x as much bw
i'm wondering how it'd go in a 2x slot
on my board w/z87 chipset, 1 gpu + 1 nic + 1 other device = gpu at 4x
most dual/quad gigabit cards are 4x.
does gpu at 4x hurt?
yep
doesn't really at 8, does for sure at 4
for 2d?
nah, 3
3d
i'm at 8x/8x on my windows box
my pc's primary job is entertaining me with explosions and such
and 16x/4x on my linux box
but the 4x is chipset :(
i like 8x/8x
yea, 8/8 is fine for the most part, esp w/pcie3
so video card moving to 4x pci-2 may hurt?
yea
even for 2d?
doubt for 2d
for 3d yes
hmm
what about chrome
couldn't tell you
hw rendering is chrome breaks pretty regularly on every platform
i dunno i'm replacing motherboard anyway.
s/is/in
<m0unds> hw rendering in chrome breaks pretty regularly on every platform
i used to find linux had amazing chrome performance compared to windows
and i managed to improve my web performance with chromium on windows.
so recently i tried my old chromium build, a whole lot of sites complain about old version, but it seems way faster.
i was mostly screwing around with allowing more than 16 connections through http proxy iirc
i had all these annoying if-modified-since requests going through to proxy
and you can't just throw 30 if modified sinces out at once :(
and get a list of what's modified
i hope http/2 fixes that kind of case, it was pretty common
it's kind of well known that web pages will load faster if you rename images, (use unique names) and set long expiriation times to avoid if-modified-since
but hardly any real sites do that.
so i'm doing business cards and a website for my music stuff
i'm thinking i should probably just go with a wordpress install for the music site
the people that are going to help me with content aren't exactly tech experts
but i want to put video files and an audio player on it
so people can pick and choose what to listen to..that sort of thing
no idea how to do that portion
Interesting.
phlux: Take a look at Concrete (C5) too
theres porbably a plugin for multimedia content
It aims to be more user-friendly, and more secure
uploading and viewing / listening
s/porbably/probably
<mnathani_> theres probably a plugin for multimedia content
I wonder if there's a good multimedia addon for Concerte
Concrete, even
Something like this: https://www.wonderplugin.com/wordpress-audio-player/
phlux: are you doing a facebook page?
yes
it seems facebook is one of the best ways for promotion these days
as you don't have to go so far out of youur way to check the page
yeah
soundcloud has an embeddable player and their free tier allows around 1hr of content