brycec: But there are more efficent ways to copy "many small files"
(as in the disk keeps going to A, then, B, then C, back to A, back to B, back to C...) mnathani: right. brycec: (versus: disk spends time reading "A", then moves to reading "B")
mnathani: I've found tar to be the most efficient and fastest handler of "many small files"
tar -c /my/dir | tar -xC /dest/dir
(make it xvC if you feel like wasting blocking CPU time on writing out progress) mercutio: there's not many multithreaded copiers for linux
and it's a complicated problem.
i think it was gizmoguy that linked me something before?
brycec: cp is actually faster than tar normally for many small files gizmoguy: hi mercutio: gizmoguy: you ahd some kind of multithreaded copy thing right?
i can't for the life of me think who else it would have been.
there's ways with xargs etc too gizmoguy: grid ftp? mercutio: i don't think it would have been grid ftp
gridftp uses udt doesn't it?
the naem does ring a bell
oh this is windows
i should have read scroll back better
if you're booting it into linux to copy, ntfs under linux sucks.
and you'd probably be better off using dd if you acn
hmm gnu parallel looks like one solution gizmoguy: hah i just have my laptop open and I saw irc go purple
I read your message and no other context :) mercutio: ahh ok
i can't for the life of me find it now gizmoguy: hmm, read scrollback now
I see we want to do it on a local machine
can't think of what it might have been if I linked you
I usually use rsync, but it kinda sucks for lots of small files mercutio: maybe it was someone else
the solution using parallel looks easyish to do
and you can rsync in a mode that just lists the files, but that's still slow
for backup so you can reimage or something, i would opt for dd/lz4 myself
ie dd it to a file, piped through lz4 which doesn't really slow it down ***: novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks
staticsafe_ has joined #arpnetworks
staticsafe|2 has quit IRC (Ping timeout: 264 seconds)
NiTeMaRe has quit IRC (Ping timeout: 258 seconds)
NiTeMaRe has joined #arpnetworks
staticsafe_ is now known as staticsafe|2
novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks jpalmer: mnathani: my guess is, threading will cause higher disk access.. meaning you start having to add seek times into the mix. every single time I've ever done a multithreaded file copy of any size, it's slower than just doing it single threaded.
mnathani: and, I just realized I was scrolled up. ignore the half-a-day-late response. chances are good, you've already finished the procedure. LOL m0unds: i started to do the same thing, haha
haha: http://www.pcgamer.com/review/mad-dog-mccree-review-april-2003-us-edition/ ***: toddf has quit IRC (Read error: Connection reset by peer)
toddf has joined #arpnetworks
ChanServ sets mode: +o toddf brycec: #devious loves ARP Networks http://sprunge.us/XPHK ***: RandalSchwartz has quit IRC (Remote host closed the connection) m0unds: hahaha ***: RandalSchwartz has joined #arpnetworks RandalSchwartz: -NickServ- Last failed attempt from: merlyn!~merlyn@vps1.merlyn.cz on May - 27 17:57:12 2014.
heh... lotta merlyns there up_the_irons: brycec: lol cool brycec: fwiw up_the_irons he found it "pricey", so I wouldn't hold my breath up_the_irons: yeah np brycec: He then turned to look at getting OpenBSD running on a free EC2 instance
so... *shrug* up_the_irons: ah brycec: Frankly if he just wants to host a blog, a full blown VPS isn't a good plan. up_the_irons: right ant: up_the_irons: ich vermisse dich in #arpnetworks-de ;) brycec: heh -: brycec joins #arpnetworks-en ant: lol up_the_irons: lol ant: i would join -es ;) up_the_irons: ant: ich neu gestartet weechat
we now have #arpnetworks-es brycec: ay carumba! -: brycec is quickly exhausting the Spanish he remembers up_the_irons: lol ant: according to duolingo i should know about 1k works ***: mikeputnam has joined #arpnetworks brycec: heh
wb mikeputnam ant: but i found that that's actually not much... brycec: I can read and understand Spanish decently
But my vocabulary for speaking is limited to getting by in the Southwestern US qbit: >.>
WHOAMI!? brycec: You are you. m0unds: yea, mine's limited to limited reading comprehension
most SW spanish speakers tend to speak way too fast for me to pick it up RandalSchwartz: I'm studying about 15-20 minutes a day, slowly getting more things understood
was using rosetta stone, but I wanted something with a bit more handholding, so I'm usng duolingo -: brycec sighs, one of these days I'll get back to German... ant: i'm taking spanish at my university. now the spanish teachers also speak spanish in class...talking about "way too fast"... brycec: I learned French in high school by immersion. No better way :) But that's at a high school pace... about 1/4-1/2 college speed. up_the_irons: brycec: how did you immerse? did you go to HS in France? brycec: up_the_irons: The teacher would only speak French with us RandalSchwartz: but never listen french? brycec: :P
For ~1hr a day, I only spoke and heard (and read and wrote) French RandalSchwartz: or maybe you meant "speak *only* french" brycec: Peut-etre
peut-être (avec des accents) m0unds: my first semester of spanish in HS was taught by actions and stuff
it's the only stuff i remember brycec: Cher KVM, pourqoi refusez-vous de fermer le dossier?
lol m0unds
FERMEZ LE DOSSIER m0unds: second semester was a different teacher and her method was bookwork and forcing us to learn ricky martin songs brycec: hahaha
m0unds: What about other spanish songs? Like the Ketch Song?
*Ketchup m0unds: haha, nope
just ricky martin
and making spanish restaurant menus RandalSchwartz: the song that is "rapper's delight" phonetically? brycec: I'm referring to https://www.youtube.com/watch?v=V0PisGe66mY RandalSchwartz BryceBot: YouTube Music: "Las Ketchup - The Ketchup Song (Asereje) (Spanish Version) (Official Video)" by AltraModaMusic (3m 31s), 5,679,178 views, 17,033 likes and 1,086 dislikes. Uploaded 2013-04-13T09:03:35.000Z. m0unds: http://i.imgur.com/nnANCJk.gif
:| RandalSchwartz: yesh - that's the one
from the daughters of "the big tomato"
which is why they call themselves "Ketchup"
but the chorus of that song is a phonetic translation of "Rapper's delight"
they've admitted that brycec: Sure...
I had no idea
Only reason I know the song is from my French class, oddly. One of the "goof off" lessons, and an exercise in singing in !English up_the_irons: phlux: what menu prog is that on top of: http://img.phluxbox.com/screenshots/aic5ah.png ?
dzen2? mnathani: jpalmer: Thanks on your input. Its a recurring issue for me, I plan on using the tar method suggested by brycec next time around brycec: mnathani: What exactly is the use case? Is this something that Clonezilla (and its tools) might be better suited for? up_the_irons: brycec: how good did you get at French with that 1 hr / day for 4(?) years? phlux: up_the_irons: it's just i3status with conky ***: novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks brycec: up_the_irons: Pretty good, actually. Those 4 years were roughly equivalent to 4 semester of college French. up_the_irons: phlux: cool
brycec: cool
:) brycec: up_the_irons: cool up_the_irons: phlux: what's your conky config to produce the one-liner? conky setups i've seen look more like: http://fc00.deviantart.net/fs70/i/2010/169/a/f/Compiz_empowered_desktop_by_laserbeam3.png
(which is quite cool in itself ;) acf_: Hey up_the_irons. any update on level3? up_the_irons: acf_: they finished provisioning new VLAN IDs (took 'em a long while) that don't conflict with our network. Now the ball is in my court to terminate the VLANs, BGP setup, etc...
i have a host upgrade to do tonight, so i'm thinking i'm going to do the bgp stuff tomorrow night brycec: Which host is on the chopping block tonight? mercutio: oh cool soon now then :) acf_: I'm certainly looking forward to it
hopefully Verizon will go that way :/ mercutio: i am just curious really
i dunno what diff it'll make to me acf_: looks like the NTT/Verizon dispute is still ongoing: http://goo.gl/hTrm8l BryceBot: http://goo.gl/hTrm8l -> http://kremvax.acfsys.net/smokeping.cgi?displaymode=n;start=2013-06-01%2018:58;end=now;target=Remote.verizon-lsanca mercutio: acf: these things arel ikely to get worse ratehr than better acf_: seems that way. this is the first one that's really impacted me though mercutio: cox and comcast have both had bad routes to cogent for ages,
that's not actually extremely bad
i've seen worse congestion than that before
that's about what new zealand users on dsl used to get on average everwhere
cos there was this handover dimensioning for dsl, which was like 48kbit per user or something
and the only way for isp's to have good speeds were to have lots of low volume users, which is silly.
it's still actually happening for some rural users, and apparently it's wosre acf_: wow. I guess I'm just used to the mostly loss-free links from big transit providers mercutio: transit is fine :)
and this problem was a few yaers ago
but it affected a LOT of users
cos DSL is common here, and monopoly etc with wholesaling
and then the wholesaler had higher dimensioning for their own connections
ie ... really bad situation.. but no-one can do anything about it
it's one of the reasons nz had tiny data caps
it was the "best" solution acf_: seems like it was mostly caused because of the lack of good backhaul? mercutio: it wasn't just that even
it was between the wholesaler provider and the isp
with actual rate limits per user
so not just straight backhaul congestino acf_: hmm. but that's why the users were limited so strictly? mercutio: nah, it's cos they could :/
i dunno
it used to cost $500+/month for 2 megabit frame relay here
then dsl came in which was a lot less than $500/month
but had 300mb and 600mb and so on data caps phlux: up_the_irons: mind if I send it to you tomorrow? i'm signing off here tonight - crying baby needs attention! mercutio: but i think telecommunicatins companies always charge what they can
if no competition prices increase
if limited competition prices tend to stagnate acf_: definitely seems that way in the US mercutio: you really need three or more companies to really get competition
and one of them beign new helps
ie "needding to get market share"
but it tends to mean one of the two larger companies has money to burn, and the other doesn't, and so one of them will drop prices even further.. to stop the third party entering the market
andt he second party struggles
and then you find cheaper internet of much lower quality
will be curious if gigabit becomes common if a whole lot of backhaul problems get worse
cos i imagine traffic will be able to move easier
ie come in different ways than expected acf_: common for consumers?
afaik Google is trying to make that happen ***: dj_goku has quit IRC (Quit: Lost terminal)
dj_goku has joined #arpnetworks
dj_goku has quit IRC (Changing host)
dj_goku has joined #arpnetworks jpalmer: mnathani: if you are sending that data to another machine, I personally prefer rsync and friends. that way if the connection gets interuppted (network hiccup, disk or machine going to sleep, power, human error, etc) the data thats already been transferred, says sane, and it just resumes roughly where the interuption occured.
s/says/stays/ BryceBot: <jpalmer> mnathani: if you are sending that data to another machine, I personally prefer rsync and friends. that way if the connection gets interuppted (network hiccup, disk or machine going to sleep, power, human error, etc) the data thats already been transferred, stays sane, and it just resumes roughly where the interuption occured. -: jpalmer pets BryceBot mercutio: it's actually faster to use tar and rsync if it aborts
it's quicker to use tar/rsync combination if you want an up to date rsync from scractch too up_the_irons: phlux: sure of course
brycec: kvr22 is on the chopping block tonight -: brycec nods
brycec decides his next host naming scheme will be based on MacGuyver and his companions brycec: *MacGyver m0unds: haha, will you have one named ANGUS? brycec: To my knowledge, he's not had a companion named Angus...
(but I'm no expert)
Oh god... MacGyver (as a show) just showed its age when MacGyver "hacked" his way past a remote-controlled lock by resonating crystal wine glasses. mercutio: yeah now days people would just use a hammer brycec: perhaps, but then there would be damage to be found mercutio: yeh
but it's mostly drug addicts that break into things now days
covert stuff probably happens from inside
i hear about "chinese hackers" quite a lot that get into things
i don't thin it's that common though
there's just a desire to blame the chinese :) brycec: In this instance, it was a private vault with 200lbs of diamonds mercutio: heh
with a frequency based lock
i didn't realise macgyver stole things brycec: Seriously, who the fuck locks/unlocks their vault with the equivalent of a TV remote?
Only in Robin Hood fashion mercutio: i dunno isn't it mostly retina based now? brycec: And he's not stealking, he's moving mercutio: i haven't seen the show in years brycec: Last vault (bank) I was in, I only saw keys mercutio: wow youtube has full episodes -: brycec is watching on Netflix mercutio: ahh can't do that here -: brycec takes it for granted mercutio: hah he has a huge phone brycec: Radiophone to be specific mercutio: oh he was at home
i have no idea what epsideo number you are on :) brycec: oh was he? I have no idea which ep you're watching :p mercutio: is the video quality ok?
i was watching episode 14 :/
i dunno why it sarted there heh brycec: season 1, ep 5. ' mercutio: but the video quality is terrible :/ brycec: And yeah, perfect quality (for the time) mercutio: just really artifacty brycec: Bummer mercutio: we can't get netflix here :/
"Sorry, Netflix is not available in your country yet." brycec: Yeah, I know. You could tunnel through ARP though, but payment may still be tricky. mercutio: yeah i can tunnel i know brycec: Or you could pir8 <.<
>.> mercutio: i tried it wit that onther oe :/
but there were heaps of adverts brycec: you mean Hulu? mercutio: yeah that's it brycec: Yeah well... Hulu sucks mercutio: oh is netflix better brycec: Substantially mercutio: i assumed it was much of a muchness brycec: Nooooo mercutio: do you get adverts on netflix? brycec: For starters, no ads
not in the slightest
no banners, no videos, NOTHING
Since Netflix is 100% paid, there's nothing to be gained from having advertisers
(not to mention the ill-will about it)
Hulu on the other hand is owned and run by Old World media, TV networks accustomed to advertising as a significant part of the revenue mercutio: i see
yeh the ads put me off
and limited content brycec: Yes and yes mercutio: how much is netflix? brycec: $8/mo last I checked mercutio: oh
that's not muc brycec: unlimited streaming, etc ***: doomviking has quit IRC (Ping timeout: 240 seconds) brycec: "Netflix is the Undisputed TV Streaming King" http://lifehacker.com/tv-streaming-head-to-head-netflix-vs-hulu-vs-amazon-pr-1536006625 mercutio: oh apparently netflix changed something that broke some people proxying brycec: (can't blame them... Netflix is liable when you break Netflix's agreements with their content providers, the ones stipulating which countries their content is available in. (Yes, licensing is just plain dumb in most every way.)) mercutio: http://www.stuff.co.nz/technology/digital-living/60080388/netflix-nz-glitch-unintentional brycec: So netflix changed their hostname scheme, and that broke $proxies squid rules, lol