#arpnetworks 2014-05-27,Tue

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
brycecmnathani: thing is if you start running multiple copies, your disk is going to spend more and more time grabbing data from file A, B, and C
But there are more efficent ways to copy "many small files"
(as in the disk keeps going to A, then, B, then C, back to A, back to B, back to C...)
[00:00]
mnathaniright. [00:02]
brycec(versus: disk spends time reading "A", then moves to reading "B")
mnathani: I've found tar to be the most efficient and fastest handler of "many small files"
tar -c /my/dir | tar -xC /dest/dir
(make it xvC if you feel like wasting blocking CPU time on writing out progress)
[00:02]
.......... (idle for 48mn)
mercutiothere's not many multithreaded copiers for linux
and it's a complicated problem.
i think it was gizmoguy that linked me something before?
brycec: cp is actually faster than tar normally for many small files
[00:51]
gizmoguyhi [00:52]
mercutiogizmoguy: you ahd some kind of multithreaded copy thing right?
i can't for the life of me think who else it would have been.
there's ways with xargs etc too
[00:52]
gizmoguygrid ftp? [00:54]
mercutioi don't think it would have been grid ftp
gridftp uses udt doesn't it?
the naem does ring a bell
oh this is windows
i should have read scroll back better
if you're booting it into linux to copy, ntfs under linux sucks.
and you'd probably be better off using dd if you acn
hmm gnu parallel looks like one solution
[00:54]
gizmoguyhah i just have my laptop open and I saw irc go purple
I read your message and no other context :)
[01:02]
mercutioahh ok
i can't for the life of me find it now
[01:03]
gizmoguyhmm, read scrollback now
I see we want to do it on a local machine
can't think of what it might have been if I linked you
I usually use rsync, but it kinda sucks for lots of small files
[01:05]
mercutiomaybe it was someone else
the solution using parallel looks easyish to do
and you can rsync in a mode that just lists the files, but that's still slow
for backup so you can reimage or something, i would opt for dd/lz4 myself
ie dd it to a file, piped through lz4 which doesn't really slow it down
[01:07]
***novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks
[01:12]
................................ (idle for 2h39mn)
staticsafe_ has joined #arpnetworks
staticsafe|2 has quit IRC (Ping timeout: 264 seconds)
[03:53]
.............. (idle for 1h8mn)
NiTeMaRe has quit IRC (Ping timeout: 258 seconds)
NiTeMaRe has joined #arpnetworks
staticsafe_ is now known as staticsafe|2
[05:03]
.......................... (idle for 2h5mn)
novae has quit IRC (Ping timeout: 245 seconds)
novae has joined #arpnetworks
[07:16]
.... (idle for 18mn)
jpalmermnathani: my guess is, threading will cause higher disk access.. meaning you start having to add seek times into the mix. every single time I've ever done a multithreaded file copy of any size, it's slower than just doing it single threaded.
mnathani: and, I just realized I was scrolled up. ignore the half-a-day-late response. chances are good, you've already finished the procedure. LOL
[07:37]
...... (idle for 29mn)
m0undsi started to do the same thing, haha [08:06]
haha: http://www.pcgamer.com/review/mad-dog-mccree-review-april-2003-us-edition/ [08:12]
......... (idle for 43mn)
***toddf has quit IRC (Read error: Connection reset by peer)
toddf has joined #arpnetworks
ChanServ sets mode: +o toddf
[08:55]
.................................................... (idle for 4h15mn)
brycec#devious loves ARP Networks http://sprunge.us/XPHK [13:12]
***RandalSchwartz has quit IRC (Remote host closed the connection) [13:13]
m0undshahaha [13:18]
***RandalSchwartz has joined #arpnetworks [13:18]
RandalSchwartz-NickServ- Last failed attempt from: merlyn!~merlyn@vps1.merlyn.cz on May - 27 17:57:12 2014.
heh... lotta merlyns there
[13:18]
............... (idle for 1h10mn)
up_the_ironsbrycec: lol cool [14:28]
brycecfwiw up_the_irons he found it "pricey", so I wouldn't hold my breath [14:29]
up_the_ironsyeah np [14:29]
brycecHe then turned to look at getting OpenBSD running on a free EC2 instance
so... *shrug*
[14:29]
up_the_ironsah [14:29]
brycecFrankly if he just wants to host a blog, a full blown VPS isn't a good plan. [14:29]
up_the_ironsright [14:29]
antup_the_irons: ich vermisse dich in #arpnetworks-de ;) [14:36]
brycecheh
brycec joins #arpnetworks-en
[14:37]
antlol [14:38]
up_the_ironslol [14:38]
anti would join -es ;) [14:39]
up_the_ironsant: ich neu gestartet weechat
we now have #arpnetworks-es
[14:40]
brycecay carumba!
brycec is quickly exhausting the Spanish he remembers
[14:42]
up_the_ironslol [14:44]
antaccording to duolingo i should know about 1k works [14:45]
***mikeputnam has joined #arpnetworks [14:45]
brycecheh
wb mikeputnam
[14:45]
antbut i found that that's actually not much... [14:45]
brycecI can read and understand Spanish decently
But my vocabulary for speaking is limited to getting by in the Southwestern US
[14:46]
qbit>.>
WHOAMI!?
[14:48]
brycecYou are you. [14:50]
m0undsyea, mine's limited to limited reading comprehension
most SW spanish speakers tend to speak way too fast for me to pick it up
[15:02]
RandalSchwartzI'm studying about 15-20 minutes a day, slowly getting more things understood
was using rosetta stone, but I wanted something with a bit more handholding, so I'm usng duolingo
[15:03]
brycecbrycec sighs, one of these days I'll get back to German... [15:04]
anti'm taking spanish at my university. now the spanish teachers also speak spanish in class...talking about "way too fast"... [15:06]
brycecI learned French in high school by immersion. No better way :) But that's at a high school pace... about 1/4-1/2 college speed. [15:10]
..................... (idle for 1h40mn)
up_the_ironsbrycec: how did you immerse? did you go to HS in France? [16:50]
brycecup_the_irons: The teacher would only speak French with us [16:53]
RandalSchwartzbut never listen french? [16:53]
brycec:P
For ~1hr a day, I only spoke and heard (and read and wrote) French
[16:53]
RandalSchwartzor maybe you meant "speak *only* french" [16:54]
brycecPeut-etre
peut-ĂȘtre (avec des accents)
[16:54]
m0undsmy first semester of spanish in HS was taught by actions and stuff
it's the only stuff i remember
[16:55]
brycecCher KVM, pourqoi refusez-vous de fermer le dossier?
lol m0unds
FERMEZ LE DOSSIER
[16:56]
m0undssecond semester was a different teacher and her method was bookwork and forcing us to learn ricky martin songs [16:56]
brycechahaha
m0unds: What about other spanish songs? Like the Ketch Song?
*Ketchup
[16:56]
m0undshaha, nope
just ricky martin
and making spanish restaurant menus
[16:57]
RandalSchwartzthe song that is "rapper's delight" phonetically? [16:57]
brycecI'm referring to https://www.youtube.com/watch?v=V0PisGe66mY RandalSchwartz [16:58]
BryceBotYouTube Music: "Las Ketchup - The Ketchup Song (Asereje) (Spanish Version) (Official Video)" by AltraModaMusic (3m 31s), 5,679,178 views, 17,033 likes and 1,086 dislikes. Uploaded 2013-04-13T09:03:35.000Z. [16:58]
m0undshttp://i.imgur.com/nnANCJk.gif
:|
[16:58]
RandalSchwartzyesh - that's the one
from the daughters of "the big tomato"
which is why they call themselves "Ketchup"
but the chorus of that song is a phonetic translation of "Rapper's delight"
they've admitted that
[16:58]
brycecSure...
I had no idea
Only reason I know the song is from my French class, oddly. One of the "goof off" lessons, and an exercise in singing in !English
[16:59]
..... (idle for 21mn)
up_the_ironsphlux: what menu prog is that on top of: http://img.phluxbox.com/screenshots/aic5ah.png ?
dzen2?
[17:21]
.... (idle for 19mn)
mnathanijpalmer: Thanks on your input. Its a recurring issue for me, I plan on using the tar method suggested by brycec next time around [17:41]
brycecmnathani: What exactly is the use case? Is this something that Clonezilla (and its tools) might be better suited for? [17:44]
......... (idle for 42mn)
up_the_ironsbrycec: how good did you get at French with that 1 hr / day for 4(?) years? [18:26]
phluxup_the_irons: it's just i3status with conky [18:36]
***novae has quit IRC (Ping timeout: 245 seconds) [18:42]
novae has joined #arpnetworks [18:49]
brycecup_the_irons: Pretty good, actually. Those 4 years were roughly equivalent to 4 semester of college French. [18:57]
up_the_ironsphlux: cool
brycec: cool
:)
[18:58]
brycecup_the_irons: cool [18:58]
up_the_ironsphlux: what's your conky config to produce the one-liner? conky setups i've seen look more like: http://fc00.deviantart.net/fs70/i/2010/169/a/f/Compiz_empowered_desktop_by_laserbeam3.png
(which is quite cool in itself ;)
[19:01]
acf_Hey up_the_irons. any update on level3? [19:04]
up_the_ironsacf_: they finished provisioning new VLAN IDs (took 'em a long while) that don't conflict with our network. Now the ball is in my court to terminate the VLANs, BGP setup, etc...
i have a host upgrade to do tonight, so i'm thinking i'm going to do the bgp stuff tomorrow night
[19:05]
brycecWhich host is on the chopping block tonight? [19:06]
mercutiooh cool soon now then :) [19:07]
acf_I'm certainly looking forward to it
hopefully Verizon will go that way :/
[19:08]
mercutioi am just curious really
i dunno what diff it'll make to me
[19:08]
acf_looks like the NTT/Verizon dispute is still ongoing: http://goo.gl/hTrm8l [19:09]
BryceBothttp://goo.gl/hTrm8l -> http://kremvax.acfsys.net/smokeping.cgi?displaymode=n;start=2013-06-01%2018:58;end=now;target=Remote.verizon-lsanca [19:09]
mercutioacf: these things arel ikely to get worse ratehr than better [19:10]
acf_seems that way. this is the first one that's really impacted me though [19:11]
mercutiocox and comcast have both had bad routes to cogent for ages,
that's not actually extremely bad
i've seen worse congestion than that before
that's about what new zealand users on dsl used to get on average everwhere
cos there was this handover dimensioning for dsl, which was like 48kbit per user or something
and the only way for isp's to have good speeds were to have lots of low volume users, which is silly.
it's still actually happening for some rural users, and apparently it's wosre
[19:20]
acf_wow. I guess I'm just used to the mostly loss-free links from big transit providers [19:24]
mercutiotransit is fine :)
and this problem was a few yaers ago
but it affected a LOT of users
cos DSL is common here, and monopoly etc with wholesaling
and then the wholesaler had higher dimensioning for their own connections
ie ... really bad situation.. but no-one can do anything about it
it's one of the reasons nz had tiny data caps
it was the "best" solution
[19:24]
acf_seems like it was mostly caused because of the lack of good backhaul? [19:26]
mercutioit wasn't just that even
it was between the wholesaler provider and the isp
with actual rate limits per user
so not just straight backhaul congestino
[19:27]
acf_hmm. but that's why the users were limited so strictly? [19:27]
mercutionah, it's cos they could :/
i dunno
it used to cost $500+/month for 2 megabit frame relay here
then dsl came in which was a lot less than $500/month
but had 300mb and 600mb and so on data caps
[19:28]
phluxup_the_irons: mind if I send it to you tomorrow? i'm signing off here tonight - crying baby needs attention! [19:29]
mercutiobut i think telecommunicatins companies always charge what they can
if no competition prices increase
if limited competition prices tend to stagnate
[19:29]
acf_definitely seems that way in the US [19:30]
mercutioyou really need three or more companies to really get competition
and one of them beign new helps
ie "needding to get market share"
but it tends to mean one of the two larger companies has money to burn, and the other doesn't, and so one of them will drop prices even further.. to stop the third party entering the market
andt he second party struggles
and then you find cheaper internet of much lower quality
will be curious if gigabit becomes common if a whole lot of backhaul problems get worse
cos i imagine traffic will be able to move easier
ie come in different ways than expected
[19:30]
acf_common for consumers?
afaik Google is trying to make that happen
[19:33]
..... (idle for 24mn)
***dj_goku has quit IRC (Quit: Lost terminal)
dj_goku has joined #arpnetworks
dj_goku has quit IRC (Changing host)
dj_goku has joined #arpnetworks
[19:58]
jpalmermnathani: if you are sending that data to another machine, I personally prefer rsync and friends. that way if the connection gets interuppted (network hiccup, disk or machine going to sleep, power, human error, etc) the data thats already been transferred, says sane, and it just resumes roughly where the interuption occured.
s/says/stays/
[20:02]
BryceBot<jpalmer> mnathani: if you are sending that data to another machine, I personally prefer rsync and friends. that way if the connection gets interuppted (network hiccup, disk or machine going to sleep, power, human error, etc) the data thats already been transferred, stays sane, and it just resumes roughly where the interuption occured. [20:02]
jpalmerjpalmer pets BryceBot [20:02]
....... (idle for 33mn)
mercutioit's actually faster to use tar and rsync if it aborts
it's quicker to use tar/rsync combination if you want an up to date rsync from scractch too
[20:35]
....................... (idle for 1h53mn)
up_the_ironsphlux: sure of course
brycec: kvr22 is on the chopping block tonight
[22:29]
..... (idle for 24mn)
brycecbrycec nods
brycec decides his next host naming scheme will be based on MacGuyver and his companions
*MacGyver
[22:56]
m0undshaha, will you have one named ANGUS? [23:06]
brycecTo my knowledge, he's not had a companion named Angus...
(but I'm no expert)
[23:13]
..... (idle for 23mn)
Oh god... MacGyver (as a show) just showed its age when MacGyver "hacked" his way past a remote-controlled lock by resonating crystal wine glasses. [23:36]
mercutioyeah now days people would just use a hammer [23:37]
brycecperhaps, but then there would be damage to be found [23:39]
mercutioyeh
but it's mostly drug addicts that break into things now days
covert stuff probably happens from inside
i hear about "chinese hackers" quite a lot that get into things
i don't thin it's that common though
there's just a desire to blame the chinese :)
[23:39]
brycecIn this instance, it was a private vault with 200lbs of diamonds [23:41]
mercutioheh
with a frequency based lock
i didn't realise macgyver stole things
[23:41]
brycecSeriously, who the fuck locks/unlocks their vault with the equivalent of a TV remote?
Only in Robin Hood fashion
[23:41]
mercutioi dunno isn't it mostly retina based now? [23:42]
brycecAnd he's not stealking, he's moving [23:42]
mercutioi haven't seen the show in years [23:42]
brycecLast vault (bank) I was in, I only saw keys [23:42]
mercutiowow youtube has full episodes [23:43]
brycecbrycec is watching on Netflix [23:43]
mercutioahh can't do that here [23:43]
brycecbrycec takes it for granted [23:43]
mercutiohah he has a huge phone [23:44]
brycecRadiophone to be specific [23:45]
mercutiooh he was at home
i have no idea what epsideo number you are on :)
[23:45]
brycecoh was he? I have no idea which ep you're watching :p [23:45]
mercutiois the video quality ok?
i was watching episode 14 :/
i dunno why it sarted there heh
[23:45]
brycecseason 1, ep 5. ' [23:45]
mercutiobut the video quality is terrible :/ [23:45]
brycecAnd yeah, perfect quality (for the time) [23:46]
mercutiojust really artifacty [23:46]
brycecBummer [23:47]
mercutiowe can't get netflix here :/
"Sorry, Netflix is not available in your country yet."
[23:47]
brycecYeah, I know. You could tunnel through ARP though, but payment may still be tricky. [23:48]
mercutioyeah i can tunnel i know [23:48]
brycecOr you could pir8 <.<
>.>
[23:48]
mercutioi tried it wit that onther oe :/
but there were heaps of adverts
[23:48]
brycecyou mean Hulu? [23:48]
mercutioyeah that's it [23:48]
brycecYeah well... Hulu sucks [23:49]
mercutiooh is netflix better [23:49]
brycecSubstantially [23:49]
mercutioi assumed it was much of a muchness [23:49]
brycecNooooo [23:49]
mercutiodo you get adverts on netflix? [23:49]
brycecFor starters, no ads
not in the slightest
no banners, no videos, NOTHING
Since Netflix is 100% paid, there's nothing to be gained from having advertisers
(not to mention the ill-will about it)
Hulu on the other hand is owned and run by Old World media, TV networks accustomed to advertising as a significant part of the revenue
[23:49]
mercutioi see
yeh the ads put me off
and limited content
[23:51]
brycecYes and yes [23:51]
mercutiohow much is netflix? [23:51]
brycec$8/mo last I checked [23:51]
mercutiooh
that's not muc
[23:51]
brycecunlimited streaming, etc [23:52]
***doomviking has quit IRC (Ping timeout: 240 seconds) [23:52]
brycec"Netflix is the Undisputed TV Streaming King" http://lifehacker.com/tv-streaming-head-to-head-netflix-vs-hulu-vs-amazon-pr-1536006625 [23:52]
mercutiooh apparently netflix changed something that broke some people proxying [23:53]
brycec(can't blame them... Netflix is liable when you break Netflix's agreements with their content providers, the ones stipulating which countries their content is available in. (Yes, licensing is just plain dumb in most every way.)) [23:55]
mercutiohttp://www.stuff.co.nz/technology/digital-living/60080388/netflix-nz-glitch-unintentional [23:57]
brycecSo netflix changed their hostname scheme, and that broke $proxies squid rules, lol [23:59]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)