if you are looking for secure backups and open source small plug for cyphertite.com ;-)
but then if you're here at arpnetworks and like your own custom backup software, there's always the backup service arpnetworks is beta'ing
personally dropbox is great if you have multiple people sharing things, I have relatives upload photos to me via dropbox so they can share pics of our kids with us etc .. business use includes sharing documents w/out giving them to google or having a dozen and one email exchanges about which document is most recent .. as a backup service? I'd kinda be leery but if you encrypt and the storage space is the right price (read: free ...
... for the first Xmb and costs after that (you can earn more via referrals but anyway)
i use spideroak for backup stuff, and duplicity + s3 for server stuff
I have looked at s3 a number of times, can't get over the bandwidth xfer costs. maybe I'm spoiled but having $10/mo for personal use regardless of bandwidth or storage footprint and $$/gb for business use regardless of bandwidth seems like a deal to me (and yes I'm talking about cyphertite again)
haha
i'm all about 11 9s' durability and availability
9's
i think i pay ~$8/mo for 100GB on spideroak, but that's more for personal stuff. my AWS bill is tiny since i don't store a lot on servers i run.
i use route53 for DNS on 6 zones and like 10-12GB of backups, and it's $4ish/mo. something like that
I have 700GB on my personal cyphertite account and climbing. $10/mo.
ah
well, i don't even have that much data (yet) on my local NAS, haha
and that's a 20TB array
kid videos and pics and archives of multimedia we'd rather not loose .. constitutes the bulk of that
yeah, video adds up fast
yea dropbox is one of those thinsg
(my big nas will eventually be used as a home surveillance system data store)
where if i wanted to upload crap i don't care about but need to share
like a personal pastebin with some larger non text files
easy to share, standard wgettable https url
but anyhting like (non f/oss) source code, user data, customer data, sql db dumps
is not a thing for db
I have a friend who was actually using it as a backing store for an accounting package he was using for personal use. I was (and still am) totally wierded out by that.
as if the dropbox had become a file xfer protocol for higher level apps
haha
YNAB uses it for synchronization
bummed about that - i'll probably figure out a way to work around it because i don't like dropbox
so in a year, the difference between
99.9999 and 99.999999999
thousanth of an hour
31.5569 seconds vs 315.569 microseconds
m0unds: copy.com (baracuda networks company) has a dropbox-alike,  25gb free
well, 20,   but you can use a referal link to signup..  and you each get an additional 5
ah
I don't think it's headless though, m0unds jpalmer
(Which was something originally sought)
Also, I will chime in that Dropbox is a terrible backup service. The advantage is quick and easy way to disperse your backups geographically (if you have many machines around the world), but you'd definitely want to look at selective sync so that all the machines backing up to it aren't storing the other machines' backups... that would get very large very quickly.
Side-note: I've used Copy with no problems, but only sparingly.
Also, ++ for duplicity and S3. My Amazon bill is around $15/mo for several-hundred GB. It's definitely a nice piece of mind to have the data stored offsite, admin'd by someone !me, and as easily configured as duplicity is.
brycec: does copy also have a proprietary bin app as opposed to an open protocol?
yes, believe it's proprietary
I know it's pretty large too, compared to DB (50Mb vs 8Mb or so)
let me know if there is an open protocol storage arena. fusefs + google drive seems to be about the only game in town that direction. maybe someone has done fusefs + s3 but as stated above I don't like xfer costs.
I've recently started playing with Owncloud and its dropbox-like sync client. It's open (built on top of webdav), and pretty sure it can be 100% headless too.
I saw that in the ports tree. thats storage on ones own disks though, right? not exactly public Owncloud servers out there (aside from inside tor..) or are there?
Yeah it's self-run. I'm sure there are hosted options, but probably not worth it.
But as far as self-run Dropbox-clones go, it's nice to have an open option.
does it scale? aka can I have 10 Owncloud servers and they seamlessly migrate data between each other? tell one 'you are going down' and have it push the data to the others?
Couldn't say. I've seen mentions of scaling/clustering, but haven't explored it yet.
Have you checked out tarsnap?
Think it's open protocol (rsync/ssh), and they host it.
(I think they might just be backing on to S3 themselves, but that's not your problem)
"The Tarsnap service is built on top of the solid platform provided by Amazon Web Services. Data archived via Tarsnap is stored to the Amazon S3 storage service"
apparently, no free accounts though ;-(
oh you didn't say you wanted free too
gosh! :p
heh
well, i should be able to try before I buy of course
tarsnap seems very similar to cyphertite from a user perspective, except cyphertite has a free 8gb account
looks like owncloud has concept of migration and backup, suggesting one manually moves accounts between instances for backup and redundancy, not quite what I was hoping for
but hey, the owncloud bits might be interesting anyway
You can always put owncloud on top of a something reliable, like glusterfs with replica=N drives, essentially RAID1 the data across each host.
sure
^ What I'm doing here for the VM cluster. Plus each system has local access to the data.
I'm basically in limbo with data storage at home
Generous employer?
I have some old openbsd systems serving openafs that I need to migrate away from, it did seamless movement of data between servers, consistent namespace, kerberos auth, acls, nice bits
however the openbsd afs client is EOL
I have an openafs update that will let me run modern openbs afs servers but no afs client as of yet
the fuse in openbsd might work if openafs let its afsd build with fuse support, not quite beaten that horse properly yet
Why migrate from AFS then? Why not migrate to something (a *NIX( that still supports it?
ah, gotcha
the lkm afsd panics after brief use from openafs
so I've been suggested just use nfs
great, amd + nfs ends me in hung mounts and bad organization due to inconsistent namespace unless I introduce symlink hell
yeah, NFS not a substitute.
so i've got some nfs storage and afs storage and local disk storage on a bunch of systems but really need something as a plan other than 'archive it all to cyphertite, find something better to store it on, and download what I use again'
Though GlusterFS's NFS brings some of that back - consistent namespace, reliable storage. Only downside is that you're still hitting just a single server, though you can hit any of the servers in the cluster for the same NFS access.
(the "mount a single server" is inherent to NFS and NFS clients specifically. If the NFS client could hit multiple servers... )
what I truly want is something that runs on openbsd that borrows the persistent namespace model of afs and the redundancy of e.g. glusterfs or whatever
(sortof like parallel-nfs, but it's not the same)
exactly
Well good luck :)
the fs client should be like a replicated database access proxy, os sees consistent filespace, fs is replicated behind the scenes to the users desire and storage space utilized where storage daemons exist, in whatever capacity local storage exists for storage daemons
if I had eyons of time I'd write my own secure network filessytem for unix (someone can take care to shoehorn it into windows if they wish) that had those properties, I know enough about how things can go wrong with the network that I would think it could be quite useful. however, time is not something i have an abundance of by any stretch.
You've described Gluster :p
so of course, you can see, anything else is just .. falling short ;-(
(I forgot to mount that, if the machine is a node of the cluster, doesn't even have to be hosting storage, you can mount localhost)
(Even both the FUSE client/connection, and NFS)
someone point me to a port to openbsd. *grin*.
there used to exist this page http://www.gluster.org/docs/index.php/GlusterFS_on_BSD
and there's apparently FreeBSD work on it https://wiki.freebsd.org/GlusterFS
I guess if you were desperate, you could run a qemu Linux VM to expose storage
thats rather dated
thats how I started with iscsi till I found iscsi-target in ports
lol
It's how I exported a Linux mdadm volume into an OpenBSD host :p
(on an old single-core box. It was dreadful.)
hmm, glusterfs seems to operate based on a file, not blocks, so a file cannot be larger than the subvolume it happens to randomly be hashed to
if I were doing it the blocks of the file would be distributed, and deduped and compressed to add to the fun
But on the up-side, data-recovery. It just sits on top of a filesystem. If, for some reason, everything goes to shit, you can still access the files.
same is true for openafs. hmm.
brycec: re: dropbox as a backup - i use duplicity and s3 for my servers
and i avoid spideroak on servers because they haven't released a cli-only client. if they do, i'd have no issue going that route for personal server backups
toddf: fyi, the backup service is no longer in beta, it's the real deal ;)
oh, so I should budget for a bill next month? ;-)
toddf: well ARP's is real cheap, $1/10GB, plus of course bandwidth from your quota (or a dedicated ipv6 vlan to the host). On the other hand, it's just one server with a RAID array, so not exactly competing with S3 or GDrive.
(It's also quite fast from my ARP VPS :p)
dedicated ipv6 vlan to the host huh?
ipv6 is deployed in the same vlan as you are given for your v4 its just that v6 is a different router and thus is not (currently) subject to metering
It's actually a dedicated vlan between your VPS and the backup host, with solely link-local IPv6 addresses
So you're not beating up up_the_irons's poor IPv6 router
(dedicated vlan == another NIC)
(One that I have subsequently mounted via sshfs and use as extra storage :D)
brycec is correct
Since that router is limited to 100mbps... And this way you get GbE to the backup server.
correct again
:)
It's as if I've been down this road before... :P
Oh and the dedicated NIC means I can get jumboframes too.
yep
i think i have MTU 9000 on all those
9000!
oh backup vlan too heh
somehow I missed you were talking about arp backups, but all cool, nice stuff that
hey up_the_irons, i meant to ask this a while ago - what network equipment vendor(s) do you use? (just curious)
mfgrs
m0unds: ebay ;)
rather
oh ah
Cisco
a couple Foundry
one Extreme
i'm writing an email to a vendor and typing in IRC :)
if only all vendors were on irc
gizmoguy: no kidding
up_the_irons: right on
:)
toddf: your v6 changes are now live
yayzorz
up_the_irons: indeed they work
Hopefully up_the_irons remembers the voodoo (which interfaces to configure) for jumboframes :)
brycec: this is not the extra vlan for backup, I've got that already, this is for me renumbering my v6 space (or I should be) so up_the_irons can have a tidy split between the CA location and a someday-other-location in v6 land; I requested some changes to the way global v6 is now presented to me, so I can renumber going forward ;-)
http://helloworldquiz.com
Some of these are much easier than others.
e.g. Shakespeare: super easy.
e.g. SuperCollider: never heard of it.
I lost at 1300.
I only made it to 600 :(
TIL there's java and scala and vala
i've written in java and scala before, but never heard of vala
what is a java?
It's an island.
Oh man they differentiate between Scheme and Racket. That's subtle.
http://www.sweetmarias.com/coffee/indonesia-asia/java?source=side java sunda mayang is awesome
that's all i know about java
java is best
I'm up to 1500 on my second try, but I apparently need to learn the difference between Delphi and Pascal to really be good at this. And Ada.
i could probably recognize code snippets of it
i royally failed
high five up_the_irons
hahaha
lol
i scored 200
i swear i got 4 hard ones in a row
i got 300
got lucky by the end there
http://www.fancy-lang.org/ - I have now (re-)learnt about this language. Neat, and looks a little too similar to the programming language I've been building...
I'm still on my second try. If you failed it the first time, give it another chance!
Lost on Dart. 3300 points.
I'm an armchair programming language enthusiast, though.
i did it again and got 300 through the power of guessing
one of my friends got 3300
mike-burns: damn, nice
brycec: back on the topic of Dropbox, do you use another backup service to backup content that  is also being synced via Dropbox (as you mention Dropbox not being a valid backup strategy)
mnathani: Oddly enough, I leave Dropbox alone (excluded from backups). Reason being: N other machines have a copy of that data, and so does Dropbox (losing their customers' data is as unlikely as me losing my data myself)
i think their own systems would be more likely to cause issues than their storage backend (S3)
brycec: Only thing that worries me about dropbox, is that when someone does a bulk delete, it syncs that across all copies on different boxes  - essentially doing a mass global delete
So you login to dropbox.com and un-delete.
right, but I hear there are limitations to doing that un-delete, besides it might be too impractical if hundreds of files get deleted, I mean un-deleting them all
I guess it depends on the usage scenario, and on each individuals needs
There is a rolling 30day window on free accounts (longer for paying customers). I can't speak to the ease of un-deleting many files.
Speaking of all this... My ownCloud Sync just ate  a bunch of files. Grrr
(Almost all are in Git/SVN, so I'm just out any uncommitted/unpushed changes.
(What's annoying is that it ate *random* files, at *some* point... I didn't notice until I tried an svn command and it bitched that it was b0rked)