http://www.xtranormal.com/watch/7023615/episode-2-all-the-cool-kids-use-ruby?page=2
heh.
heh
it's true
Thought it would be appropriate here.
:p
oi up_the_irons I'll get the IP addresses sorted out within a few hours.
Anyone know of a distributed filesystem with ZFS-like compression?
why not ZFS?
RandalSchwartz: I need a filesystem that can span multiple servers
i.e. Lustre, GPFS, etc
zfs can export NFS
well aware
and so
NFS is actually somewhat of a slow protocol, plus until 4.1 becomes prevalent theres no way to solve a single name space issue, load balancing between storage servers
and so
I'm looking for something that'll take me to 500TB or so
you need to specify why you are disqualifying it
No single name space between multiple machines
no automatic load balancing between servers
a chatty protocol, that wastes bandwidth
so you're blindly prejudiced.  ok. :)
prove to me that its not then :-)
just as long as we know.
I use NFS extensively right now for 200 or so TB of space
my main goal is to solve the single silo effect, of having multiple NFS servers
where inevitably one server will be filled more than another and require massive rebalancing
and I don't have the luxury of making the application layer take care of load balancing
Also RandalSchwartz, I'm not blind in anyway on this matter, unless you feel like backing up that statement
ballen: I don't know much this stuff, but why can't you just use 4.1?
lack of support
mainly on the Solaris side, which is odd since its their protocol
I'm trying to find a enterprise type solution, similar to Isilon, GPFS, that allows me to use compression. As my data gets a 2-3x compression ratio at GZIP-1. Also I have a few SuperMicro 36 drive servers, that are purpose built for ZFS RAIDZ
it of course doesn't have to be commercial, OSS is fine as well
just a couple of requirements that no one seems to have figured out
checking out MooseFS: http://www.moosefs.org/
Yeah.
which may run on top of ZFS
I've also tried GlusterFS which seems to be a bit more popular, but fails to run on Sol11
What sort of data are you working with?
genomic
OCFS?
looked at its docs
its a Linux only FS I believe
does anyone have enough experience with Btrfs to say thats its production ready?
ballen: Surely NFS would be quick enough for that purpose... You don't really need super quick response times right?
ballen: I don't have enough experience with BTRFS yet, but I haven't beeb bitten.
latency isn't a huge issue no
I think SLES11 had a "technology preview" of BTRFS.
But that would be a fairly old version now.
but then I fall into, if I have 10 NFS servers with 42TB each, thats 10 silos of 42TBs
pNFS is promising
Yeah that sucks.
to solve that
there was some work in OpenSolaris with it
but can't get anything out of Oracle as when they'll implement it
Is there a real need for ZFS rather than just EXT?
So let me describe how this hardware is setup
its 36 drives in a single 4U
2 for OS in a mirror
theres 5 SAS2 controllers
each with 2 SAS iPass connectors (i.e. 4 drives)
each controller gets 8 drives, 4 on the last
RAIDZ2 is setup then in 8 drive groups
across the controllers
so that if a controller was lost that filesystem would keep going
so if I moved to EXT3 or a classic FS, I'd have that to contend with
note that hardware design guarantees each drive full SAS2 bandwidth all the way through the PCIe bus
probably overkill
what inexpensive
but*
Yeah that is a little tricky... I think LVM could manage that though.
possibly
The performance hit from LVM is fairly minimal.
plus I don't want to give up on-the-fly compression
otherwise I might as well just buy double the hardware
Yeah I've never done on the fly compression so I don't know what the go with it is.
double the hardware means double the problems and upkeep.
well since my data is primary very large text files, they compress quite well
yep, plus double the space and power
Yeah excellent compression
basically 2.5x avg
some datasets are a bit better in at 3.2x
so I've been researching, looking how to leverage ZFS as a storage backend
for a GlusterFS like setup
Ideally, Ceph is the option, but its much much to immature
Yeah on Linux you would have to use BTRFS or Reiser4.
Right
Reiser4 is mature though, not sure about support these days though.
I haven't used BTRFS + compression enough to know how it effects CPU load
or rather, how it effects throughput due to extra CPU load
Nor how good BTRFS' raid5/6 implementation is.
Yeah but you can just do the RAID stuff outside of the FS.
thats true
I have the feeling my ISP doesn't respect DNS TTLs.
Which is odd because overall they have a great setup.
that would be odd
Eh maybe it was just a temporary thing. Today they seem to have honored the TTLs perfectly.