***: GTAXL has quit IRC (Ping timeout: 264 seconds)
tigerpaw has quit IRC (Quit: purrr)
toddf has quit IRC (Ping timeout: 255 seconds)
DDevine has joined #arpnetworks
hien has joined #arpnetworks
hien has quit IRC (Quit: leaving)
toddf has joined #arpnetworks
ChanServ sets mode: +o toddf
koan_ is now known as koan
DDevine has quit IRC (Ping timeout: 264 seconds)
Olipro has quit IRC (Remote host closed the connection)
Guest32912 has joined #arpnetworks
Guest32912 has quit IRC (Client Quit)
Olipro_ has joined #arpnetworks
Olipro_ is now known as Olipro
tigerpaw has joined #arpnetworks
tigerpaw- has joined #arpnetworks
tigerpaw has quit IRC (Ping timeout: 276 seconds)
nomadlogic has joined #arpnetworks
tigerpaw- has quit IRC (Quit: rawr?)
tigerpaw has joined #arpnetworks
nomadlogic has quit IRC (Ping timeout: 264 seconds)
nomadlogic has joined #arpnetworks
DDevine has joined #arpnetworks
nomadlogic has quit IRC (Ping timeout: 264 seconds)
ballen has joined #arpnetworks
ChanServ sets mode: +o ballen
DDevine has quit IRC (Remote host closed the connection)
DDevine has joined #arpnetworks DDevine: http://www.xtranormal.com/watch/7023615/episode-2-all-the-cool-kids-use-ruby?page=2 mike-burns: heh. RandalSchwartz: heh
it's true DDevine: Thought it would be appropriate here.
:p
oi up_the_irons I'll get the IP addresses sorted out within a few hours. ballen: Anyone know of a distributed filesystem with ZFS-like compression? RandalSchwartz: why not ZFS? ballen: RandalSchwartz: I need a filesystem that can span multiple servers
i.e. Lustre, GPFS, etc RandalSchwartz: zfs can export NFS ballen: well aware RandalSchwartz: and so ballen: NFS is actually somewhat of a slow protocol, plus until 4.1 becomes prevalent theres no way to solve a single name space issue, load balancing between storage servers
and so
I'm looking for something that'll take me to 500TB or so RandalSchwartz: you need to specify why you are disqualifying it ballen: No single name space between multiple machines
no automatic load balancing between servers
a chatty protocol, that wastes bandwidth RandalSchwartz: so you're blindly prejudiced. ok. :) ballen: prove to me that its not then :-) RandalSchwartz: just as long as we know. ballen: I use NFS extensively right now for 200 or so TB of space
my main goal is to solve the single silo effect, of having multiple NFS servers -: DDevine hates that silo effect. ballen: where inevitably one server will be filled more than another and require massive rebalancing
and I don't have the luxury of making the application layer take care of load balancing
Also RandalSchwartz, I'm not blind in anyway on this matter, unless you feel like backing up that statement DDevine: ballen: I don't know much this stuff, but why can't you just use 4.1? ballen: lack of support
mainly on the Solaris side, which is odd since its their protocol
I'm trying to find a enterprise type solution, similar to Isilon, GPFS, that allows me to use compression. As my data gets a 2-3x compression ratio at GZIP-1. Also I have a few SuperMicro 36 drive servers, that are purpose built for ZFS RAIDZ
it of course doesn't have to be commercial, OSS is fine as well
just a couple of requirements that no one seems to have figured out
checking out MooseFS: http://www.moosefs.org/ DDevine: Yeah. ballen: which may run on top of ZFS
I've also tried GlusterFS which seems to be a bit more popular, but fails to run on Sol11 DDevine: What sort of data are you working with? ballen: genomic DDevine: OCFS? ballen: looked at its docs
its a Linux only FS I believe
does anyone have enough experience with Btrfs to say thats its production ready? DDevine: ballen: Surely NFS would be quick enough for that purpose... You don't really need super quick response times right?
ballen: I don't have enough experience with BTRFS yet, but I haven't beeb bitten. ballen: latency isn't a huge issue no DDevine: I think SLES11 had a "technology preview" of BTRFS.
But that would be a fairly old version now. ballen: but then I fall into, if I have 10 NFS servers with 42TB each, thats 10 silos of 42TBs
pNFS is promising DDevine: Yeah that sucks. ballen: to solve that
there was some work in OpenSolaris with it
but can't get anything out of Oracle as when they'll implement it DDevine: Is there a real need for ZFS rather than just EXT? ballen: So let me describe how this hardware is setup
its 36 drives in a single 4U
2 for OS in a mirror
theres 5 SAS2 controllers
each with 2 SAS iPass connectors (i.e. 4 drives)
each controller gets 8 drives, 4 on the last
RAIDZ2 is setup then in 8 drive groups
across the controllers
so that if a controller was lost that filesystem would keep going
so if I moved to EXT3 or a classic FS, I'd have that to contend with
note that hardware design guarantees each drive full SAS2 bandwidth all the way through the PCIe bus
probably overkill
what inexpensive
but* DDevine: Yeah that is a little tricky... I think LVM could manage that though. ballen: possibly DDevine: The performance hit from LVM is fairly minimal. ballen: plus I don't want to give up on-the-fly compression
otherwise I might as well just buy double the hardware DDevine: Yeah I've never done on the fly compression so I don't know what the go with it is.
double the hardware means double the problems and upkeep. ballen: well since my data is primary very large text files, they compress quite well
yep, plus double the space and power DDevine: Yeah excellent compression ballen: basically 2.5x avg
some datasets are a bit better in at 3.2x
so I've been researching, looking how to leverage ZFS as a storage backend
for a GlusterFS like setup
Ideally, Ceph is the option, but its much much to immature DDevine: Yeah on Linux you would have to use BTRFS or Reiser4. ballen: Right DDevine: Reiser4 is mature though, not sure about support these days though. ballen: I haven't used BTRFS + compression enough to know how it effects CPU load
or rather, how it effects throughput due to extra CPU load
Nor how good BTRFS' raid5/6 implementation is. DDevine: Yeah but you can just do the RAID stuff outside of the FS. ballen: thats true DDevine: I have the feeling my ISP doesn't respect DNS TTLs.
Which is odd because overall they have a great setup. ballen: that would be odd DDevine: Eh maybe it was just a temporary thing. Today they seem to have honored the TTLs perfectly. ***: nomadlogic has joined #arpnetworks
nerdd_ has quit IRC (Ping timeout: 240 seconds)
nerdd has joined #arpnetworks