[ale] Giant storage system suggestions

[JD]

On 07/11/2012 05:03 PM, Alex Carver wrote:
> I'm trying to design a storage system for some of my data in a way that 
> will be useful to duplicate the design for a project at work.

We have 2 requirements at this point.
a) Cheap
b) 10TB usable with room to grow

A few more questions need to be asked to get started understanding the
requirements more completely.

* What services do you require?
** CIFS
** NFS
** iSCSI
** AoE
** rsync
** others?

* How will you backup all this data automatically?
** tape
** duplicity
** lvm
** zsend

* Which file system(s) do you want/need?

* Storage Performance?
** 10/100 connection
** GigE connection
** 10GigE connection
** multiple bonded 10GigE connections?

* Transport connections?
** ethernet over copper
** fibre channel over optical
** infiniband

* Are there any unusual distance requirements for access to the storage?

* What is the largest partition size required?  This feeds into backups and
future data migration options.  You WILL need to migrate the data in the future.

* How critical is the data?
** Bad day or week critical
** Lose my job critical
** Someone dies critical
** 300+ people die critical

* Budget?
** SW - is commercial SW an option at all?
** HW - RAID cards fail occasionally, so you'll want an identical spare available.
** Support - things that might take me a week to figure out are solved in an
hour by a professional in the business.

* RAID Options
** RAID6
** RAID10
** RAIDz
** RAIDz2

* Any data replication requirements?

* Any HA data requirements?

At this level, I'd expect hardware to be picky, so be certain that anything you
piece together is listed as supported between the RAID, expansion, external
array, protocols, physical connections and motherboard.

My initial brainstorm said "he needs ZFS", but only you can decide if that is
possible.  Last time I checked, ZFS under Linux isn't a first-class supported
solution.

Sorry, no real answer from me, just more questions.  Good luck and please post
more about your attempts and final solution.  Actually, the final solution would
be a fantastic ALE presentation.