[ale] Chunk size on RAID arrays

[Danny Cox]

Greg,

On Sun, 2003-07-20 at 14:06, Greg wrote:
> 	So - any tips/suggestions/war stories dealing with the chunk size parameter
> for RAID 5 arrays ????

	As I mentioned in the talk: it depends on your application.  If you
have the time, try several different chunk sizes with the types of
applications you mentioned.  Use bonnie, bonnie++, dd (copying files
roughly the size you'll be dealing with), etc., and see what effect
chunksize has.

	In my case, I found that bonnie showed faster disk I/O up until a 128
chunksize.  Any higher didn't seem to make much difference.  Of course
bonnie tries to measure just disk I/O, and to keep the buffer cache out
of it.  That's why it want's to use a file size greater than your memory
size.  Hardware RAID can also introduce more cacheing, so you'll just
have to try it.

> 	ok, I just saw Danny Cox's email and I might have to look at XFS again ...
> runs at near I/O speed huh ? hmmm  That's definitely a pretty darned big
> plus ! [Thanks Danny for the info and war story.  And thanks for the RAID
> ALE talk also (too bad I was too ignorant to ask these questions then].

	You're welcome.  I hope it was informative and humorous as well.

	The things you mention above are just the type of files SGI was
thinking of when they designed XFS.  MP3s (also consider Ogg Vorbis. 
It's supposed to be somewhat better quality and better compression than
MP3s at this stage.  All my songs are in Ogg.), and dvd movies are
generally considered large files at multiple megabytes (multiple
gigabytes for the movies of course) and XFS excels at handling them.

	Now, I'm not saying that ext3 is worse.  I've not actually measured
them.  ext3 may work just fine for your application.  Then again, ext3
is supposedly the replacement for ext2, and as such is a generic FS. 
XFS was designed from the ground up to handle multiple large files.

	One caveat about XFS.  I know of one person who disliked this "feature"
so much, that he refuses to use XFS.  If, when creating a file, the
system experiences a reboot (power loss, tripping over the cord, kernel
bug), the content of that file is zero filled (ASCII NUL). This is for
security reasons.  I don't remember if it affects the whole file if
you're just appending at the time of the reboot or not.  Of course, a
UPS all but eliminates this issue.

	If anyone's interested, I can describe how the Alias/Wavefront software
worked on an SGI workstation circa 1996 and how it was rendered to
tape.  It was pretty painful.  I'm sure it's much better now.

-- 
kernel, n.: A part of an operating system that preserves the
medieval traditions of sorcery and black art.

Danny

_______________________________________________
Ale mailing list
Ale at ale.org
http://www.ale.org/mailman/listinfo/ale