[ale] Mounting /home and /data with cifs on a server?
Robert L. Harris
robert.l.harris at gmail.com
Sun Oct 25 23:01:59 EDT 2009
My environment is all linux and I'm using 32768 for my rsize and
wsize values. I was
using UDP, a while back I had read it was better performance but it
could easily have
been wrong, obviously. I have a dedicated gig network just for my
storage network
traffic. It should be using nfs-v4 if not 3, I believe it defaulted to
4 I saw in a log.
Most of my problem is the nfs server is a coraid CLN system which has
a considerable
problem with IO wait but it is atrocious on reading directory listings
for huge directories.
An "ls -la" on a directory with 50k files can take a minute and a half
to return.
Robert
On 10/25/09 6:55 PM, scott wrote:
> NOTE: I do NAS (NFS/CIFS/iSCSI) for a living. I do ALOT of it.
>
> Ok. I have been following the thread remotely since I have been at the
> BarCampATL3 and then I had work to do.
>
> If you are having performance issues with NFS, then one of few things
> are the culprit:
> * using a Window Server (or desktop) as the NFS server (they can do
> NFS but not that well)
> * using UDP for NFS instead of TCP. Please o Please use TCP based
> NFS, it is much much better. Trust me.
> * your NFS server is over whelmed.
> * your network is having some sort of bandwith issue or latency issue.
>
> Looking at the two protocols, NFS (v3) and CIFS, the NFS protocol is a
> LOT less chatty on the wire. CIFS is approximately 20-30x more chatty
> (all depends on size of files, larger the files the less chatty it
> is). Also you are now you are probably asking why I specified v3.
> Well v2 is UDP only and that introduces a whole slew of other issues.
> And v3 has been the default for a long time now. There is a v4 but
> most people are not using it yet. As far as my testing (not live
> production experience) that v3 and v4 performance is the same.
>
> I would suspect that you are not using NFS over tcp. It is easy to
> do. add the "tcp" option in the options section of the fstab. Also
> you will want to set your read/write buffer size. Otherwise they are
> small. Do at least 16k but not more than 32k. Getting too large will
> also give performance problems. My sweet spot that I have found (and
> it works most places) is 32k.
>
> Now to your question, about userids != the uids being used. That is a
> very common problem. What you will have to do, is setup AD (active
> directory) and add the extra fields used by linux into it. Then use
> AD for ALL AUTHENICATIONS for all hosts (linux via pam_* and
> Windows). Then you have to fudge around with a bunch of things. It
> will take weeks worth of work if you work on it 8+ hours a day. And
> it isnt very well documented.
>
> The other thing is let it use the uid of the single user. Sucks but
> it works.
>
> for your information, in my house, I have a single windows box (I play
> games on it and only that), several Macs (yes, I am a mac guy) and
> many many linux machines (some physical some virtual). I run NFS
> between all the Linux hosts. I have samba running CIFS on a single
> linux host sharing out a single folder to my windows box. Macs I used
> ExanDrive to mount drives via ssh. I could do NFS via the cli. So
> 99% of my traffic/usage is NFS or SSH (more of the former over the
> later). Especially since the window machine isnt ever on unless I am
> gaming.
>
> If you ever want to talk about the NFS performance issues, please let
> me know.
>
>
> On Oct 25, 2009, at 3:08 PM, Robert L. Harris wrote:
>
>
>> Has anyone mounted /home or /data (random dir containing user data)
>> from a CIFS
>> mount instead of NFS? I'm looking at some performance issues with a
>> NAS
>> server
>> and it appears CIFS is more repsonsive. The only problem is if I
>> mount
>> /home
>> with:
>>
>> //lincoln/home /home cifs
>> credentials=/root/samba-creds.txt,workgroup=foobar 0 0
>>
>> and use the creds of root to mount the dirs, then any files I create
>> as
>> user "robert" end
>> up created owned by user root on the server, and thus user root on the
>> clients hosing
>> up /home, etc.
>>
>> Thoughts?
>> Robert
>>
>>
>>
>> --
>>
>> :wq!
>> ====================================================================
>> Robert L. Harris | GPG Key ID: E344DA3B
>> @ x-hkp://pgp.mit.edu
>> DISCLAIMER:
>> These are MY OPINIONS With Dreams To Be A King,
>> ALONE. I speak for First One Should Be A Man
>> no-one else. - Manowar
>>
>>
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
--
:wq!
====================================================================
Robert L. Harris | GPG Key ID: E344DA3B
@ x-hkp://pgp.mit.edu
DISCLAIMER:
These are MY OPINIONS With Dreams To Be A King,
ALONE. I speak for First One Should Be A Man
no-one else. - Manowar
More information about the Ale
mailing list