[OmniOS-discuss] My NFS low performance test.

[Ian Kaufman]

Hi Muhammad,

What is the output of "sharectl get nfs"? Have you done any NFS tuning?

Even though the system says you have 7GB of RAM free, you are running
a very slim system for ZFS. Do you have compression enabled?
Deduplication? How big is your filesystem? Here are some RAM rules of
thumb for ZFS (anfd other advice):

For decent ZFS performance, you need at least 1GB of RAM for every TB
of data, plus at least 1 extra GB for OS

If deduplication is turned on, you need about 4GB of RAM per TB of
data in addition to normal system and ZFS RAM if using 64K blocks on
average, if using smaller 8K blocks on average, then you need about
32GB of RAM per TB of data.

RAIDZ2 vdevs should have at least 5 disks and no more than 10. Some
believe 6 or 8 disks per vdev for RAIDZ2 is optimal.

Improve read performance with L2ARC and SSDs - improve write
performance with either RAMDISK SSDs or SLC Flash Memory SSDs, and
mirror the devices.

Ian




On Wed, Sep 11, 2013 at 11:42 PM, Muhammad Yousuf Khan <sirtcp at gmail.com> wrote:
> Just notice and would like to share my new finding.
>
> as i mentioned earlier i am using RAIDz2 and hosted a VM on NFS. so for
> testing i "SCP"  the whole HD image of VM to some other Linux system so that
> i can learn if the issue is with Drive speed or the issue is with NFS
> protocol, (of course not a bug with NFS but i have done something wrong in
> configuring)
>
> here is the SCP result. the  virtual harddisk image size is 32 GB.
> i am running this command from OmniOS SSH.
>
>
> vm-100001-disk-1.qco  50% |**************               | 16461 MB    09:42
> ETA
>
>
> you can see it copy 16 GB and i confirm that it took 3 to 4 minutes to copy
> that huge data.
>
> it seems fine to me. same RAIDz2 but  protocol change (SCP on SSH) so i
> think the issue is with NFS. not with the Hardware speed/specs limitation.
>
> any idea please.
>
> Thanks,
> Myk
>
>
>
> On Thu, Sep 12, 2013 at 2:40 AM, Muhammad Yousuf Khan <sirtcp at gmail.com>
> wrote:
>>
>> first of  sorry for starting a new thread for same issue explained earlier
>> because. i
>> mistakenly deleted the email. so my apologizes.
>>
>> my omniOS server hardware is as under.
>> dual 3.2GHz Processors, 12 GB RAM. dell 490 workstation
>> 4 sata drives set with RAIDz2
>> 1 40GB very old IDE drive for OmniOS root system
>>
>> i have 3 servers for testing.
>> 1. KVM virtualization server (Proxmox)
>> 2. OmniOS (NFS)
>> 3. Desktop machine
>>
>> all three are connected via 1GB LAN broadcast domain.
>>
>> i have hosted a test VM (windows 2003 server) on NFS and when i am trying
>> to copy the data (2gb File) from VM via Microsoft sharing to a desktop
>> machine. i doing something very abnormal which i can not understand why
>>
>> just FYI : all the virtual machines hosted on KVM (proxmox server) local
>> harddrives are on mdadm linux raid. and working great. and i see no issue in
>> copy and pasting data from or to the local VM machines.
>>
>> In attached graphics you will see 2 test of sending 2GB file.
>> in "snap a.1"  for few seconds file transfer stuck at 2 percent and then
>> due to unknown reasons bumps to 15%. and it continues for few seconds and
>> when file reaches the end it again goes down to 3 Percent  as you can see in
>> "snap a.2"
>>
>> in "snap b.1" you can see it starts at 3% max and for 2 minutes it
>> continues and speed only  changes btw 1% to 3%. see in "snap b.2"
>>
>> some says disable sync for testing and see if performance increases, sorry
>> it didnt help me. same thing happens, whether i disable or enable the sync
>> it makes no difference for me
>>
>> and Some says to upgrade the RAM which i am already looking but just a
>> part of my confusion when i see vmstat it shows me i have 7GB free RAM
>>
>> is there anyone can please explain me why and what is happening behind all
>> this, the workstation hosting OmniOS is a good machine. why it is doing some
>> thing very unexpected.
>>
>> Any advice, suggestion, tip etc. will be highly appreciated.
>>
>> Thanks
>> Myk
>>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu