[OmniOS-discuss] Slow NFS speeds at rsize > 128k

[Richard Elling]

> On Jan 7, 2015, at 2:28 AM, Stephan Budach <stephan.budach at JVM.DE> wrote:
> 
> Hello everyone,
> 
> I am sharing my zfs via NFS to a couple of OVM nodes. I noticed really bad NFS read performance, when rsize goes beyond 128k, whereas the performance is just fine at 32k. The issue is, that the ovs-agent, which is performing the actual mount, doesn't accept or pass any NFS mount options to the NFS server.

The other issue is that illumos/Solaris on x86 tuning of server-side size settings does
not work because the compiler optimizes away the tunables. There is a trivial fix, but it
requires a rebuild.

> To give some numbers, a rsize of 1mb results in a read throughput of approx. 2Mb/s, whereas a rsize of 32k gives me 110Mb/s. Mounting a NFS export from a OEL 6u4 box has no issues with this, as the read speeds from this export are 108+MB/s regardles of the rsize of the NFS mount.

Brendan wrote about a similar issue in the Dtrace book as a case study. See chapter 5
case study on ZFS 8KB mirror reads.

> 
> The OmniOS box is currently connected to a 10GbE port at our core 6509, but the NFS client is connected through a 1GbE port only. MTU is at 1500 and can currently not be upped.
> Anyone having a tip, why a rsize of 64k+ will result in such a performance drop?

It is entirely due to optimizations for small I/O going way back to the 1980s.
 -- richard

> 
> Thanks,
> budy
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20150107/074799d8/attachment.html>