[OmniOS-discuss] Slow NFS speeds at rsize > 128k

[Stephan Budach]

Am 07.01.15 um 18:00 schrieb Richard Elling:
>
>> On Jan 7, 2015, at 2:28 AM, Stephan Budach <stephan.budach at JVM.DE 
>> <mailto:stephan.budach at JVM.DE>> wrote:
>>
>> Hello everyone,
>>
>> I am sharing my zfs via NFS to a couple of OVM nodes. I noticed 
>> really bad NFS read performance, when rsize goes beyond 128k, whereas 
>> the performance is just fine at 32k. The issue is, that the 
>> ovs-agent, which is performing the actual mount, doesn't accept or 
>> pass any NFS mount options to the NFS server.
>
> The other issue is that illumos/Solaris on x86 tuning of server-side 
> size settings does
> not work because the compiler optimizes away the tunables. There is a 
> trivial fix, but it
> requires a rebuild.
>
>> To give some numbers, a rsize of 1mb results in a read throughput of 
>> approx. 2Mb/s, whereas a rsize of 32k gives me 110Mb/s. Mounting a 
>> NFS export from a OEL 6u4 box has no issues with this, as the read 
>> speeds from this export are 108+MB/s regardles of the rsize of the 
>> NFS mount.
>
> Brendan wrote about a similar issue in the Dtrace book as a case 
> study. See chapter 5
> case study on ZFS 8KB mirror reads.
>
>>
>> The OmniOS box is currently connected to a 10GbE port at our core 
>> 6509, but the NFS client is connected through a 1GbE port only. MTU 
>> is at 1500 and can currently not be upped.
>> Anyone having a tip, why a rsize of 64k+ will result in such a 
>> performance drop?
>
> It is entirely due to optimizations for small I/O going way back to 
> the 1980s.
>  -- richard
But, doesn't that mean, that Oracle Solaris will have the same issue or 
has Oracle addressed that in recent Solaris versions? Not, that I am 
intending to switch over, but that would be something I'd like to give 
my SR engineer to chew on…

In any way, the first bummer is, that Oracle chose to not have it's 
ovs-agent be capable of accepting and passing the NFS mount options…

Cheers,
budy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20150107/f3d5cc7f/attachment.html>