[OmniOS-discuss] Samba Performance

[Michael Talbott]

Hi all. I've been transitioning a file server to OmniOS for many reasons (abandoning zfsonlinux). But I seem to have one last issue I'd like to resolve. It seems to be running into a performance issue with samba. I'm not using the built in zfs smb sharing because I need more flexibility in our environment than it offers such as sharing subfolders, shadow copies, etc.

I'm running the latest LTS version of Omni on a well equipped server with 2x Xeon E5-2630 v2 @ 2.60GHz, 128G ECC ram with dual intel 10g NICs, and two dual LSI 6G SAS cards. The zpool has 8 x raidz2s that have 10 4TB drives in each totaling 290TB. Disk performance is not an issue.

I've compiled samba 4.2.1 and netatalk from source, got winbind working nicely with our AD environment. Samba is even happily kerberized. Everything authenticates and functions correctly. But, while netatalk gives me line speed performance (120-150MB/s on a gigabit workstation), samba won't budge above 40-60MB/s (same speeds using MacOS and Windows 7,8,2012 clients).

Using the same hardware on CentOS 7 with zfsonlinux, samba gives me just about the same throughput as netatalk. In linux, I could tune it with socket options giving it a bigger buffer and it made a big difference. But using the same options on Omni doesn't seem to have any significant affect (actually seems to slow it down a bit).

In smb.conf on both OSs:
	socket options = TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=2097152 SO_RCVBUF=2097152

In CentOS7 in sysctl.conf:
net.ipv4.tcp_rmem = 10000000 10000000 10000000
net.ipv4.tcp_wmem = 10000000 10000000 10000000
net.ipv4.tcp_mem = 10000000 10000000 10000000
net.ipv4.tcp_sack = 0
net.core.rmem_max = 524287
net.core.wmem_max = 524287
net.core.rmem_default = 524287
net.core.wmem_default = 524287
net.core.optmem_max = 524287
net.core.netdev_max_backlog = 300000

And then in Omni, I've set these ip properties
root at store3:# ipadm show-prop
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   max_buf               rw   16777216     16777216     1048576      8192-1073741824
tcp   recv_buf              rw   16777216     16777216     128000       2048-16777216
tcp   send_buf              rw   16777216     16777216     49152        4096-16777216

I just can't get samba on Omni to go any faster than 60MB/s. I've tried adjusting those buffers, removing the socket options in smb.conf altogether, but, to no avail.

Anyone else out there running samba on Omni and getting faster throughput? Anyone have any ideas of how I could get more throughput with samba?

Thanks,

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20150519/a427c16e/attachment-0001.html>