[OmniOS-discuss] Low latency networking

[Richard Elling]

Hi Chris,
Check all cable ends for errors. In an extreme case, we sent an Engineer to Germany
to solve a "performance problem" that turned out to be a bad SFP+ transceiver. The bill
was, shall we say, higher than the average service call :-)
 -- richard

On Aug 4, 2012, at 4:41 PM, Chris Nagele wrote:

> I've been testing an environment before we send it over to the data
> center. The setup is an OmniOS install (Supermicro X9, 24 drives,
> DDRdrive for ZIL) used as an NFS file server and multiple Debian
> machines connected via directly attached 10GB SFP+ twin-axial cables.
> We went with the 10Gb in order to get lower latency to compensate for
> the high TCP overhead of NFS.
> 
> For some reason, I can't get ping times below 0.2ms when using OmniOS.
> If I install Debian on the same machines I get consistent 0.08ms
> between the machines. In addition, iperf shows a maximum of 6.6Gb/s on
> OmniOS, and about 9.89Gb/s on Debian.
> 
> We are using the x520-DA2, which is well supported in Illumos. I also
> tried on Nexenta and OpenIndiana with the same result. I was thinking
> of trying different cards (Myricom maybe), different cabling (CX4?),
> or even Infiniband since it is known for low latency.
> 
> Any recommendations? I am wondering if I just need to adjust some
> tunables or maybe update a driver.
> 
> Thanks,
> Chris
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

--
ZFS Performance and Training
Richard.Elling at RichardElling.com
+1-760-896-4422



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20120804/23423944/attachment.html>