[OmniOS-discuss] Low latency networking

[Chris Nagele]

Update:

I was able to get iperf results of 9.89Gb/s each way, between Debian
and OmniOS.

I accomplished this by setting intr_throttling=1; in the
/kernel.drv/ixgbe.conf file on OmniOS.

Reference: http://support.ap.dell.com/support/edocs/network/intelpro/r195641/en/ixgbe.htm

For some reason though, ping times from OmniOS to Debian are still
much slower, above 0.2ms. I know it is nit picking, but we invested in
10g for lower latency, so I am trying to get the most out of it. From
Debian to OmniOS I get about 0.08ms still.

-Chris


On Sun, Aug 5, 2012 at 10:41 AM, Chris Nagele <nagele at wildbit.com> wrote:
> Hi Richard,
>
>> Check all cable ends for errors. In an extreme case, we sent an Engineer to
>> Germany
>> to solve a "performance problem" that turned out to be a bad SFP+
>> transceiver. The bill
>> was, shall we say, higher than the average service call :-)
>
> At first I thought that was the issue, but I tested in Debian and got
> sub 1ms ping times and much faster NFS benchmarks. I adjusted the
> ixgbe.conf file and set the following:
>
> default_mtu=9000;
> intr_throttling=0;
>
> Reference: http://www.oracle.com/technetwork/database/focus-areas/bi-datawarehousing/sas/o11-075-sas-grid-computing-485336.html
>
> This got the ping times down from 0.225ms to 0.17ms. I'm still trying
> to get closer to the 0.08ms that I get on Debian.
>
> What are your thoughts on Infiniband?
>
> Chris