[OmniOS-discuss] ixgbe: breaking aggr on 10GbE X540-T2

[Stephan Budach]

Hi Dale,

Am 17.05.16 um 20:55 schrieb Dale Ghent:
> On May 17, 2016, at 8:30 AM, Stephan Budach <stephan.budach at JVM.DE> wrote:
>
>> I have checked all of my ixgbe interfaces and they all report that now flow controll is in place, as you can see:
>>
>> root at zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe0
>> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
>> ixgbe0       flowctrl        rw   no no             no,tx,rx,bi
>> root at zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe1
>> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
>> ixgbe1       flowctrl        rw   no no             no,tx,rx,bi
>> root at zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe2
>> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
>> ixgbe2       flowctrl        rw   no no             no,tx,rx,bi
>> root at zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe3
>> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
>> ixgbe3       flowctrl        rw   no no             no,tx,rx,bi
>>
>> I then checked the ports on the Nexus switches and found out, that they do have outbound-flowcontrol enabled, but that is the case on any of those Nexus ports, including those, where this issue doesn't exist.
> Optimally you would have flow control turned off on both sides, as the switch still expects the ixgbe NIC to respond appropriately. To be honest, the only time to use ethernet flow control is if you are operating the interfaces for higher-level protocols which do not provide any sort of direct flow control themselves, such as FCoE. If the vast majority of traffic is TCP, leave it to the TCP stack to manage any local congestion on the link.
>
> /dale
I just wanted to wrap this up… I recently swapped that old Sun server 
with a new Supermicro X10-type, which has 4 10 GbE NICs on board, 
installed OmniOS r018 and my RSF-1 cluster software on it. Configured my 
two LACP aggregations and there hasn't been any issue since.
So, either it's something on the old server - it's a Sun Fire X4170M2 - 
or something on the Intel cards.

Cheers,
Stephan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20160603/2d785b4c/attachment.html>