[OmniOS-discuss] The ixgbe driver, Lindsay Lohan, and the Greek economy

[Chris Siebenmann]

> Chris, thanks for your specific details. I'd appreciate it if you
> could tell me which copper NIC you tried, as well as to pass on the
> iSCSI tuning parameters.

 Our copper NIC experience is with onboard X540-AT2 ports on SuperMicro
hardware (which have the guaranteed 10-20 msec lock hold) and dual-port
82599EB TN cards (which have some sort of driver/hardware failure under
load that eventually leads to 2-second lock holds). I can't recommend
either with the current driver; we had to revert to 1G networking in
order to get stable servers.

 The iSCSI parameter modifications we do, across both initiators and
targets, are:

	initialr2t		no
	firstburstlength	128k
	maxrecvdataseglen	128k		[only on Linux backends]
	maxxmitdataseglen	128k		[only on Linux backends]

The OmniOS initiator doesn't need tuning for more than the first two
parameters; on the Linux backends we tune up all four. My extended
thoughts on these tuning parameters and why we touch them can be found
here:

   http://utcc.utoronto.ca/~cks/space/blog/tech/UnderstandingiSCSIProtocol
   http://utcc.utoronto.ca/~cks/space/blog/tech/LikelyISCSITuning

The short version is that these parameters probably only make a small
difference but their overall goal is to do 128KB ZFS reads and writes
in single iSCSI operations (although they will be fragmented at the TCP
layer) and to do iSCSI writes without a back-and-forth delay between
initiator and target (that's 'initialr2t no').

 I think basically everyone should use InitialR2T set to no and in
fact that it should be the software default. These days only unusually
limited iSCSI targets should need it to be otherwise and they can change
their setting for it (initiator and target must both agree to it being
'yes', so either can veto it).

	- cks