[OmniOS-discuss] Ang: Re: Ang: Re: infiniband

[Chris Zembower]

ConnectX-2 cards work very well with OmniOS, at least over SRP, and those
HCA's can be found for next to nothing. Comstar rules. I feel there's some
new momentum behind Infiniband, and hoping that a new driver for the latest
hardware will end up in Illumos sometime in the near future (I'm willing to
pay a fair bounty). That said, QDR is still extremely fast and can easily
co-mingle with FDR fabrics.

The subnet manager can be any linux or windows box sitting on the fabric,
no need for a managed switch in my experience, although my fabrics are
modestly sized and quite stable. To reiterate what others have said, the
subnet manager has nothing to do with target/initiator data transfer.
OpenSM just scans the fabric at regular intervals for topology changes and
handles session setup and teardown. I think of OpenSM as a sort of DHCP
server for Infiniband, and in that analogy, it would be strange to believe
that even the most resource-deprived DHCP server would affect data transfer
on a network. A managed switch is akin to a DHCP server embedded in a
router. Convenient, but not necessary.

Greg, I'll probably contact you off-list to learn about your FreeBSD
implementation. My ConnectX-3 FDR fabric is currently ZoL-based because I
couldn't find concrete documentation on the ULP's available in FreeBSD
OFED, whereas Linux is quite straightforward, albeit arguably not the most
robust ZFS implementation. I'm doing Linux zvol's exported over SRP via
SCST on CentOS to a mix of Linux and Windows initiators and it's been
rock-solid for about a year now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20141024/3baf0041/attachment.html>