[OmniOS-discuss] Infiniband In Omni

[Matthew Lagoe]

Also for infiniband you need a subnet manager, if your switch doesn’t have a built in one you have to run like a linux box on the same network just for traffic to work, as there wasn’t any I was able to find for openindiana back in the day, that’s why we went with 10gbe.

 

From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Narayan Desai
Sent: Monday, September 29, 2014 11:03 AM
To: Matthew Mabis
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Infiniband In Omni

 

I hate to sound negative, but you're in for a bad time. The current drivers were written by sun/oracle, and only just happen to still work. Parts (like IPoIB) work, but others (EoIB) will randomly panic. Hardware support is limited to connectx 1-2. And if anything goes wrong, there isn't anyone that can really help you. 

 

That all said, we had decent success with iscsi/iSER over connectX 1 cards. Fujita Syoyo forward ported the last version of the OFED stack that Sun/Oracle modified up to more recent illumos. Those bits are here:

https://github.com/syoyo/solaris-infiniband-tools

 -nld

 

On Mon, Sep 29, 2014 at 1:20 PM, Matthew Mabis <mmabis at vmware.com> wrote:

Hey All

 

I was wondering if someone who has been using IB in their lab has any good advice for someone just starting to integrate IB into their ZFS environment.  I have Connect-X (HCA) Card put into the host ( i do have a CX3 but i have heard its not natively supported and there were issues with the drivers) what i am looking for is this

 

    Any Best Practices for using IPoIB with OmniOS?

    Should i use the Stock drivers build into solaris (are there different ones that work better more stable?)

    Are there Management tools i can install to run further tests?

    Any other Guidelines you might recommend.

 

What i am going to attempt is my 3 ESXI hosts have CX-3 (VPI Pro Cards) connected to a SX6012 Switch i have a CX4 to QSFP cable coming to hook it into the switch and wanted to see if there would be any problems (yes the CX card is 20Gb/s) but this is the best i can offer it based on the drivers.

 

Anything someone might see as an issue (Gonna try iSCSI/NFS with This method direct lines to the hosts) uses CIFS on the 1Gb Ether. 

Thanks

Matt Mabis


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20140929/a4d2ed02/attachment.html>