[OmniOS-discuss] Ang: Re: Ang: Re: infiniband

[Greg Mason]

Enabling IB support for our ZFS-based filers is important enough to us that we have made the decision to migrate away from OmniOS/Illumos over to FreeBSD. FreeBSD 9 has a partial OFED stack, FreeBSD 10 is further along, and development continues for IB support in FreeBSD. I’ve successfully tested a Mellanox ConnectX-3 card on our FDR fabric on a FreeBSD 10.0-RELEASE system. This was with the subnet manager running on a Mellanox switch (which we regularly upgrade).

If you’d like more details, please contact me off-list, I’m happy to share :)

-Greg

Greg Mason
Michigan State University
Institute for Cyber-Enabled Research
High Performance Computing Center

web: www.hpcc.msu.edu
email: gmason at msu.edu

> On Oct 24, 2014, at 11:35 AM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
> 
> I wonder if Nexenta has added more/better support for IB? I believe
> that Illumian 1.0 is finally out (after having run the Alpha for
> years), and I know that they did have some interest in supporting IB
> originally. If they have, then I would hope some of that support would
> make it back upstream.
> 
> Mayhaps I have an email to send out ...
> 
> Ian
> 
> On Fri, Oct 24, 2014 at 7:58 AM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
>> I use a QLogic 12300 with SM built in. I haven't had any issues,
>> saving my cluster frontend's cycles for other things.
>> 
>> Ian
>> 
>> On Fri, Oct 24, 2014 at 7:55 AM, Michael Rasmussen <mir at miras.org> wrote:
>>> On Fri, 24 Oct 2014 13:17:39 +0200
>>> Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
>>> 
>>>> 
>>>> Well, I can only find one person in that thread that BELIEVES that running SM on a switch is preferable to run OpenSM software. He hardly comes with any documentation of this, and I can't see any other that agrees...
>>>> 
>>>> But of coarse, if you provide too little resources for OpenSM it will cause problems, or if you have any problems with the S/W or the configuration...
>>>> 
>>> I think the issue here is that OpenSM is a single threaded application
>>> so when the CPU which is assigned to it maxes out performance will
>>> drop. So I guess it comes down to a trade-off between the
>>> performance of the host CPU and the performance of the switch hardware.
>>> 
>>> --
>>> Hilsen/Regards
>>> Michael Rasmussen
>>> 
>>> Get my public GnuPG keys:
>>> michael <at> rasmussen <dot> cc
>>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
>>> mir <at> datanom <dot> net
>>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
>>> mir <at> miras <dot> org
>>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
>>> --------------------------------------------------------------
>>> /usr/games/fortune -es says:
>>> The default Magic Word, "Abracadabra", actually is a corruption of the
>>> Hebrew phrase "ha-Bracha dab'ra" which means "pronounce the blessing".
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> 
>> 
>> 
>> 
>> --
>> Ian Kaufman
>> Research Systems Administrator
>> UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
> 
> 
> 
> -- 
> Ian Kaufman
> Research Systems Administrator
> UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20141024/c3986dbb/attachment.html>