[OmniOS-discuss] Illumos and Infiniband

[Chris Zembower]

Well, this is all very encouraging! I guess there is interest and actual
usage out there.

Unfortunately, I need FDR-ish speeds (at least 4GB/s), the entire network
is setup for FDR, and I only have ConnectX-3 adapters, so it's Linux for me
until some new developments emerge.

Dan, I can somewhat vouch for the stability of ZFS on Debian and CentOS
using the ZoL kernel implementation. My Debian system (the one using IB) is
at 65 days uptime with a 24/7 heavy load, managing 100TB+ of datasets in a
quasi-production (replicated) environment. First disk replacement went
smoothly last week. Time will tell... The CentOS install has been up for
close to a year. However, in terms of raw disk access speeds, neither of
these systems performs anywhere near as well as OmniOS or OI.

Thanks, all. I look forward to joining the conversation here in general.



On Wed, Jan 29, 2014 at 3:38 PM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:

> Heh - mistype. But that cease and desist we sent to IPoIB ...
>
> :)
>
> On Wed, Jan 29, 2014 at 11:42 AM, Dan Swartzendruber <dswartz at druber.com>
> wrote:
> >> On my systems, the X-2 cards claim 32 gbps. I sued IPoIB, and have
> >> seen transfers NFS transfers over 6 gbps and snapshot send/receive as
> >> high as 1.8 gbps while the system was simultaneously utilized during
> >> cluster computation.
> >>
> >> Ian
> >
> > I assume you meant 'used', not 'sued'?  If not, your lawyer is in the
> > wrong line of work :)
> >
>
>
>
> --
> Ian Kaufman
> Research Systems Administrator
> UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20140129/5707de29/attachment.html>