[OmniOS-discuss] Ang: Re: LUN (in)visibility

[Tom Robinson]

On 10/12/15 18:55, Johan Kragsterman wrote:
> You say "infiniband". Do you mean SRP? Where do you have your subnet manager? In the IB switch? If so, did you check the switch SM logs?
>
> I suppose you checked the data links? dladm show-link? What exactly did you check?
>
> How about multipath? How many paths did/do you have to each LUN? I know there were a discussion about too many paths to a LUN earlier on this list. That was fibre channel, though.
>
> I can't really comment on iScsi since I never use it...

Hi Johan,

Yes, SRP. As I said, we had everything working fine before which means we also have a subnet
manager. The SM actually runs on it's own litlte box.

----------      -----------      -----
| storage|======|IB Switch|======|KVM|
----------      -----------      -----
                  |     |
                ----   ------
                |SM|   |ESXi|
                ----   ------

Normally there are only three paths; one iSCSI and two SRP.

I spent a lot of time hunting around on the KVM system looking for clues as at that time I didn't
see any issues else where in the setup.

On OmniOS, in /var/adm/messages I had this:

Oct 26 07:28:43 monza.motec.com.au genunix: [ID 408789 kern.warning] WARNING: hermon0: fault
detected external to device; service unavailable
Oct 26 07:28:43 monza.motec.com.au genunix: [ID 451854 kern.warning] WARNING: hermon0: port 2 down
Oct 26 07:28:47 monza.motec.com.au genunix: [ID 408822 kern.info] NOTICE: hermon0: fault detected
external to device; service still unavailable
Oct 26 07:28:47 monza.motec.com.au genunix: [ID 611667 kern.info] NOTICE: hermon0: port 1 down
Oct 26 07:30:12 monza.motec.com.au genunix: [ID 408789 kern.notice] NOTICE: hermon0: fault cleared
external to device; service available
Oct 26 07:30:12 monza.motec.com.au genunix: [ID 451854 kern.notice] NOTICE: hermon0: port 2 up
Oct 26 07:30:12 monza.motec.com.au genunix: [ID 408822 kern.info] NOTICE: hermon0: no fault external
to device; service available
Oct 26 07:30:12 monza.motec.com.au genunix: [ID 611667 kern.info] NOTICE: hermon0: port 1 up
Oct 26 07:31:31 monza.motec.com.au genunix: [ID 408789 kern.warning] WARNING: hermon0: fault
detected external to device; service unavailable
Oct 26 07:31:31 monza.motec.com.au genunix: [ID 451854 kern.warning] WARNING: hermon0: port 2 down
Oct 26 07:31:38 monza.motec.com.au genunix: [ID 408822 kern.info] NOTICE: hermon0: fault detected
external to device; service still unavailable
Oct 26 07:31:38 monza.motec.com.au genunix: [ID 611667 kern.info] NOTICE: hermon0: port 1 down
Oct 26 07:32:12 monza.motec.com.au genunix: [ID 408789 kern.notice] NOTICE: hermon0: fault cleared
external to device; service available
Oct 26 07:32:12 monza.motec.com.au genunix: [ID 451854 kern.notice] NOTICE: hermon0: port 2 up
Oct 26 07:32:12 monza.motec.com.au genunix: [ID 408822 kern.info] NOTICE: hermon0: no fault external
to device; service available
Oct 26 07:32:12 monza.motec.com.au genunix: [ID 611667 kern.info] NOTICE: hermon0: port 1 up

Isn't the hermon0 driver for the Mellanox cards?

Kind regards,
Tom

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <https://omniosce.org/ml-archive/attachments/20151211/be0e5f8f/attachment-0001.bin>