[OmniOS-discuss] multipathing with compellent
Theo Schlossnagle
jesus at omniti.com
Thu Oct 18 15:58:29 EDT 2012
This is a more general Illumos problem, you will open your
troubleshooting to a larger audience by interfacing with the
illumos-discuss list. I encourage you to post there! I'm subscribed
there as well, but I haven't done multipathing against compellent and
it appears no one else on the omnios-discuss list has either.
Anything learned in the illumos-discuss group can be applied directly
to OmniOS.
On Thu, Oct 18, 2012 at 3:51 PM, Majeed <mabuqu at ilstu.edu> wrote:
> Hi Again, just wanted to update the list with my latest progress and
> thoughts on this topic. I think the issue lies in libmpscsi_vhci.so's lack
> of support for multipathing with compellent.
>
> When I do "mpathadm show mpath-support libmpscsi_vhci.so" on omniOS I see
> many venders and models listed but not compellent. We are going to spin up a
> solaris 11 machine to see what it looks like. I wonder if this was forked
> just before SUN/oracle added support for compellent (i could be way off base
> here).
>
> When I do the same command on solaris 10 u10, at the very bottom of the list
> i see.
> Vendor: COMPELNT
> Product: Compellent Vol
> Revision:
> Supported Load Balance Types:
> round-robin
>
> I have verified that multipathing is working properly on other arrays. I
> zoned up omniOS with all controllers on a 6140 (CSM200) that sit in multiple
> fabrics, mapped a lun, and it looks wonderful. mpatham reports as expected,
> and all paths are condensed to a single block device in format's output.
>
> % mpathadm list LU
> /dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
> Total Path Count: 4
> Operational Path Count: 4
>
>
>
> It seems there are some options in /kernel/drv/scsi_vhci.conf to force
> scsi_vhci to accept 3rd party devices (hopefully compellent) for failover,
> but this is kinda getting over my head. It seems like if I had a proper
> ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<x>" file for compellent and I
> knew the proper VenderID and ProductID for the compellent devices, i might
> be able to force it. Im not sure if can use the information from the format
> inquiry to populate these values.
>
> format> inquiry
> Vendor: COMPELNT
> Product: Compellent Vol
> Revision: 0505
>
>
> Any thoughts on this would be excellent.
>
> Thanks,
> Majeed Qulbain
>
>
>
>
>
>
>
>
>
> On 10/18/2012 09:30 AM, Majeed wrote:
>>
>> I just noticed that we do have access to the mpathadm command by
>> installing the "pkg:/storage/mpathadm at 0.5.11-0.151002" package. After
>> installing I don't see any paths with "mpathadm list LU" which is what im
>> used to on solaris 10 (i know I should be comparing to solaris 11, just
>> havn't played with it much). I wonder if there are any other bits that I
>> need to install or other configuration tweaks to get this going.
>>
>> Thanks in advance for any pointers,
>> Majeed
>>
>> Some other related commands that might assist troubleshooting:
>>
>> % uname -a
>> SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc
>>
>> % mpathadm list initiator-port
>> Initiator Port: w5d4ae520937bbb00
>> Initiator Port: iqn.1986-03.com.sun:01:b06a64f606ff.507f35b3,4000002a00ff
>> Initiator Port: w5d4ae520937bbb00
>> Initiator Port: 2101001b32b40767
>> Initiator Port: 2100001b32940767
>>
>> % luxadm -e port
>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0:devctl
>> CONNECTED
>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0:devctl
>> CONNECTED
>>
>> % luxadm -v probe
>>
>> Found Fibre Channel device(s):
>> Node WWN:5000d310004a0602 Device Type:Disk device
>> Logical Path:/dev/rdsk/c16t5000D310004A064Bd1s2
>> Node WWN:5000d310004a0602 Device Type:Disk device
>> Logical Path:/dev/rdsk/c17t5000D310004A064Cd1s2
>>
>> % luxadm disp /dev/rdsk/c16t5000D310004A064Bd1s2
>> DEVICE PROPERTIES for disk: /dev/rdsk/c16t5000D310004A064Bd1s2
>> Vendor: COMPELNT
>> Product ID: Compellent Vol
>> Revision: 0505
>> Serial Num: 00004a06-00000014
>> Unformatted capacity: 512000.000 MBytes
>> Read Cache: Enabled
>> Minimum prefetch: 0x100
>> Maximum prefetch: 0xf0
>> Device Type: Disk device
>> Path(s):
>>
>> /dev/rdsk/c16t5000D310004A064Bd1s2
>>
>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1:c,raw
>> Controller /dev/cfg/c16
>> Device Address 5000d310004a064b,1
>> Host controller port WWN 2100001b32940767
>> Controller /dev/cfg/c17
>> Device Address 5000d310004a064c,1
>> Host controller port WWN 2101001b32b40767
>>
>>
>> On 10/17/2012 06:25 PM, Majeed wrote:
>>>
>>> Hi All, We are doing some testing with omniOS in our datacenter and would
>>> love connect it up to our compellent san via fc connectivity. I slapped
>>> omniOS onto a dell r610 that has a qlogic dual port hba (each port is pathed
>>> to separate fabrics), performed zoning in each fabric, and assigned some
>>> luns. Now im not sure if compellent supports Solaris 11 (i can contact them
>>> and see about that), but it definately support solaris 10 which is what we
>>> use on other production systems.
>>>
>>> I'v actually made it pretty far through the process which is really
>>> exciting, except for the last multipathing bit. After mapping a volume to
>>> the host I can't seem to condense all the devices down to a single
>>> mulipathed device. It seems multipathing is enabled for the fp driver. I
>>> noticed there is not a mpathadm command.
>>>
>>> Anyone have any ideas? I'll post some output from the commands i think of
>>> below. Let me know if you would like to see any other information.
>>>
>>> Thanks much,
>>> Majeed
>>>
>>>
>>> On solaris 10 the main thing we have to do is add the following to
>>> /kernel/drv/scsi_vhci.conf (reconfigure reboot)
>>> device-type-scsi-options-list =
>>> "COMPELNTCompellent Vol", "symmetric-option";
>>> symmetric-option = 0x1000000;
>>>
>>>
>>> Here is the format output:
>>> AVAILABLE DISK SELECTIONS:
>>> 0. c13t5000CCA0213ED1E1d0 <HITACHI-HUC106030CSS600-A360 cyl 36470
>>> alt 2 hd 255 sec 63>
>>>
>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 40/disk at w5000cca0213ed1e1,0
>>> 1. c14t5000CCA0213EFC7Dd0 <HITACHI-HUC106030CSS600-A360 cyl 36469
>>> alt 2 hd 255 sec 63>
>>>
>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 80/disk at w5000cca0213efc7d,0
>>> 2. c16t5000D310004A064Bd1 <COMPELNT-CompellentVol-0505 cyl 65268
>>> alt 2 hd 255 sec 63>
>>>
>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1
>>> 3. c17t5000D310004A064Cd1 <COMPELNT-CompellentVol-0505 cyl 65268
>>> alt 2 hd 255 sec 63>
>>>
>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0/disk at w5000d310004a064c,1
>>> Specify disk (enter its number): Specify disk (enter its number):
>>>
>>>
>>>
>>> % fcinfo hba-port
>>> HBA Port WWN: 2100001b32940767
>>> Port Mode: Initiator
>>> Port ID: 91c00
>>> OS Device Name: /dev/cfg/c16
>>> Manufacturer: QLogic Corp.
>>> Model: 375-3356-02
>>> Firmware Version: 05.03.01
>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>> Serial Number: 0402H00-0941774287
>>> Driver Name: qlc
>>> Driver Version: 20100408-3.01
>>> Type: N-port
>>> State: online
>>> Supported Speeds: 1Gb 2Gb 4Gb
>>> Current Speed: 4Gb
>>> Node WWN: 2000001b32940767
>>> Max NPIV Ports: 127
>>> NPIV port list:
>>> HBA Port WWN: 2101001b32b40767
>>> Port Mode: Initiator
>>> Port ID: a1900
>>> OS Device Name: /dev/cfg/c17
>>> Manufacturer: QLogic Corp.
>>> Model: 375-3356-02
>>> Firmware Version: 05.03.01
>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>> Serial Number: 0402H00-0941774287
>>> Driver Name: qlc
>>> Driver Version: 20100408-3.01
>>> Type: N-port
>>> State: online
>>> Supported Speeds: 1Gb 2Gb 4Gb
>>> Current Speed: 4Gb
>>> Node WWN: 2001001b32b40767
>>> Max NPIV Ports: 127
>>> NPIV port list:
>>>
>>>
>>> % cfgadm -al -o show_SCSI_LUN
>>> Ap_Id Type Receptacle Occupant
>>> Condition
>>> c16 fc-fabric connected configured
>>> unknown
>>> c16::5000d310004a0649 unknown connected unconfigured
>>> unknown
>>> c16::5000d310004a064b,1 disk connected configured
>>> unknown
>>> c17 fc-fabric connected configured
>>> unknown
>>> c17::5000d310004a064a unknown connected unconfigured
>>> unknown
>>> c17::5000d310004a064c,1 disk connected configured
>>> unknown
>>>
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
--
Theo Schlossnagle
http://omniti.com/is/theo-schlossnagle
More information about the OmniOS-discuss
mailing list