[OmniOS-discuss] multipathing with compellent
Majeed
mabuqu at ilstu.edu
Thu Oct 18 16:06:00 EDT 2012
Thanks Theo, that makes sense. I will definitely do that.
Majeed
On 10/18/2012 02:58 PM, Theo Schlossnagle wrote:
> This is a more general Illumos problem, you will open your
> troubleshooting to a larger audience by interfacing with the
> illumos-discuss list. I encourage you to post there! I'm subscribed
> there as well, but I haven't done multipathing against compellent and
> it appears no one else on the omnios-discuss list has either.
> Anything learned in the illumos-discuss group can be applied directly
> to OmniOS.
>
> On Thu, Oct 18, 2012 at 3:51 PM, Majeed<mabuqu at ilstu.edu> wrote:
>> Hi Again, just wanted to update the list with my latest progress and
>> thoughts on this topic. I think the issue lies in libmpscsi_vhci.so's lack
>> of support for multipathing with compellent.
>>
>> When I do "mpathadm show mpath-support libmpscsi_vhci.so" on omniOS I see
>> many venders and models listed but not compellent. We are going to spin up a
>> solaris 11 machine to see what it looks like. I wonder if this was forked
>> just before SUN/oracle added support for compellent (i could be way off base
>> here).
>>
>> When I do the same command on solaris 10 u10, at the very bottom of the list
>> i see.
>> Vendor: COMPELNT
>> Product: Compellent Vol
>> Revision:
>> Supported Load Balance Types:
>> round-robin
>>
>> I have verified that multipathing is working properly on other arrays. I
>> zoned up omniOS with all controllers on a 6140 (CSM200) that sit in multiple
>> fabrics, mapped a lun, and it looks wonderful. mpatham reports as expected,
>> and all paths are condensed to a single block device in format's output.
>>
>> % mpathadm list LU
>> /dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
>> Total Path Count: 4
>> Operational Path Count: 4
>>
>>
>>
>> It seems there are some options in /kernel/drv/scsi_vhci.conf to force
>> scsi_vhci to accept 3rd party devices (hopefully compellent) for failover,
>> but this is kinda getting over my head. It seems like if I had a proper
>> ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<x>" file for compellent and I
>> knew the proper VenderID and ProductID for the compellent devices, i might
>> be able to force it. Im not sure if can use the information from the format
>> inquiry to populate these values.
>>
>> format> inquiry
>> Vendor: COMPELNT
>> Product: Compellent Vol
>> Revision: 0505
>>
>>
>> Any thoughts on this would be excellent.
>>
>> Thanks,
>> Majeed Qulbain
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 10/18/2012 09:30 AM, Majeed wrote:
>>> I just noticed that we do have access to the mpathadm command by
>>> installing the "pkg:/storage/mpathadm at 0.5.11-0.151002" package. After
>>> installing I don't see any paths with "mpathadm list LU" which is what im
>>> used to on solaris 10 (i know I should be comparing to solaris 11, just
>>> havn't played with it much). I wonder if there are any other bits that I
>>> need to install or other configuration tweaks to get this going.
>>>
>>> Thanks in advance for any pointers,
>>> Majeed
>>>
>>> Some other related commands that might assist troubleshooting:
>>>
>>> % uname -a
>>> SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc
>>>
>>> % mpathadm list initiator-port
>>> Initiator Port: w5d4ae520937bbb00
>>> Initiator Port: iqn.1986-03.com.sun:01:b06a64f606ff.507f35b3,4000002a00ff
>>> Initiator Port: w5d4ae520937bbb00
>>> Initiator Port: 2101001b32b40767
>>> Initiator Port: 2100001b32940767
>>>
>>> % luxadm -e port
>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0:devctl
>>> CONNECTED
>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0:devctl
>>> CONNECTED
>>>
>>> % luxadm -v probe
>>>
>>> Found Fibre Channel device(s):
>>> Node WWN:5000d310004a0602 Device Type:Disk device
>>> Logical Path:/dev/rdsk/c16t5000D310004A064Bd1s2
>>> Node WWN:5000d310004a0602 Device Type:Disk device
>>> Logical Path:/dev/rdsk/c17t5000D310004A064Cd1s2
>>>
>>> % luxadm disp /dev/rdsk/c16t5000D310004A064Bd1s2
>>> DEVICE PROPERTIES for disk: /dev/rdsk/c16t5000D310004A064Bd1s2
>>> Vendor: COMPELNT
>>> Product ID: Compellent Vol
>>> Revision: 0505
>>> Serial Num: 00004a06-00000014
>>> Unformatted capacity: 512000.000 MBytes
>>> Read Cache: Enabled
>>> Minimum prefetch: 0x100
>>> Maximum prefetch: 0xf0
>>> Device Type: Disk device
>>> Path(s):
>>>
>>> /dev/rdsk/c16t5000D310004A064Bd1s2
>>>
>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1:c,raw
>>> Controller /dev/cfg/c16
>>> Device Address 5000d310004a064b,1
>>> Host controller port WWN 2100001b32940767
>>> Controller /dev/cfg/c17
>>> Device Address 5000d310004a064c,1
>>> Host controller port WWN 2101001b32b40767
>>>
>>>
>>> On 10/17/2012 06:25 PM, Majeed wrote:
>>>> Hi All, We are doing some testing with omniOS in our datacenter and would
>>>> love connect it up to our compellent san via fc connectivity. I slapped
>>>> omniOS onto a dell r610 that has a qlogic dual port hba (each port is pathed
>>>> to separate fabrics), performed zoning in each fabric, and assigned some
>>>> luns. Now im not sure if compellent supports Solaris 11 (i can contact them
>>>> and see about that), but it definately support solaris 10 which is what we
>>>> use on other production systems.
>>>>
>>>> I'v actually made it pretty far through the process which is really
>>>> exciting, except for the last multipathing bit. After mapping a volume to
>>>> the host I can't seem to condense all the devices down to a single
>>>> mulipathed device. It seems multipathing is enabled for the fp driver. I
>>>> noticed there is not a mpathadm command.
>>>>
>>>> Anyone have any ideas? I'll post some output from the commands i think of
>>>> below. Let me know if you would like to see any other information.
>>>>
>>>> Thanks much,
>>>> Majeed
>>>>
>>>>
>>>> On solaris 10 the main thing we have to do is add the following to
>>>> /kernel/drv/scsi_vhci.conf (reconfigure reboot)
>>>> device-type-scsi-options-list =
>>>> "COMPELNTCompellent Vol", "symmetric-option";
>>>> symmetric-option = 0x1000000;
>>>>
>>>>
>>>> Here is the format output:
>>>> AVAILABLE DISK SELECTIONS:
>>>> 0. c13t5000CCA0213ED1E1d0<HITACHI-HUC106030CSS600-A360 cyl 36470
>>>> alt 2 hd 255 sec 63>
>>>>
>>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 40/disk at w5000cca0213ed1e1,0
>>>> 1. c14t5000CCA0213EFC7Dd0<HITACHI-HUC106030CSS600-A360 cyl 36469
>>>> alt 2 hd 255 sec 63>
>>>>
>>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 80/disk at w5000cca0213efc7d,0
>>>> 2. c16t5000D310004A064Bd1<COMPELNT-CompellentVol-0505 cyl 65268
>>>> alt 2 hd 255 sec 63>
>>>>
>>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1
>>>> 3. c17t5000D310004A064Cd1<COMPELNT-CompellentVol-0505 cyl 65268
>>>> alt 2 hd 255 sec 63>
>>>>
>>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0/disk at w5000d310004a064c,1
>>>> Specify disk (enter its number): Specify disk (enter its number):
>>>>
>>>>
>>>>
>>>> % fcinfo hba-port
>>>> HBA Port WWN: 2100001b32940767
>>>> Port Mode: Initiator
>>>> Port ID: 91c00
>>>> OS Device Name: /dev/cfg/c16
>>>> Manufacturer: QLogic Corp.
>>>> Model: 375-3356-02
>>>> Firmware Version: 05.03.01
>>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>>> Serial Number: 0402H00-0941774287
>>>> Driver Name: qlc
>>>> Driver Version: 20100408-3.01
>>>> Type: N-port
>>>> State: online
>>>> Supported Speeds: 1Gb 2Gb 4Gb
>>>> Current Speed: 4Gb
>>>> Node WWN: 2000001b32940767
>>>> Max NPIV Ports: 127
>>>> NPIV port list:
>>>> HBA Port WWN: 2101001b32b40767
>>>> Port Mode: Initiator
>>>> Port ID: a1900
>>>> OS Device Name: /dev/cfg/c17
>>>> Manufacturer: QLogic Corp.
>>>> Model: 375-3356-02
>>>> Firmware Version: 05.03.01
>>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>>> Serial Number: 0402H00-0941774287
>>>> Driver Name: qlc
>>>> Driver Version: 20100408-3.01
>>>> Type: N-port
>>>> State: online
>>>> Supported Speeds: 1Gb 2Gb 4Gb
>>>> Current Speed: 4Gb
>>>> Node WWN: 2001001b32b40767
>>>> Max NPIV Ports: 127
>>>> NPIV port list:
>>>>
>>>>
>>>> % cfgadm -al -o show_SCSI_LUN
>>>> Ap_Id Type Receptacle Occupant
>>>> Condition
>>>> c16 fc-fabric connected configured
>>>> unknown
>>>> c16::5000d310004a0649 unknown connected unconfigured
>>>> unknown
>>>> c16::5000d310004a064b,1 disk connected configured
>>>> unknown
>>>> c17 fc-fabric connected configured
>>>> unknown
>>>> c17::5000d310004a064a unknown connected unconfigured
>>>> unknown
>>>> c17::5000d310004a064c,1 disk connected configured
>>>> unknown
>>>>
>>>>
>>>> _______________________________________________
>>>> OmniOS-discuss mailing list
>>>> OmniOS-discuss at lists.omniti.com
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
--
Majeed Abu-Qulbain
Lead Unix Admin - CISS
309-438-3455
Illinois State University
More information about the OmniOS-discuss
mailing list