[OmniOS-discuss] Problem with passed-through HBA

[Dan Swartzendruber]

I have an IBM M1015 HBA connected to a JBOD chassis with 6 SAS disks in a
3x2 raid10.  I was running this under ZFS on linux with no issues
(virtualized under ESXi 5.1).  I installed OmniOS with the latest updates,
shutdown the Ubuntu ZoL guest, removed the HBA from its config, added the
PCI card to the OmniOS guest and powered it up.  Only 4 of the 6 disks
come up, and as luck (or maybe not luck?) would have it, both were in the
same vdev, so 'no pool for you!'  I was freaking out about losing data -
tried swapping drives, booting the guest with drives added 1 by 1 to no
avail.  The errors I see in the log say something about 'device X failed
to power on'.  These messages are from the mptsas driver.  Out of
desperation, I shutdown the OmniOS guest, moved the HBA back to the Ubuntu
appliance and booted it.  All 6 drives spin up just fine, and I have a
perfectly healthy pool.  I can only assume it's a bug in the Illumos
mptsas driver (and/or some evil interaction between that driver and PCI
Passthrough?)  I would love to switch to OmniOS, since it's more stable
than ZoL (at least right now), and easier to configure.  But not having a
usable pool is an obvious showstopper :(  And please don't suggest
skipping passthrough and passing all 6 drives in via RDM...