[OmniOS-discuss] nvme-device hangs after power-on - after soft-reboot all is okay

[Frank M.]

Hi,

I have a problem with a NVMe-Device Samsung SM951 in a virtualized Illumos-environment.
My default setup is a HP G8 Server (I have tried others) as ESXi6.02-Host and a newest OmniOS-bloody-VM.

I´ve learned to setup the nvme from here: https://www.mail-archive.com/smartos-discuss@lists.smartos.org/msg02873.html.
Also I have read this: https://www.illumos.org/issues/6908 and other informations on web.

The progress of my installation you can read here (if you speak german) - please follow all posts of effemmess:  
http://www.hardwareluxx.de/community/f101/zfs-stammtisch-570052-272.html#post24644356
The problem in http://www.hardwareluxx.de/community/f101/zfs-stammtisch-570052-272.html#post24660135 is solved -it was a old updated vm, now I use an fresh installed vm...
But previosly I thought, there is something with the powerstates of the nvme. Because of that I made these additions to /etc/vmware/passthru.map (info is from here: http://www.vmware.com/pdf/vsp_4_vmdirectpath_host.pdf:
# Samsung NVMe
144d  a801  d3d0     false
144d  a802  d3d0     false
These additions are active till now...

The actual problem is, that after every cold-boot of the vm the nvme hangs. I test this by running the command "format". I cant see in any logs something interesting.
Now I reset the vm via second ssh and the command "reboot -p". Now works the nvme very well for hours. As long as I only reboot the vm all is okay, but if I cold-boot the vm, the nvme hangs again.
The whole process is reproducible. Gea says in hardwareluxx-forum, he has also problems with "format" with intel-NVMe.

Btw, in all Windows-VMs the NVMe runs promptly after every cold-vm-start.
Now I testet it in a freshly installed Openindiana-vm - full updated.
Here I have a critical fault after every cold-boot and soft-boot. Also a "fmadm repaired dev:////pci@0,0/pci15ad,7a0@15/pci144d,a801@0" doesn't help.

In summary, the NVMe runs always very well on bar-metal and on Windows-vm's and after soft-boot also very well in OmniOS.

Can anybody help?

Best regards,
 Frank 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20160616/5a1baf17/attachment.html>