[OmniOS-discuss] Slow Drive Detection and boot-archive

Fri Jul 17 23:07:09 UTC 2015

I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.

Is this normal?
Is there a way to speed that up?

I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.

# svccfg -s boot-archive setprop start/timeout_seconds=280

I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>

Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online

________________________
Michael Talbott
Systems Administrator
La Jolla Institute

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20150717/8f4205b0/attachment.html>