From danmcd at omniti.com Tue Mar 1 18:55:42 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 13:55:42 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes Message-ID: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> Please "pkg update" your r151006 (old LTS), r151014 (LTS), or r151016 (Stable) systems. All of the aforementioned releases will get new versions of OpenSSL that addresses the DROWN attack (CVE-2016-0800), and an update to Perl that addresses an environment duplication attack (CVE-2016-2381). Furthermore, r151014 & r151016 will receive OpenSSH updates that catch it up with certain SunSSH features (like GSSAPI support) that are currently in bloody. Also, r151014 will receive small SMF updates to NTP and ISC DHCP that enable auto-restart of these services upon any future software updates. OmniOS bloody will receive a full refresh update within the next 72 hours. NOTE that SSLv2 and MD2 support are deprecated with this update (OpenSSL 1.0.2g for r151014 and later, OpenSSL 1.0.1s for r151006). Happy patching! Dan p.s. r151006 still gets security updates, but that will stop soon. I'll discuss under a separate email. From colin at omniti.com Tue Mar 1 19:16:45 2016 From: colin at omniti.com (Colin Roche-Dutch) Date: Tue, 1 Mar 2016 14:16:45 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> Message-ID: Hello, The new OpenSSL update to address the DROWN attack is causing issues with the pkg system, specifically with python due to the SSLv2 removal. Please DO NOT update to the recently released OpenSSL package yet. Dan will be sending a follow up email once he has a fix for this in place that may include additional information. If you have any questions, please let us know. -Thanks, Colin Roche-Dutch On Tue, Mar 1, 2016 at 1:55 PM, Dan McDonald wrote: > Please "pkg update" your r151006 (old LTS), r151014 (LTS), or r151016 > (Stable) systems. > > All of the aforementioned releases will get new versions of OpenSSL that > addresses the DROWN attack (CVE-2016-0800), and an update to Perl that > addresses an environment duplication attack (CVE-2016-2381). > > Furthermore, r151014 & r151016 will receive OpenSSH updates that catch it > up with certain SunSSH features (like GSSAPI support) that are currently in > bloody. Also, r151014 will receive small SMF updates to NTP and ISC DHCP > that enable auto-restart of these services upon any future software updates. > > OmniOS bloody will receive a full refresh update within the next 72 hours. > > NOTE that SSLv2 and MD2 support are deprecated with this update (OpenSSL > 1.0.2g for r151014 and later, OpenSSL 1.0.1s for r151006). > > Happy patching! > Dan > > p.s. r151006 still gets security updates, but that will stop soon. I'll > discuss under a separate email. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Tue Mar 1 19:40:05 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 20:40:05 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> Message-ID: <20160301204005.6529b604@sleipner.datanom.net> Hi Colin, On Tue, 1 Mar 2016 14:16:45 -0500 Colin Roche-Dutch wrote: > The new OpenSSL update to address the DROWN attack is causing issues with > the pkg system, specifically with python due to the SSLv2 removal. Please > DO NOT update to the recently released OpenSSL package yet. > And if we already have done the update what to do then? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Punning is the worst vice, and there's no vice versa. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Tue Mar 1 19:42:29 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 20:42:29 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> Message-ID: <20160301204229.4443cf4c@sleipner.datanom.net> On Tue, 1 Mar 2016 14:16:45 -0500 Colin Roche-Dutch wrote: > The new OpenSSL update to address the DROWN attack is causing issues with > the pkg system, specifically with python due to the SSLv2 removal. Please > DO NOT update to the recently released OpenSSL package yet. > I guess this is what you see? :-( # pkg update -nv Traceback (most recent call last): File "/usr/bin/pkg", line 67, in import pkg.actions as actions File "/usr/lib/python2.6/vendor-packages/pkg/actions/__init__.py", line 68, in globals(), locals(), [modname]) File "/usr/lib/python2.6/vendor-packages/pkg/actions/legacy.py", line 39, in import generic File "/usr/lib/python2.6/vendor-packages/pkg/actions/generic.py", line 48, in import pkg.variant as variant File "/usr/lib/python2.6/vendor-packages/pkg/variant.py", line 35, in from pkg.misc import EmptyI File "/usr/lib/python2.6/vendor-packages/pkg/misc.py", line 30, in import OpenSSL.crypto as osc File "/usr/lib/python2.6/vendor-packages/OpenSSL/__init__.py", line 45, in from OpenSSL import rand, SSL ImportError: ld.so.1: python2.6: fatal: relocation error: file /usr/lib/python2.6/vendor-packages/OpenSSL/64/SSL.so: symbol SSLv2_method: referenced symbol not found -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: He hated being thought of as one of those people that wore stupid ornamental armour. It was gilt by association. -- Terry Pratchett, "Night Watch" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From colin at omniti.com Tue Mar 1 20:09:36 2016 From: colin at omniti.com (Colin Roche-Dutch) Date: Tue, 1 Mar 2016 15:09:36 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301204229.4443cf4c@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> Message-ID: Hi Michael, If you updated a zone but not the global yet, you can revert the zbe back to the old version of openssl: pkg -R /root update openssl@ previous in this case would be: For 016: pkg://omnios/library/security/openssl at 1.0.2.6 ,5.11-0.151016:20160128T191210Z For 014: pkg://omnios/library/security/openssl at 1.0.2.6-0.151014 :20160128T191031Z For 06: pkg://omnios/library/security/openssl at 1.0.1.18 ,5.11-0.151006:20160202T152535Z If you updated a global zone, you will need to boot to the backup BE that was created during the pkg update to revert to the previous version. Also as an update, the broken package has been removed from the r151006/r151014/r151016 repos. -Thanks, Colin On Tue, Mar 1, 2016 at 2:42 PM, Michael Rasmussen wrote: > On Tue, 1 Mar 2016 14:16:45 -0500 > Colin Roche-Dutch wrote: > > > The new OpenSSL update to address the DROWN attack is causing issues with > > the pkg system, specifically with python due to the SSLv2 removal. Please > > DO NOT update to the recently released OpenSSL package yet. > > > I guess this is what you see? :-( > # pkg update -nv > Traceback (most recent call last): > File "/usr/bin/pkg", line 67, in > import pkg.actions as actions > File "/usr/lib/python2.6/vendor-packages/pkg/actions/__init__.py", line > 68, in > globals(), locals(), [modname]) > File "/usr/lib/python2.6/vendor-packages/pkg/actions/legacy.py", line > 39, in > import generic > File "/usr/lib/python2.6/vendor-packages/pkg/actions/generic.py", line > 48, in > import pkg.variant as variant > File "/usr/lib/python2.6/vendor-packages/pkg/variant.py", line 35, in > > from pkg.misc import EmptyI > File "/usr/lib/python2.6/vendor-packages/pkg/misc.py", line 30, in > > import OpenSSL.crypto as osc > File "/usr/lib/python2.6/vendor-packages/OpenSSL/__init__.py", line 45, > in > from OpenSSL import rand, SSL > ImportError: ld.so.1: python2.6: fatal: relocation error: file > /usr/lib/python2.6/vendor-packages/OpenSSL/64/SSL.so: symbol SSLv2_method: > referenced symbol not found > > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > He hated being thought of as one of those people that wore stupid > ornamental armour. It was gilt by association. > -- Terry Pratchett, "Night Watch" > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Mar 1 20:23:17 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 15:23:17 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> Message-ID: <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> > On Mar 1, 2016, at 3:09 PM, Colin Roche-Dutch wrote: > > Also as an update, the broken package has been removed from the r151006/r151014/r151016 repos. Thank you Colin for the quick save. We're working out the best course of forward action. That people still *use* SSLv2 post-2010 amazes me, but apparently I shouldn't be amazed. Stay tuned, we'll keep you up to date on it. Dan From pasztor at sagv5.gyakg.u-szeged.hu Tue Mar 1 20:53:38 2016 From: pasztor at sagv5.gyakg.u-szeged.hu (=?iso-8859-2?Q?P=C1SZTOR_Gy=F6rgy?=) Date: Tue, 1 Mar 2016 21:53:38 +0100 Subject: [OmniOS-discuss] theoretical question: oi151a9->omni151014 Message-ID: <20160301205338.GA25952@linux.gyakg.u-szeged.hu> Hi, It is a theoretical question, what I will write, but I would like to know answers and thoughts about it, since I will try that in practice soon: I have an OpenIndiana 151a9 on my home nas. Most of the services are run from zones. There are only one (or few) thing(s), which run from the GZ: nis-master, and nfs/samba shares. (Sharings are handled through the appropriate zfs attributes, there is nothing extra around that.) There is a (nis-)slave is in one of the zones. Most of the zones are "typical" (if that word exists at all in this context :D) openindiana zones too with the same 151a9 version. Exceptions: One of the zones run virtualbox VMs. (Currently only one, but previously, I had two, and later, maybe I will use more then that.) One of the zones run omnios: pasztor at omni:~$ pkg info entire |grep FMRI FMRI: pkg://omnios/entire at 11-0.151014:20150402T192159Z In one of the zones, I run a pkgrepo, I tried to build packages based on sfe specfiles, which was not exactly there, when I started to use my nas. eg. smartmontools. Then I added this repo to my gz. One extra thing about this home-nas: the rpool is on a 64G pendrive now. The zones, and my important data is on the spinning disks. (The name of that pool is tank. :D) I would like to use OmniOS on the host system instead. I had many reasons for why I would like that. In theory I have two idea how to implement this "upgrade": Idea #1: * stop all the zones. * export zone configurations, and save them onto a dataset on tank * Make snapshot about all the important zfs datasets in rpool, * then make a backup copy to tank using send&receive. * Plug out the pendrive * Plug in an empty new pendrive, and install omnios onto it * import zone configurations * attach the zones * start the zones * fix non-working things, eg.: recreate or reorganize my nis service * be happy! If something fails, I have a fallback: plug back the original pendrive Idea #2: * plug in the new & empty pendrive into the system * follow this guide to mirror the rpool: http://wiki.openindiana.org/oi/2.1+Post-installation * test the system if one of the pendrives is enough * plug out one of the pendrives, and consider it as a fallback * try some magic with pkg publishers /* magic happens here !!! */ * try to upgrade entire to omnios's 151014 /* and here :D */ * handle / fix the unexpected gotchas /* maybe here too :) */ * be happy! If something fails, I can have two fallback: * the pendrive, which I removed * the previous BE, which contains my last working OI system :) If I am honest, then I like idea#2 much more. It may contain adventures, but would be a really nice way if it'd work. It was also an adventure, when oracle stoped opensolaris, or at least things around that. But we wanted to upgrade our production storage server at my previous workplace from build143 to openindiana 151a, and we had to manually handle some package dependencies, etc. and that was not even well tested or well documented on openindiana's wiki to, how to do that, but at least, it was a good starting point. I think, this situation is a bit similar to that old story :) Does sy. has some good advice, best practice, mathematical proof about that it will never ever work, etc? One though that come into my mind: Try the whole situation (idea#2) first in a virtualbox machine, to see at least if this kind of upgrade could work. Not with this kind of complexity, just a freshly installed oi151a8 upgrade to a9, than to omni151014. (I have not done this yet, but I think, this will be the 0th step.) Did anybody tried this ever? Kind regards, Gyu From mir at miras.org Tue Mar 1 22:23:48 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 23:23:48 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> Message-ID: <20160301232348.17dbc5ab@sleipner.datanom.net> On Tue, 1 Mar 2016 15:09:36 -0500 Colin Roche-Dutch wrote: > > Also as an update, the broken package has been removed from the > r151006/r151014/r151016 repos. > Reactivated backup BE before update and did a reboot. pkg work again. Rerun pkg update which still include and update of libssl which is still broken so the broken package is not removed, at least not from r151014. I will revert back again and not touch any update until this has been sorted out. This is my production server so I have no more service window available. After new update: Traceback (most recent call last): File "/usr/bin/pkg", line 67, in import pkg.actions as actions File "/usr/lib/python2.6/vendor-packages/pkg/actions/__init__.py", line 68, in globals(), locals(), [modname]) File "/usr/lib/python2.6/vendor-packages/pkg/actions/legacy.py", line 39, in import generic File "/usr/lib/python2.6/vendor-packages/pkg/actions/generic.py", line 48, in import pkg.variant as variant File "/usr/lib/python2.6/vendor-packages/pkg/variant.py", line 35, in from pkg.misc import EmptyI File "/usr/lib/python2.6/vendor-packages/pkg/misc.py", line 30, in import OpenSSL.crypto as osc File "/usr/lib/python2.6/vendor-packages/OpenSSL/__init__.py", line 45, in from OpenSSL import rand, SSL ImportError: ld.so.1: python2.6: fatal: relocation error: file /usr/lib/python2.6/vendor-packages/OpenSSL/64/SSL.so: symbol SSLv2_method: referenced symbol not found -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Q: What do you call a principal female opera singer whose high C is lower than those of other principal female opera singers? A: A deep C diva. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Tue Mar 1 22:36:06 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 23:36:06 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301232348.17dbc5ab@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> Message-ID: <20160301233606.59d5d1dc@sleipner.datanom.net> On Tue, 1 Mar 2016 23:23:48 +0100 Michael Rasmussen wrote: > On Tue, 1 Mar 2016 15:09:36 -0500 > Colin Roche-Dutch wrote: > > > > > Also as an update, the broken package has been removed from the > > r151006/r151014/r151016 repos. > > > Reactivated backup BE before update and did a reboot. pkg work again. > Rerun pkg update which still include and update of libssl which is > still broken so the broken package is not removed, at least not from > r151014. I will revert back again and not touch any update until this has been sorted out. This is my production server so I have no more service window available. > Did a pkg refresh --full which gives this: Changed packages: omnios runtime/perl 5.16.1-0.151014:20150402T192953Z -> 5.16.1-0.151014:20160301T161253Z runtime/perl/manual 5.16.1-0.151014:20150402T193025Z -> 5.16.1-0.151014:20160301T161325Z service/network/ntp 4.2.8.6-0.151014:20160127T163640Z -> 4.2.8.6-0.151014:20160301T160738Z system/zones/brand/ipkg 0.5.11-0.151014:20150402T184311Z -> 0.5.11-0.151014:20160226T182222Z system/zones/brand/lipkg 0.5.11-0.151014:20150402T184311Z -> 0.5.11-0.151014:20160226T182223Z Services: restart_fmri: svc:/network/ntp:default And voila no libssl. Was this the missing magic? Can I safely upgrade now? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Kinkler's First Law: Responsibility always exceeds authority. Kinkler's Second Law: All the easy problems have been solved. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Tue Mar 1 22:36:30 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 17:36:30 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301232348.17dbc5ab@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> Message-ID: <3F85157A-116D-4FB2-B74A-CC0BF6B45F5F@omniti.com> > On Mar 1, 2016, at 5:23 PM, Michael Rasmussen wrote: > > On Tue, 1 Mar 2016 15:09:36 -0500 > Colin Roche-Dutch wrote: > >> >> Also as an update, the broken package has been removed from the >> r151006/r151014/r151016 repos. >> > Reactivated backup BE before update and did a reboot. pkg work again. > Rerun pkg update which still include and update of libssl which is > still broken so the broken package is not removed, at least not from > r151014. I will revert back again and not touch any update until this has been sorted out. This is my production server so I have no more service window available. > > After new update: When PRECISELY did you update this? And if you have the broken BE around, Could you please utter this: beadm mount broken-BE /mnt pkg -R /mnt list -v openssl beadm umount /mnt Thanks, Dan From danmcd at omniti.com Tue Mar 1 22:48:19 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 17:48:19 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301233606.59d5d1dc@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <20160301233606.59d5d1dc@sleipner.datanom.net> Message-ID: <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> > On Mar 1, 2016, at 5:36 PM, Michael Rasmussen wrote: > > And voila no libssl. Was this the missing magic? Can I safely upgrade > now? Yes. ALSO -- I am very close to having a fixed openssl 1.0.2g, so if you wait a bit, you can get that again as well. I'm VERY sorry for breaking things, Dan From mir at miras.org Tue Mar 1 22:53:21 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 23:53:21 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <20160301233606.59d5d1dc@sleipner.datanom.net> <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> Message-ID: <20160301235321.63c0302b@sleipner.datanom.net> On Tue, 1 Mar 2016 17:48:19 -0500 Dan McDonald wrote: > > Yes. ALSO -- I am very close to having a fixed openssl 1.0.2g, so if you wait a bit, you can get that again as well. > I will wait then. > I'm VERY sorry for breaking things, No worries. The excellent BE utility in Omnios showed its potential and saved my nights sleep;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Behind every great man, there is a woman -- urging him on. -- Harry Mudd, "I, Mudd", stardate 4513.3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Tue Mar 1 22:56:45 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 1 Mar 2016 23:56:45 +0100 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <3F85157A-116D-4FB2-B74A-CC0BF6B45F5F@omniti.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <3F85157A-116D-4FB2-B74A-CC0BF6B45F5F@omniti.com> Message-ID: <20160301235645.006e1eef@sleipner.datanom.net> On Tue, 1 Mar 2016 17:36:30 -0500 Dan McDonald wrote: > > And if you have the broken BE around, Could you please utter this: > > beadm mount broken-BE /mnt > pkg -R /mnt list -v openssl > beadm umount /mnt > # beadm mount omnios-r151014-mar-1 /mnt # pkg -R /mnt list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.2.7-0.151014:20160301T160526Z i-- For your information this is from the current BE: # pkg list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.2.6-0.151014:20160128T191031Z i-- -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: It's not really a rule--it's more like a trend. -- Larry Wall in <199710221721.KAA24321 at wall.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Tue Mar 1 22:57:25 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 17:57:25 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <20160301233606.59d5d1dc@sleipner.datanom.net> <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> Message-ID: > On Mar 1, 2016, at 5:48 PM, Dan McDonald wrote: > > Yes. ALSO -- I am very close to having a fixed openssl 1.0.2g, so if you wait a bit, you can get that again as well. Fixed! New mail coming out soon. Dan From danmcd at omniti.com Tue Mar 1 22:58:55 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 17:58:55 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301235645.006e1eef@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <3F85157A-116D-4FB2-B74A-CC0BF6B45F5F@omniti.com> <20160301235645.006e1eef@sleipner.datanom.net> Message-ID: <935F5A81-C954-468A-BA4D-650E209C2CE5@omniti.com> > On Mar 1, 2016, at 5:56 PM, Michael Rasmussen wrote: > > # beadm mount omnios-r151014-mar-1 /mnt > # pkg -R /mnt list -v openssl > FMRI IFO > pkg://omnios/library/security/openssl at 1.0.2.7-0.151014:20160301T160526Z i-- > > For your information this is from the current BE: > # pkg list -v openssl > FMRI IFO > pkg://omnios/library/security/openssl at 1.0.2.6-0.151014:20160128T191031Z i-- > > -- That tracks! And you can move forward now with this: r151014(omnios-build/build)[0]% pkg list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.2.7-0.151014:20160301T224747Z i-- r151014(omnios-build/build)[0]% Full announcement coming soon. Dan From danmcd at omniti.com Tue Mar 1 23:06:26 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 18:06:26 -0500 Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled Message-ID: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> First off, I apologize for breaking pkg(5) and who-know-what else. I honestly didn't think SSLv2 would still be used, even just by software autoconf'ed to detect it. Second off, I have rebuild OpenSSL 1.0.2g and 1.0.1s to INCLUDE the SSLv2_* entry points, from which all of the pkg(5) breakage resulted. Make sure you're installing the very latest versions: r151006(~)[0]% pkg list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.1.19,5.11-0.151006:20160301T225327Z i-- r151006(~)[0]% r151014(~)[0]% pkg list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.2.7-0.151014:20160301T224747Z i-- r151014(~)[0]% r151016(~)[0]% pkg list -v openssl FMRI IFO pkg://omnios/library/security/openssl at 1.0.2.7-0.151016:20160301T215356Z i-- r151016(~)[0]% Those also demonstrate pkg(5) working with latest OpenSSL. :) Bloody's fate remains up in the air. I'm contemplating removing SSLv2 support from bloody, and when it ships, r151018. This will require, however, some godawful bootstrapping, akin to the gcc version change I did for r151015/6. Anyone who's a fan of bloody should followup on this thread to tell me what you think. Thanks, and again, sorry, Dan From danmcd at omniti.com Tue Mar 1 23:07:27 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 18:07:27 -0500 Subject: [OmniOS-discuss] SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes In-Reply-To: <20160301235321.63c0302b@sleipner.datanom.net> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <20160301232348.17dbc5ab@sleipner.datanom.net> <20160301233606.59d5d1dc@sleipner.datanom.net> <107AB630-69D8-4845-B0CD-9A41BC39032B@omniti.com> <20160301235321.63c0302b@sleipner.datanom.net> Message-ID: > On Mar 1, 2016, at 5:53 PM, Michael Rasmussen wrote: > > No worries. The excellent BE utility in Omnios showed its potential and > saved my nights sleep;-) If I had a dollar for every time beadm(1M) saved my bacon, I wouldn't have to work. :) Thanks again, Dan From danmcd at omniti.com Tue Mar 1 23:30:44 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 1 Mar 2016 18:30:44 -0500 Subject: [OmniOS-discuss] EOSL coming for 151016, April 1st Message-ID: This is the last month for OmniOS r151006 to potentially receive updates. With the April release of r151018, April 1st will be the end of service life (EOSL) for OmniOS r151006. PLEASE update any r151006 machines to the very latest r151006, and THEN to r151014. This is keeping with the OmniOS release cycle: http://omnios.omniti.com/wiki.php/ReleaseCycle Thanks, Dan From mir at miras.org Tue Mar 1 23:34:28 2016 From: mir at miras.org (Michael Rasmussen) Date: Wed, 2 Mar 2016 00:34:28 +0100 Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled In-Reply-To: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> References: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> Message-ID: <20160302003428.6a9d8cc7@sleipner.datanom.net> On Tue, 1 Mar 2016 18:06:26 -0500 Dan McDonald wrote: > Those also demonstrate pkg(5) working with latest OpenSSL. :) > I can confirm this working on r151014. Thanks Dan - you deserve your good nights sleep;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: You can't break eggs without making an omelet. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From jdg117 at elvis.arl.psu.edu Tue Mar 1 23:56:27 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Tue, 01 Mar 2016 18:56:27 -0500 Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled In-Reply-To: Your message of "Tue, 01 Mar 2016 18:06:26 EST." <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> References: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> Message-ID: <201603012356.u21NuRsr002153@elvis.arl.psu.edu> In message <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004 at omniti.com>, Dan McDonald writ es: >First off, I apologize for breaking pkg(5) and who-know-what else. I honestly > didn't think SSLv2 would still be used, even just by software autoconf'ed to >detect it. I hit this recently with a modern Perl module or Ruby gem that raised hell about the lack of exported SSLv2 functions from the modern openssl. Oh soooo much cruft out there to either be stripped out or exploited. Thanks for trying... John groenveld at acm.org From bfriesen at simple.dallas.tx.us Wed Mar 2 00:08:11 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Tue, 1 Mar 2016 18:08:11 -0600 (CST) Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled In-Reply-To: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> References: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> Message-ID: On Tue, 1 Mar 2016, Dan McDonald wrote: > > Bloody's fate remains up in the air. I'm contemplating removing > SSLv2 support from bloody, and when it ships, r151018. This will > require, however, some godawful bootstrapping, akin to the gcc > version change I did for r151015/6. Anyone who's a fan of bloody > should followup on this thread to tell me what you think. If you remove SSLv2 APIs without bumping the major interface of the library, then you will curse all already-built user applications with the same fate which befell Python. If you bump the major interface of the library, then the old library still needs to be available to support existing apps. We are already on the latest OpenSSL release on the newest branch so until upstream makes a breaking release (e.g. the planned 1.1.0), then it is not so convenient for OmniOS to do so. If you wait for 1.1.0, then it may be much easier. Perhaps it is possible to tweak the library (or config file) so that SSLv2 won't acutally be used. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From miamikelvin at gmail.com Wed Mar 2 10:37:07 2016 From: miamikelvin at gmail.com (Miami Kelvin) Date: Wed, 2 Mar 2016 13:37:07 +0300 Subject: [OmniOS-discuss] poweradm on OmniOs Message-ID: Hello! I am new to OmniOS, I am learning few Omni admin commands using OPenSolaris Bible and other books as information source. I am having a problem running the poweradm command on Omni even after enabling svc:/system/power it return -bash: poweradm command not found What could be the problem -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at qutic.com Wed Mar 2 11:06:07 2016 From: mailinglists at qutic.com (qutic development) Date: Wed, 2 Mar 2016 12:06:07 +0100 Subject: [OmniOS-discuss] https://pkg.omniti.com (was: SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes) In-Reply-To: <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> Message-ID: Hi Dan, > Am 01.03.2016 um 21:23 schrieb Dan McDonald : > > We're working out the best course of forward action. That people still *use* SSLv2 post-2010 amazes me, but apparently I shouldn't be amazed. I would like to connect via https to the omnios repos under pkg.omniti.com. Would that be possible in the near future? - Stefan From ben at bens.me.uk Wed Mar 2 11:08:51 2016 From: ben at bens.me.uk (Ben Summers) Date: Wed, 2 Mar 2016 11:08:51 +0000 Subject: [OmniOS-discuss] https://pkg.omniti.com (was: SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes) In-Reply-To: References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> Message-ID: <0421CA0C-3C3D-421E-912B-0E399498E419@bens.me.uk> > On 2 Mar 2016, at 11:06, qutic development wrote: > > Hi Dan, > >> Am 01.03.2016 um 21:23 schrieb Dan McDonald : >> >> We're working out the best course of forward action. That people still *use* SSLv2 post-2010 amazes me, but apparently I shouldn't be amazed. > > I would like to connect via https to the omnios repos under pkg.omniti.com. > > Would that be possible in the near future? This was rejected previously due to the significant additional latency of https. Now that packages are signed properly, you don't need https to assure integrity of the software. If you wish to avoid disclosing your updates to passive observers, you could use a local mirror. Ben From mailinglists at qutic.com Wed Mar 2 11:41:38 2016 From: mailinglists at qutic.com (qutic development) Date: Wed, 2 Mar 2016 12:41:38 +0100 Subject: [OmniOS-discuss] https://pkg.omniti.com (was: SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes) In-Reply-To: <0421CA0C-3C3D-421E-912B-0E399498E419@bens.me.uk> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> <0421CA0C-3C3D-421E-912B-0E399498E419@bens.me.uk> Message-ID: <1051B600-4BBF-49C0-9A70-D1A4DF52BEA2@qutic.com> > Am 02.03.2016 um 12:08 schrieb Ben Summers : > > This was rejected previously due to the significant additional latency of https. Please, please do not spread myth from the last century. This is not true! Add a proper tls-termination in front and you are good to go. > Now that packages are signed properly, you don't need https to assure integrity of the software. Yes signed packages are fine, but not my case. As you now your county is taking a full take - on all they can get! > If you wish to avoid disclosing your updates to passive observers, you could use a local mirror. Yes I could be, but that does not make the internet a better and more secure place! - Stefan From ben at bens.me.uk Wed Mar 2 11:52:07 2016 From: ben at bens.me.uk (Ben Summers) Date: Wed, 2 Mar 2016 11:52:07 +0000 Subject: [OmniOS-discuss] https://pkg.omniti.com (was: SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes) In-Reply-To: <1051B600-4BBF-49C0-9A70-D1A4DF52BEA2@qutic.com> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> <0421CA0C-3C3D-421E-912B-0E399498E419@bens.me.uk> <1051B600-4BBF-49C0-9A70-D1A4DF52BEA2@qutic.com> Message-ID: <28874BE7-9A49-4809-8D4C-69592B720956@bens.me.uk> > On 2 Mar 2016, at 11:41, qutic development wrote: > > >> Am 02.03.2016 um 12:08 schrieb Ben Summers : >> >> This was rejected previously due to the significant additional latency of https. > > > Please, please do not spread myth from the last century. This is not true! > Add a proper tls-termination in front and you are good to go. I believe this was measured. pkg makes lots of small requests and doesn't appear to be very clever with session management. What results did you get when you benchmarked it? > >> Now that packages are signed properly, you don't need https to assure integrity of the software. > > > Yes signed packages are fine, but not my case. As you now your county is taking a full take - on all they can get! Which country? > >> If you wish to avoid disclosing your updates to passive observers, you could use a local mirror. > > > Yes I could be, but that does not make the internet a better and more secure place! I found OmniTI to be really open to making security improvements, and I'm sure they would be very interested in learning about your specific concerns. Ben From mailinglists at qutic.com Wed Mar 2 13:39:26 2016 From: mailinglists at qutic.com (qutic development) Date: Wed, 2 Mar 2016 14:39:26 +0100 Subject: [OmniOS-discuss] https://pkg.omniti.com (was: SECURITY UPDATE FOR OpenSSL & Perl; plus other fixes) In-Reply-To: <28874BE7-9A49-4809-8D4C-69592B720956@bens.me.uk> References: <7A769B3F-8890-4B31-A0D0-6CA7AD71982A@omniti.com> <20160301204229.4443cf4c@sleipner.datanom.net> <983E070C-7C55-4D98-AA22-67AA1211BD84@omniti.com> <0421CA0C-3C3D-421E-912B-0E399498E419@bens.me.uk> <1051B600-4BBF-49C0-9A70-D1A4DF52BEA2@qutic.com> <28874BE7-9A49-4809-8D4C-69592B720956@bens.me.uk> Message-ID: <68C10D10-43D0-40F1-8BCE-A8FF8CF86D23@qutic.com> > Am 02.03.2016 um 12:52 schrieb Ben Summers : > > I believe this was measured. pkg makes lots of small requests and doesn't appear to be very clever with session management. > > What results did you get when you benchmarked it? Sorry my answer shouldn?t sound that harsh. We do have our own pkg-Server over https with a tls-terminating loadbalancer and request times are mostly under 20ms. >> Yes signed packages are fine, but not my case. As you now your county is taking a full take - on all they can get! > Which country? Great Britain > I found OmniTI to be really open to making security improvements, and I'm sure they would be very interested in learning about your specific concerns. Thank you for pointing that out. I would be happy to get direct feedback. - Stefan From peter.tribble at gmail.com Wed Mar 2 19:26:45 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Wed, 2 Mar 2016 19:26:45 +0000 Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled In-Reply-To: References: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> Message-ID: On Wed, Mar 2, 2016 at 12:08 AM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Tue, 1 Mar 2016, Dan McDonald wrote: > >> >> Bloody's fate remains up in the air. I'm contemplating removing SSLv2 >> support from bloody, and when it ships, r151018. This will require, >> however, some godawful bootstrapping, akin to the gcc version change I did >> for r151015/6. Anyone who's a fan of bloody should followup on this thread >> to tell me what you think. >> > > If you remove SSLv2 APIs without bumping the major interface of the > library, then you will curse all already-built user applications with the > same fate which befell Python. If you bump the major interface of the > library, then the old library still needs to be available to support > existing apps. > > We are already on the latest OpenSSL release on the newest branch so until > upstream makes a breaking release (e.g. the planned 1.1.0), then it is not > so convenient for OmniOS to do so. If you wait for 1.1.0, then it may be > much easier. > IIRC, 1.1.0 has this change already. That's fine, as it's a new release and is allowed to make incompatible changes. > Perhaps it is possible to tweak the library (or config file) so that SSLv2 > won't acutally be used. > Actually, no. What would be ideal is that openssl provided stub functions that return an error, so symbol resolution works fine (but anything actually calling SSLv2 will fail). As it is, they're yanking the functions and breaking binary compatibility by default. Things are made worse by the fact that consumers of the openssl library (things like wget, libcurl) tend to blindly enable SSLv2 support if it's present in the openssl implementation they're built against. Often without a way of disabling it otherwise. So you either have to work out how to manually disable SSLv2 for those consumers, or build them on a system that has openssl installed but with SSLv2 disabled. Then, of course, you have to make sure that updated consumers get pushed out and updated by users *before* you push out a "fixed" openssl. And if end users have built applications, then they're up the creek without a paddle. It's just a mess. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Wed Mar 2 19:59:08 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 2 Mar 2016 13:59:08 -0600 (CST) Subject: [OmniOS-discuss] PHEW! OpenSSL 1.0.2g and 1.0.1s NOW OUT, albeit with SSLv2_* enabled In-Reply-To: References: <15079DC7-D6C8-45D1-9CA5-B7BE5CB18004@omniti.com> Message-ID: On Wed, 2 Mar 2016, Peter Tribble wrote: > > IIRC, 1.1.0 has this change already. That's fine, as it's a new release and is allowed > to make incompatible changes. Yes, that is why I mentioned it. > Perhaps it is possible to tweak the library (or config file) so that SSLv2 won't acutally be used. > > > Actually, no. What would be ideal is that openssl provided stub functions that return > an error, so symbol resolution works fine (but anything actually calling SSLv2 will fail). > As it is, they're yanking the functions and breaking binary compatibility by default. As long as all SSLv2 code has been stripped out, this is safest. Otherwise it will be very difficult for OmniOS users to upgrade since programs will refuse to run. There is still a question of what existing application code might do (continue on, quit, crash, lock-up?) if an error is reported by a stub function. > Things are made worse by the fact that consumers of the openssl library (things like wget, > libcurl) tend to blindly enable SSLv2 support if it's present in the openssl implementation > they're built against. Often without a way of disabling it otherwise. So you either have to > work out how to manually disable SSLv2 for those consumers, or build them on a system > that has openssl installed but with SSLv2 disabled. Then, of course, you have to make > sure that updated consumers get pushed out and updated by users *before* you push > out a "fixed" openssl. And if end users have built applications, then they're up the creek > without a paddle. It's just a mess. OmniOS has decided to be responsible for the absolute minimum number of "consumers" so it is not in a position to correct the consumers. In contrast, Red Hat Linux provides a huge set of applications and so it can re-issue all those applications built against the new library. Considering all sources of harm, it is likely safest for OmniOS to wait for the 1.1.0 release, and preserve the existing library (with SSLv2 functions as they appear in 1.0.2g or 1.0.1s) across upgrades. Then warn consumers to rebuild their applications. This security problem primarily impacts SSL servers rather than clients. Only a subset of OpenSSL consumers act as servers. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From guorong.koh at gmail.com Wed Mar 2 23:47:18 2016 From: guorong.koh at gmail.com (Guo-Rong Koh) Date: Thu, 03 Mar 2016 10:17:18 +1030 Subject: [OmniOS-discuss] poweradm on OmniOs In-Reply-To: References: Message-ID: <1456962438.11089.22.camel@gmail.com> On Wed, 2016-03-02 at 13:37 +0300, Miami Kelvin wrote: > Hello! > > I am new to OmniOS, I am learning few Omni admin commands using > OPenSolaris Bible and other books as information source. > I am having a problem running the poweradm command on Omni even after > enabling svc:/system/power? > it return -bash: poweradm command not found > What could be the problem I think poweradm only exists in Oracle Solaris 11. Look into /etc/power.conf and /usr/sbin/pmconfig for OmniOS. regards, Guo-Rong -------------- next part -------------- An HTML attachment was scrubbed... URL: From ergi.thanasko at avsquad.com Thu Mar 3 00:06:13 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Thu, 3 Mar 2016 00:06:13 +0000 Subject: [OmniOS-discuss] floadm or dtrace for volume traffic control Message-ID: Hi guys, Is there any tools like flowadm or drace that can control bandwidth usage on a per volume basis instead of host ip. Also flowadm will use both in/out summary to limit bandwidth. We are looking for something deeper that we can seperate control on incoming or outgoing traffic or even at a Volume level . Please let me know et From fabio at fabiorabelo.wiki.br Thu Mar 3 14:00:05 2016 From: fabio at fabiorabelo.wiki.br (=?UTF-8?Q?F=C3=A1bio_Rabelo?=) Date: Thu, 3 Mar 2016 11:00:05 -0300 Subject: [OmniOS-discuss] LSI3108 Message-ID: Hi to all I just did a try in google to find if recent LSI3108 are supported in OmniOS, but did not find anuthing .. Someone knows the answer ? Recent SAS 12 GB/s LSI 3008 and 3108 works with OmniOS ? F?bio Rabelo -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at dojcak.sk Thu Mar 3 14:17:05 2016 From: martin at dojcak.sk (Martin Dojcak) Date: Thu, 3 Mar 2016 15:17:05 +0100 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: References: Message-ID: <56D84761.1010808@dojcak.sk> Hi, look at illumos Hardware Compatibility List: http://illumos.org/hcl/ there is support for LSI 3008 and 3108 hba MD On 03/03/2016 03:00 PM, F?bio Rabelo wrote: > Hi to all > > I just did a try in google to find if recent LSI3108 are supported in OmniOS, but did not > find anuthing .. > > Someone knows the answer ? > > Recent SAS 12 GB/s LSI 3008 and 3108 works with OmniOS ? > > > F?bio Rabelo > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Martin Doj??k pgp: http://pgp.dojcak.sk/ From danmcd at omniti.com Thu Mar 3 14:43:54 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 3 Mar 2016 09:43:54 -0500 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: <56D84761.1010808@dojcak.sk> References: <56D84761.1010808@dojcak.sk> Message-ID: <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> > On Mar 3, 2016, at 9:17 AM, Martin Dojcak wrote: > > Hi, > > look at illumos Hardware Compatibility List: > > http://illumos.org/hcl/ > > there is support for LSI 3008 and 3108 hba Note that the 3108 is HW-RAID and uses the mr_sas(7D) driver, while 3008 (with IT firmware) is a simpler HBA and uses the mpt_sas(7D) driver. Dan From fabio at fabiorabelo.wiki.br Thu Mar 3 15:57:31 2016 From: fabio at fabiorabelo.wiki.br (=?UTF-8?Q?F=C3=A1bio_Rabelo?=) Date: Thu, 3 Mar 2016 12:57:31 -0300 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> Message-ID: OK, reading the manual for the controller ... I do not found anything like "passthru" What I did found was a fuction called "expose Hard Disk to Host" ... And, looks like is is what I need .... Coming back when I have the card in my hands ... Thanks again .... F?bio Rabelo 2016-03-03 12:31 GMT-03:00 F?bio Rabelo : > OK .. > > So, I have to try it out at my own chance and risk ... > > Not today, but soon ... > > Thanks > > > F?bio Rabelo > > 2016-03-03 12:25 GMT-03:00 Dan McDonald : > >> >> > On Mar 3, 2016, at 10:21 AM, F?bio Rabelo >> wrote: >> > >> > >> > If 3108 are RAID, I still can configure my "tipical" RAID Z3 using >> each Hard Disk individualy ? >> > >> > Or OmniOS will only "see" the RAID array ? >> >> You'll have to use the MegaRAID BIOS crud to configure it to not HW-RAID >> the drives. Some of these let you have "passthru" which presents the raw >> SCSI device up to the OS. This is ideal. Older ones (e.g. 2108) would >> only let you create HW-RAID groups of one disk. This adds a layer of >> HW-RAID crap even though it's still a single drive. >> >> I know the 2208 would let you passthru. I've never tried a 3108 board, >> but I hope it's sane like the 2208. >> >> Dan >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From omnios at citrus-it.net Thu Mar 3 16:06:53 2016 From: omnios at citrus-it.net (Andy Fiddaman) Date: Thu, 3 Mar 2016 16:06:53 +0000 (UTC) Subject: [OmniOS-discuss] LSI3108 In-Reply-To: <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> Message-ID: On Thu, 3 Mar 2016, Dan McDonald wrote: ; ; > On Mar 3, 2016, at 9:17 AM, Martin Dojcak wrote: ; > ; > Hi, ; > ; > look at illumos Hardware Compatibility List: ; > ; > http://illumos.org/hcl/ ; > ; > there is support for LSI 3008 and 3108 hba ; ; Note that the 3108 is HW-RAID and uses the mr_sas(7D) driver, while 3008 (with IT firmware) is a simpler HBA and uses the mpt_sas(7D) driver. We are running some Dell R730s with PERC H730s (using LSI 3108) in pass-through mode with the mr_sas driver. No problems here but I'd still recommend choosing something that works with mpt_sas as that has been better maintained and is well tried and tested. reaper# (132) megacli -AdpAllInfo -a0 ... Versions ================ Product Name : PERC H730 Mini Serial No : 4CF00C5 FW Package Build: 25.2.2-0004 ... Direct PD Mapping : Yes dmesg: mr_sas0: 0x1000:0x5d 0x1028:0x1f49, irq:15 drv-ver:6.503.00.00ILLUMOS mr_sas0: TBOLT device detected mr_sas0: msi_enable = 1 mr_sas0: enable_fp = 1 mr_sas0: ctio_enable = 1 mr_sas0 PCI Express-device: pci1028,1f49 at 0, mr_sas0 mr_sas0 is /pci at 0,0/pci8086,2f02 at 1/pci1028,1f49 at 0 scsi: [ID 583861 kern.info] sd0 at mr_sas0: target 0 lun 1 mr_sas0: Phys. device found: tgt 0 dtype 0: SEAGATE ST300MP0005 scsi: [ID 583861 kern.info] sd1 at mr_sas0: target 1 lun 1 mr_sas0: Phys. device found: tgt 1 dtype 0: SEAGATE ST300MP0005 ... -- Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ Registered in England and Wales | Company number 4899123 From fabio at fabiorabelo.wiki.br Thu Mar 3 18:46:15 2016 From: fabio at fabiorabelo.wiki.br (=?UTF-8?Q?F=C3=A1bio_Rabelo?=) Date: Thu, 3 Mar 2016 15:46:15 -0300 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> Message-ID: I am a long time user of the IBM 1015 .. Problem : I have a customer who wants to build a pure SSD Storage with Sandisk 12 Gb/s SAS interface ! Is will run several SQL VMs in it to replace an aging SUN Server . Capicce ?? I do not have a choice to use 6 Gb/s HBAs ... Thanks ... F?bio Rabelo 2016-03-03 13:06 GMT-03:00 Andy Fiddaman : > > On Thu, 3 Mar 2016, Dan McDonald wrote: > > ; > ; > On Mar 3, 2016, at 9:17 AM, Martin Dojcak wrote: > ; > > ; > Hi, > ; > > ; > look at illumos Hardware Compatibility List: > ; > > ; > http://illumos.org/hcl/ > ; > > ; > there is support for LSI 3008 and 3108 hba > ; > ; Note that the 3108 is HW-RAID and uses the mr_sas(7D) driver, while 3008 (with IT firmware) is a simpler HBA and uses the mpt_sas(7D) driver. > > We are running some Dell R730s with PERC H730s (using LSI 3108) in > pass-through mode with the mr_sas driver. No problems here but I'd still > recommend choosing something that works with mpt_sas as that has been > better maintained and is well tried and tested. > > reaper# (132) megacli -AdpAllInfo -a0 > ... > Versions > ================ > Product Name : PERC H730 Mini > Serial No : 4CF00C5 > FW Package Build: 25.2.2-0004 > ... > Direct PD Mapping : Yes > > dmesg: > > mr_sas0: 0x1000:0x5d 0x1028:0x1f49, irq:15 drv-ver:6.503.00.00ILLUMOS > mr_sas0: TBOLT device detected > mr_sas0: msi_enable = 1 > mr_sas0: enable_fp = 1 > mr_sas0: ctio_enable = 1 > mr_sas0 PCI Express-device: pci1028,1f49 at 0, mr_sas0 > mr_sas0 is /pci at 0,0/pci8086,2f02 at 1/pci1028,1f49 at 0 > scsi: [ID 583861 kern.info] sd0 at mr_sas0: target 0 lun 1 > mr_sas0: Phys. device found: tgt 0 dtype 0: SEAGATE ST300MP0005 > scsi: [ID 583861 kern.info] sd1 at mr_sas0: target 1 lun 1 > mr_sas0: Phys. device found: tgt 1 dtype 0: SEAGATE ST300MP0005 > ... > > > -- > Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk > Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ > Registered in England and Wales | Company number 4899123 > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Thu Mar 3 18:53:05 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 3 Mar 2016 19:53:05 +0100 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> Message-ID: <20160303195305.4473d80c@sleipner.datanom.net> On Thu, 3 Mar 2016 15:46:15 -0300 F?bio Rabelo wrote: > > I do not have a choice to use 6 Gb/s HBAs ... > According to specs 3008 also provides 12 Gb/s SAS? http://www.avagotech.com/products/server-storage/io-controllers/sas-3008 -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: It's better to be wanted for murder that not to be wanted at all. -- Marty Winch -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Thu Mar 3 18:53:40 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 3 Mar 2016 13:53:40 -0500 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> Message-ID: <4C9DDF4C-9E05-4027-A6A7-56658D8F5E6E@omniti.com> > On Mar 3, 2016, at 1:46 PM, F?bio Rabelo wrote: > > > I have a customer who wants to build a pure SSD Storage with Sandisk > 12 Gb/s SAS interface ! Get an LSI 3008-based card. The 9300, for example. Dan From rjahnel at ellipseinc.com Thu Mar 3 18:56:36 2016 From: rjahnel at ellipseinc.com (Richard Jahnel) Date: Thu, 3 Mar 2016 18:56:36 +0000 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: <4C9DDF4C-9E05-4027-A6A7-56658D8F5E6E@omniti.com> References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> <4C9DDF4C-9E05-4027-A6A7-56658D8F5E6E@omniti.com> Message-ID: <65DC5816D4BEE043885A89FD54E273FC6CF8DD35@MAIL101.Ellipseinc.com> Speaking of which, does anyone have a diskmap.py that works with the sas3 lsi cards? Richard Jahnel Network Engineer On-Site.com | Ellipse Design 866-266-7483 ext. 4408 Direct: 669-800-6270 -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Dan McDonald Sent: Thursday, March 03, 2016 12:54 PM To: F?bio Rabelo ; Dan McDonald Cc: omnios-discuss Subject: Re: [OmniOS-discuss] LSI3108 > On Mar 3, 2016, at 1:46 PM, F?bio Rabelo wrote: > > > I have a customer who wants to build a pure SSD Storage with Sandisk > 12 Gb/s SAS interface ! Get an LSI 3008-based card. The 9300, for example. Dan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From lotheac at iki.fi Fri Mar 4 12:17:48 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Fri, 4 Mar 2016 14:17:48 +0200 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: <20151008075920.GA10733@gutsman.lotheac.fi> References: <20151008075920.GA10733@gutsman.lotheac.fi> Message-ID: <20160304121748.GE3623@kekkonen.niksula.hut.fi> On Thu, Oct 08 2015 10:59:20 +0300, Lauri Tirkkonen wrote: > Assertion failed: ilen <= SPA_MAXBLOCKSIZE, file ../common/libzfs_sendrecv.c, line 1706, function recv_read I have a repro of this now: a 17MB incremental replication stream, which coredumps 'zfs recv' (even when used with -n), on OmniOS r151014 and bloody alike. The filesystem in question is large, contains sensitive data, and has 162 child filesystems. The snapshots between which the incremental stream was generated were taken recursively. mdb has this to say: > $c libc.so.1`_lwp_kill+7(1, 6, 80467b8, fee91cdd, fef63000, 8046810) libc.so.1`raise+0x22(6, 0, 80467f8, fee6edb9) libc.so.1`abort+0xf3(8046810, 8046810, 6c, fed732cc, 65737341, 6f697472) libc.so.1`_assert(fedabf16, fedabefa, 7a3, feda93ad) libzfs.so.1`recv_read+0x3d(80a4548, 0, 80a7008, 1140fbc, 0, 8047390) libzfs.so.1`recv_read_nvlist+0x4a(80a4548, 0, 1140fbc, 804722c, 0, 8047390) libzfs.so.1`zfs_receive_package+0x100(80a4548, 0, 8047d9c, 8047bec, 80478e8, 8047390) libzfs.so.1`zfs_receive_impl+0x527(80a4548, 8047d9c, 0, 8047bec, 0, 0) libzfs.so.1`zfs_receive+0xc3(80a4548, 8047d9c, 80a1f88, 8047bec, 0, 0) zfs_do_receive+0x3dd(3, 8047cb0, 80768a0, 801, 0, 3) main+0x22c(fee10180, fef6d6a8, 8047ca0, 80557f7, 4, 8047cac) _start+0x83(4, 8047d90, 8047d94, 8047d99, 8047d9c, 0) and zstreamdump reports: SUMMARY: Total DRR_BEGIN records = 164 Total DRR_END records = 165 Total DRR_OBJECT records = 0 Total DRR_FREEOBJECTS records = 0 Total DRR_WRITE records = 0 Total DRR_WRITE_BYREF records = 0 Total DRR_WRITE_EMBEDDED records = 0 Total DRR_FREE records = 0 Total DRR_SPILL records = 0 Total records = 329 Total write size = 0 (0x0) Total stream length = 18194612 (0x115a0b4) Because there is sensitive data involved, I'm reluctant to share either the core or the stream, but would appreciate any cluebats. Thanks. -- Lauri Tirkkonen | lotheac @ IRCnet From lotheac at iki.fi Fri Mar 4 15:43:17 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Fri, 4 Mar 2016 17:43:17 +0200 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: <20160304121748.GE3623@kekkonen.niksula.hut.fi> References: <20151008075920.GA10733@gutsman.lotheac.fi> <20160304121748.GE3623@kekkonen.niksula.hut.fi> Message-ID: <20160304154317.GG3623@kekkonen.niksula.hut.fi> On Fri, Mar 04 2016 14:17:48 +0200, Lauri Tirkkonen wrote: > On Thu, Oct 08 2015 10:59:20 +0300, Lauri Tirkkonen wrote: > > Assertion failed: ilen <= SPA_MAXBLOCKSIZE, file ../common/libzfs_sendrecv.c, line 1706, function recv_read > > I have a repro of this now: a 17MB incremental replication stream, which > coredumps 'zfs recv' (even when used with -n), on OmniOS r151014 and > bloody alike. The filesystem in question is large, contains sensitive > data, and has 162 child filesystems. The snapshots between which the > incremental stream was generated were taken recursively. I managed to tracked this down, and the root cause is a very large amount of snapshots (169k) on the dataset from which the replication stream was generated. Looking at zstreamdump, it seems like references to all existing snapshots for the source dataset and its children are in there (even though I generated the incremental stream between two snapshots taken right after one another), and that caused a dmu_replay_record_t in the stream to be larger than what recv expected: > 80478e8::print dmu_replay_record_t { drr_type = 0 (DRR_BEGIN) drr_payloadlen = 0x1140fbc (SPA_MAXBLOCKSIZE is 1<<24, which is less than that) I fixed snapshot pruning for the source dataset and it's working again, but this still seems like a bug - I'll try to get steps to repro and report to illumos later. -- Lauri Tirkkonen | lotheac @ IRCnet From danmcd at omniti.com Fri Mar 4 15:54:09 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 4 Mar 2016 10:54:09 -0500 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: <20160304121748.GE3623@kekkonen.niksula.hut.fi> References: <20151008075920.GA10733@gutsman.lotheac.fi> <20160304121748.GE3623@kekkonen.niksula.hut.fi> Message-ID: <848E86A7-2CB5-42BA-990F-1318832F3344@omniti.com> > On Mar 4, 2016, at 7:17 AM, Lauri Tirkkonen wrote: > > Because there is sensitive data involved, I'm reluctant to share either the > core or the stream, but would appreciate any cluebats. Thanks. The length in question arrives via a drr_payloadlen. See zfs_receive_package(), line 2589 on illumos-gate: 2585 /* 2586 * Read in the nvlist from the stream. 2587 */ 2588 if (drr->drr_payloadlen != 0) { 2589 error = recv_read_nvlist(hdl, fd, drr->drr_payloadlen, 2590 &stream_nv, flags->byteswap, zc); 2591 if (error) { 2592 error = zfs_error(hdl, EZFS_BADSTREAM, errbuf); 2593 goto out; 2594 } 2595 } Can you zstreamdump and see if any drr_payloadlen looks iffy? Also, are you sending this stream from a post-illumos-5027 (Large Block Size) box to a pre-illumos-5027 receiver? A quick disassembly of recv_read() can tell you what it thinks SPA_MAXBLOCKSIZE is. Otherwise, for some reason the drr_payloadlen of one of these payloads is WAY OFF for some reason. Hope this helps, Dan From lotheac at iki.fi Fri Mar 4 15:57:32 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Fri, 4 Mar 2016 17:57:32 +0200 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: References: <20151008075920.GA10733@gutsman.lotheac.fi> <20160304121748.GE3623@kekkonen.niksula.hut.fi> <20160304154317.GG3623@kekkonen.niksula.hut.fi> Message-ID: <20160304155732.GH3623@kekkonen.niksula.hut.fi> On Fri, Mar 04 2016 10:55:15 -0500, Dan McDonald wrote: > > > On Mar 4, 2016, at 10:43 AM, Lauri Tirkkonen wrote: > > > > I fixed snapshot pruning for the source dataset and it's working again, > > but this still seems like a bug - I'll try to get steps to repro and > > report to illumos later. > > Damn... two steps ahead of me! > > Make sure you let the open-zfs list know alongside illumos. This is a bug for zfs at lists.illumos.org and developer at open-zfs.org. Is an illumos-gate bug report not enough? -- Lauri Tirkkonen | lotheac @ IRCnet From danmcd at omniti.com Fri Mar 4 15:55:15 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 4 Mar 2016 10:55:15 -0500 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: <20160304154317.GG3623@kekkonen.niksula.hut.fi> References: <20151008075920.GA10733@gutsman.lotheac.fi> <20160304121748.GE3623@kekkonen.niksula.hut.fi> <20160304154317.GG3623@kekkonen.niksula.hut.fi> Message-ID: > On Mar 4, 2016, at 10:43 AM, Lauri Tirkkonen wrote: > > I fixed snapshot pruning for the source dataset and it's working again, > but this still seems like a bug - I'll try to get steps to repro and > report to illumos later. Damn... two steps ahead of me! Make sure you let the open-zfs list know alongside illumos. This is a bug for zfs at lists.illumos.org and developer at open-zfs.org. Thanks! Dan From danmcd at omniti.com Fri Mar 4 16:41:49 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 4 Mar 2016 11:41:49 -0500 Subject: [OmniOS-discuss] zfs recv assertion failed when scrubbing source pool In-Reply-To: <20160304155732.GH3623@kekkonen.niksula.hut.fi> References: <20151008075920.GA10733@gutsman.lotheac.fi> <20160304121748.GE3623@kekkonen.niksula.hut.fi> <20160304154317.GG3623@kekkonen.niksula.hut.fi> <20160304155732.GH3623@kekkonen.niksula.hut.fi> Message-ID: <0B872DDE-5B1F-4225-AA79-A7CFB57CA3E4@omniti.com> > On Mar 4, 2016, at 10:57 AM, Lauri Tirkkonen wrote: > > Is an illumos-gate bug report not enough? Create it, but ping the list. The OpenZFS one, in particular, doesn't necessarily have people tracking the illumos bug list. Dan From stephan.budach at JVM.DE Sun Mar 6 14:44:47 2016 From: stephan.budach at JVM.DE (Stephan Budach) Date: Sun, 6 Mar 2016 15:44:47 +0100 Subject: [OmniOS-discuss] RSF-1/ZFS panics node when offlining one iSCSI storage mirror Message-ID: <56DC425F.1020906@jvm.de> Hi, I have set up a rather simple RSF-1 project, where two RSF-1 nodes connect to two storage heads via iSCSI. I have deployed one network and two disc heatbeats and I was trying all sorts of possible failures, when I noted that one node would panic, if I offlined an iSCSI target on one storage node and thus shutting down one side of a zpool mirror completely. Issueing a zpool status would't return and after a while the host got nuked. I then onlined the target again and waited until the node returned and than removed the local iSCSI initiator on the RSF-1 node instead, which resulted in a degraded, but functional zpool and this time, the node didn't get nuked. What is the difference between these two approaches and can I setup my systems such as that offlining a target doesn't lead to this behaviour? I'd imagine, that a target failure might as well occur as any other sofware fault. Thanks, Stephan From fred.fliu at gmail.com Mon Mar 7 05:30:18 2016 From: fred.fliu at gmail.com (Fred Liu) Date: Mon, 7 Mar 2016 13:30:18 +0800 Subject: [OmniOS-discuss] [zfs] an interesting survey -- the zpool with most disks you have ever built In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> Message-ID: 2016-03-05 0:01 GMT+08:00 Freddie Cash : > On Mar 4, 2016 2:05 AM, "Fred Liu" wrote: > > 2016-03-04 13:47 GMT+08:00 Freddie Cash : > >> > >> Currently, I just use a simple coordinate system. Columns are letters, > rows are numbers. > >> "smartos-discuss at lists.smartos.org" >? > > developer ? > > illumos-developer ? > > omnios-discuss ? > > Discussion list for OpenIndiana ? > > illumos-zfs ? > > "zfs-discuss at list.zfsonlinux.org" ? > > "freebsd-fs at FreeBSD.org" ? > > "zfs-devel at freebsd.org" > > >> Each disk is partitioned using GPT with the first (only) partition > starting at 1 MB and covering the whole disk, and labelled with the > column/row where it is located (disk-a1, disk-g6, disk-p3, etc). > > > > [Fred]: So you manually pull off all the drives one by one to locate > them? > > ?When putting the system together for the first time, I insert each disk > one at a time, wait for it to be detected, partition it, then label it > based on physical location.? Then do the next one. It's just part of the > normal server build process, whether it has 2 drives, 20 drives, or 200 > drives. > > ?We build all our own servers from off-the-shelf parts; we don't buy > anything pre-built from any of the large OEMs.? > [Fred]: Gotcha! > >> The pool is created using the GPT labels, so the label shows in "zpool > list" output. > > > > [Fred]: What will the output look like? > > ?From our smaller backups server, with just 24 drive bays: > > $ zpool status storage > > pool: storage > > state: ONLINE > > status: Some supported features are not enabled on the pool. The pool can > > still be used, but some features are unavailable. > > action: Enable all features using 'zpool upgrade'. Once this is done, > > the pool may no longer be accessible by software that does not support > > the features. See zpool-features(7) for details. > > scan: scrub canceled on Wed Feb 17 12:02:20 2016 > > config: > > > NAME STATE READ WRITE CKSUM > > storage ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > gpt/disk-a1 ONLINE 0 0 0 > > gpt/disk-a2 ONLINE 0 0 0 > > gpt/disk-a3 ONLINE 0 0 0 > > gpt/disk-a4 ONLINE 0 0 0 > > gpt/disk-a5 ONLINE 0 0 0 > > gpt/disk-a6 ONLINE 0 0 0 > > raidz2-1 ONLINE 0 0 0 > > gpt/disk-b1 ONLINE 0 0 0 > > gpt/disk-b2 ONLINE 0 0 0 > > gpt/disk-b3 ONLINE 0 0 0 > > gpt/disk-b4 ONLINE 0 0 0 > > gpt/disk-b5 ONLINE 0 0 0 > > gpt/disk-b6 ONLINE 0 0 0 > > raidz2-2 ONLINE 0 0 0 > > gpt/disk-c1 ONLINE 0 0 0 > > gpt/disk-c2 ONLINE 0 0 0 > > gpt/disk-c3 ONLINE 0 0 0 > > gpt/disk-c4 ONLINE 0 0 0 > > gpt/disk-c5 ONLINE 0 0 0 > > gpt/disk-c6 ONLINE 0 0 0 > > raidz2-3 ONLINE 0 0 0 > > gpt/disk-d1 ONLINE 0 0 0 > > gpt/disk-d2 ONLINE 0 0 0 > > gpt/disk-d3 ONLINE 0 0 0 > > gpt/disk-d4 ONLINE 0 0 0 > > gpt/disk-d5 ONLINE 0 0 0 > > gpt/disk-d6 ONLINE 0 0 0 > > cache > > gpt/cache0 ONLINE 0 0 0 > > gpt/cache1 ONLINE 0 0 0 > > > errors: No known data errors > > The 90-bay systems look the same, just that the letters go all the way to > p (so disk-p1 through disk-p6). And there's one vdev that uses 3 drives > from each chassis (7x 6-disk vdev only uses 42 drives of the 45-bay > chassis, so there's lots of spares if using a single chassis; using two > chassis, there's enough drives to add an extra 6-disk vdev). > [Fred]: It looks like the gpt label shown in "zpool status" only works in FreeBSD/FreeNAS. Are you using FreeBSD/FreeNAS? I can't find the similar possibilities in Illumos/Linux. Thanks, Fred > > *illumos-zfs* | Archives > > | > Modify > > Your Subscription > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Mar 7 14:41:17 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 7 Mar 2016 09:41:17 -0500 Subject: [OmniOS-discuss] RSF-1/ZFS panics node when offlining one iSCSI storage mirror In-Reply-To: <56DC425F.1020906@jvm.de> References: <56DC425F.1020906@jvm.de> Message-ID: <5760FDEB-C954-497A-B434-FB5AEA871146@omniti.com> > On Mar 6, 2016, at 9:44 AM, Stephan Budach wrote: > > when I noted that one node would panic, AS A RULE -- if you have an OmniOS box panic, you should save off the corefile (vmdump.N) and be able to share it with the list. I understand this may be an RSF-1 panic, BUT if it's not, it'd be nice to know. You can upload it to uploads.omniti.com if you wish, just request an upload token. Dan From stephan.budach at JVM.DE Mon Mar 7 17:51:07 2016 From: stephan.budach at JVM.DE (Stephan Budach) Date: Mon, 7 Mar 2016 18:51:07 +0100 Subject: [OmniOS-discuss] RSF-1/ZFS panics node when offlining one iSCSI storage mirror In-Reply-To: <5760FDEB-C954-497A-B434-FB5AEA871146@omniti.com> References: <56DC425F.1020906@jvm.de> <5760FDEB-C954-497A-B434-FB5AEA871146@omniti.com> Message-ID: <56DDBF8B.6000207@jvm.de> Hi Dan, Am 07.03.16 um 15:41 schrieb Dan McDonald: >> On Mar 6, 2016, at 9:44 AM, Stephan Budach wrote: >> >> when I noted that one node would panic, > AS A RULE -- if you have an OmniOS box panic, you should save off the corefile (vmdump.N) and be able to share it with the list. I understand this may be an RSF-1 panic, BUT if it's not, it'd be nice to know. > > You can upload it to uploads.omniti.com if you wish, just request an upload token. > > Dan > thanks - I will keep that in mind and I actually had a core dump available, but since I was testing around, I didn't mean to occupy anyone's time more than absolutely necessary and so I dumoed them. Speaking of that incident, I have lowered the iSCSI connection timeout to 60s, which seems to be the lowest value supported by issueing a iscsiadm modify initiator-node -T conn-login-max=60 and afterwards I used stmfadm offline target on the storage node to cut the target off. This time, the initiator timed out after 60s and that particular zpool changed it's status to degraded without anything happening. I still have to test that under load, but I will probably push that to next weekend. Thanks, Stephan From danmcd at omniti.com Tue Mar 8 02:47:10 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 7 Mar 2016 21:47:10 -0500 Subject: [OmniOS-discuss] Introducing io-lx -- an attempt to port LX Zones to OmniOS Message-ID: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> Hello! 700+ cherrypicked commits later, and at least I haven't obviously regressed lipkg/ipkg zones. :) This github repo: https://github.com/danmcd/io-lx-public/ is the beginning of my attempt to port LX zones over to OmniOS. I KNOW there's a lot more work to be done, but I wanted to make sure this repo is viewable by the public so: 1.) People know about it. 2.) People who are interested in helping can do so. 3.) People can laugh at all of the mismerges and other flub-ups I'm sure I haven't caught yet. :) 4.) I will be context-switching to OmniOS r151018 ramp-up very soon, and want to checkpoint state. I've Bcc:ed the SmartOS (home of LX zones) and the illumos (where I hope this work can be upstreamed at some point) lists. Discussion on io-lx (Illumos-OmniOS-LX) should take place either on github or on the OmniOS mailing list. So far: - I've sidepulled 700+ commits, ending with illumos-joyent's 8443e038ef8eb3ca6a95818d6e90b2a1eb4e9cb6. There are some more illumos-joyent commits I'll need to bring over, but my cherrypicking script and setups are primed & tested, so I *should* be able to make short work of it. - I've smoke-tested a global-only OmniOS boot, and a single-lipkg zone OmniOS boot. After some mismerge-related consternation, I booted the lipkg zone as well. A rudimentary ppriv(1) comparison between root shells in global and lipkg zones was part of the smoke test. - Any files I didn't know where they went I put into the "brand/lx" package. The inotify feature, for example, is in there, and perhaps it shouldn't be. - If you ONU to io-lx, note that to install brand/lx, you will need to UNINSTALL the "illumos-gate" consolidation package first. I don't have changes in omnios-build or any of the other repos (like pkg5) yet. I hope to avoid outside-illumos changes until it's time to figure out the admin model. After I'm comfortable with no ipkg/lipkg regressions, I will need to spend some design time figuring out how LX zones will look on OmniOS. I will not be porting vmadm(1M) over from SmartOS, so I need to think about how LX will fit in with traditional zone tools. I may discover other problems, but until I start the '018 process, and immediately after '018 ships, I will need to ensure no ipkg/lipkg regressions first and foremost. It's not much, but it's a start. Thanks for your time & patience, Dan From miniflowtrader at gmail.com Tue Mar 8 05:42:13 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Tue, 8 Mar 2016 00:42:13 -0500 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings Message-ID: Is it possible that CIFS will ignore TCP buffer settings after a while? I've confirmed my systems max transfer rate using iperf and have tuned my buffers accordingly. For whatever reason CIFS seems to forget these settings after a while as speed drops significantly. Issuing a restart of the service immediately appears to restore the setting as transfer speed becomes normal again. Any ideas why this would happen? -------------- next part -------------- An HTML attachment was scrubbed... URL: From garrett at damore.org Tue Mar 8 05:51:30 2016 From: garrett at damore.org (Garrett D'Amore) Date: Mon, 7 Mar 2016 21:51:30 -0800 Subject: [OmniOS-discuss] [smartos-discuss] Introducing io-lx -- an attempt to port LX Zones to OmniOS In-Reply-To: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> References: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> Message-ID: <9103423F-E8B3-4E60-A84E-06EE84B223B4@damore.org> Congratulations. Hopefully I'll have some time to help you. I'm excited about this effort. Sent from my iPhone > On Mar 7, 2016, at 6:47 PM, Dan McDonald wrote: > > Hello! > > 700+ cherrypicked commits later, and at least I haven't obviously regressed lipkg/ipkg zones. :) > > This github repo: > > https://github.com/danmcd/io-lx-public/ > > is the beginning of my attempt to port LX zones over to OmniOS. I KNOW there's a lot more work to be done, but I wanted to make sure this repo is viewable by the public so: > > 1.) People know about it. > > 2.) People who are interested in helping can do so. > > 3.) People can laugh at all of the mismerges and other flub-ups I'm sure I haven't caught yet. :) > > 4.) I will be context-switching to OmniOS r151018 ramp-up very soon, and want to checkpoint state. > > I've Bcc:ed the SmartOS (home of LX zones) and the illumos (where I hope this work can be upstreamed at some point) lists. Discussion on io-lx (Illumos-OmniOS-LX) should take place either on github or on the OmniOS mailing list. > > So far: > > - I've sidepulled 700+ commits, ending with illumos-joyent's 8443e038ef8eb3ca6a95818d6e90b2a1eb4e9cb6. There are some more illumos-joyent commits I'll need to bring over, but my cherrypicking script and setups are primed & tested, so I *should* be able to make short work of it. > > - I've smoke-tested a global-only OmniOS boot, and a single-lipkg zone OmniOS boot. After some mismerge-related consternation, I booted the lipkg zone as well. A rudimentary ppriv(1) comparison between root shells in global and lipkg zones was part of the smoke test. > > - Any files I didn't know where they went I put into the "brand/lx" package. The inotify feature, for example, is in there, and perhaps it shouldn't be. > > - If you ONU to io-lx, note that to install brand/lx, you will need to UNINSTALL the "illumos-gate" consolidation package first. I don't have changes in omnios-build or any of the other repos (like pkg5) yet. I hope to avoid outside-illumos changes until it's time to figure out the admin model. > > After I'm comfortable with no ipkg/lipkg regressions, I will need to spend some design time figuring out how LX zones will look on OmniOS. I will not be porting vmadm(1M) over from SmartOS, so I need to think about how LX will fit in with traditional zone tools. I may discover other problems, but until I start the '018 process, and immediately after '018 ships, I will need to ensure no ipkg/lipkg regressions first and foremost. > > It's not much, but it's a start. > > Thanks for your time & patience, > Dan > > > > ------------------------------------------- > smartos-discuss > Archives: https://www.listbox.com/member/archive/184463/=now > RSS Feed: https://www.listbox.com/member/archive/rss/184463/22103350-51080293 > Modify Your Subscription: https://www.listbox.com/member/?member_id=22103350&id_secret=22103350-a7c9ea51 > Powered by Listbox: http://www.listbox.com From jimklimov at cos.ru Tue Mar 8 06:37:09 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Tue, 08 Mar 2016 07:37:09 +0100 Subject: [OmniOS-discuss] [developer] Introducing io-lx -- an attempt to port LX Zones to OmniOS In-Reply-To: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> References: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> Message-ID: <990DEB1A-6DED-4AD0-8CD9-FC463A964170@cos.ru> 8 ????? 2016??. 3:47:10 CET, Dan McDonald ?????: >Hello! > >700+ cherrypicked commits later, and at least I haven't obviously >regressed lipkg/ipkg zones. :) > >This github repo: > > https://github.com/danmcd/io-lx-public/ > >is the beginning of my attempt to port LX zones over to OmniOS. I KNOW >there's a lot more work to be done, but I wanted to make sure this repo >is viewable by the public so: > >1.) People know about it. > >2.) People who are interested in helping can do so. > >3.) People can laugh at all of the mismerges and other flub-ups I'm >sure I haven't caught yet. :) > >4.) I will be context-switching to OmniOS r151018 ramp-up very soon, >and want to checkpoint state. > >I've Bcc:ed the SmartOS (home of LX zones) and the illumos (where I >hope this work can be upstreamed at some point) lists. Discussion on >io-lx (Illumos-OmniOS-LX) should take place either on github or on the >OmniOS mailing list. > >So far: > >- I've sidepulled 700+ commits, ending with illumos-joyent's >8443e038ef8eb3ca6a95818d6e90b2a1eb4e9cb6. There are some more >illumos-joyent commits I'll need to bring over, but my cherrypicking >script and setups are primed & tested, so I *should* be able to make >short work of it. > >- I've smoke-tested a global-only OmniOS boot, and a single-lipkg zone >OmniOS boot. After some mismerge-related consternation, I booted the >lipkg zone as well. A rudimentary ppriv(1) comparison between root >shells in global and lipkg zones was part of the smoke test. > >- Any files I didn't know where they went I put into the "brand/lx" >package. The inotify feature, for example, is in there, and perhaps it >shouldn't be. > >- If you ONU to io-lx, note that to install brand/lx, you will need to >UNINSTALL the "illumos-gate" consolidation package first. I don't have >changes in omnios-build or any of the other repos (like pkg5) yet. I >hope to avoid outside-illumos changes until it's time to figure out the >admin model. > >After I'm comfortable with no ipkg/lipkg regressions, I will need to >spend some design time figuring out how LX zones will look on OmniOS. >I will not be porting vmadm(1M) over from SmartOS, so I need to think >about how LX will fit in with traditional zone tools. I may discover >other problems, but until I start the '018 process, and immediately >after '018 ships, I will need to ensure no ipkg/lipkg regressions first >and foremost. > >It's not much, but it's a start. > >Thanks for your time & patience, >Dan > > > >------------------------------------------- >illumos-developer >Archives: https://www.listbox.com/member/archive/182179/=now >RSS Feed: >https://www.listbox.com/member/archive/rss/182179/22416750-c03c8c44 >Modify Your Subscription: >https://www.listbox.com/member/?member_id=22416750&id_secret=22416750-eb7e3ed7 >Powered by Listbox: http://www.listbox.com Cool! Will this include modernized linux distro support? As for the admin model... I tinkered with original lx24 guest zones back in the day, and it looked similar to usual solaris zones in terms of experience, except the console bootup reported some services as missing or disabled. Do you expect to produce something very different nowadays? Thanks for the effort, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Tue Mar 8 06:42:45 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Tue, 08 Mar 2016 07:42:45 +0100 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings In-Reply-To: References: Message-ID: 8 ????? 2016??. 6:42:13 CET, Mini Trader ?????: >Is it possible that CIFS will ignore TCP buffer settings after a while? > >I've confirmed my systems max transfer rate using iperf and have tuned >my >buffers accordingly. For whatever reason CIFS seems to forget these >settings after a while as speed drops significantly. Issuing a restart >of >the service immediately appears to restore the setting as transfer >speed >becomes normal again. > >Any ideas why this would happen? > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss As a random guess from experience with other network stuff - does the speed-drop happen on a running connection or new ones too? Do you have concurrent transfers at this time? Some other subsystems (no idea if this one too) use best speeds for new or recently awakened dormant connections, so short-lived bursts are fast - at expence of long-running active bulk transfers (deemed to be bulk because they run for a long time). HTH, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From josh at sysmgr.org Tue Mar 8 06:54:15 2016 From: josh at sysmgr.org (Joshua M. Clulow) Date: Mon, 7 Mar 2016 22:54:15 -0800 Subject: [OmniOS-discuss] [developer] Introducing io-lx -- an attempt to port LX Zones to OmniOS In-Reply-To: <990DEB1A-6DED-4AD0-8CD9-FC463A964170@cos.ru> References: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> <990DEB1A-6DED-4AD0-8CD9-FC463A964170@cos.ru> Message-ID: On 7 March 2016 at 22:37, Jim Klimov wrote: > Cool! Will this include modernized linux distro support? At Joyent, we have pre-built zone images (which we ship as "zfs send" streams) for at least Ubuntu 14.04 and 15.04, Centos 6 and 7, Debian 7 and 8, and Alpine 3. You could arguably distribute the same files in a tar archive which you would extract into a zone root. From what I remember of the old days, a seed archive was the way to bootstrap an LX zone. The LX brand software has been substantially enhanced since the Sun EOL of the 2.4-era bits; e.g., support for modern system calls and other Linux facilities, 64-bit support, etc. Cheers. -- Joshua M. Clulow UNIX Admin/Developer http://blog.sysmgr.org From jimklimov at cos.ru Tue Mar 8 09:34:27 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Tue, 08 Mar 2016 10:34:27 +0100 Subject: [OmniOS-discuss] [developer] Introducing io-lx -- an attempt to port LX Zones to OmniOS In-Reply-To: References: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> <990DEB1A-6DED-4AD0-8CD9-FC463A964170@cos.ru> Message-ID: <905BE4EC-5EB2-4AB5-B291-404F485AA496@cos.ru> 8 ????? 2016??. 7:54:15 CET, "Joshua M. Clulow" ?????: >On 7 March 2016 at 22:37, Jim Klimov wrote: >> Cool! Will this include modernized linux distro support? > >At Joyent, we have pre-built zone images (which we ship as "zfs send" >streams) for at least Ubuntu 14.04 and 15.04, Centos 6 and 7, Debian 7 >and 8, and Alpine 3. You could arguably distribute the same files in >a tar archive which you would extract into a zone root. From what I >remember of the old days, a seed archive was the way to bootstrap an >LX zone. > >The LX brand software has been substantially enhanced since the Sun >EOL of the 2.4-era bits; e.g., support for modern system calls and >other Linux facilities, 64-bit support, etc. > > >Cheers. FWIW and IIRC, there were brand scripts for different linux distros and versions that could mix installation (packaged, tarhalled, etc) and configuration (console, net, etc). Jim -- Typos courtesy of K-9 Mail on my Samsung Android From svavar at pipar-tbwa.is Tue Mar 8 11:17:33 2016 From: svavar at pipar-tbwa.is (=?iso-8859-2?q?Svavar_=D6rn_Eysteinsson?=) Date: Tue, 8 Mar 2016 11:17:33 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard Message-ID: Hi all. I am having real trouble booting up installation of the newest Omnios on my Intel/Supermicro Server. The specs are : Intel S5520SC Motherboard 2x Intel Xeon E5520 @ 2.27Ghz Supermicro SC933 Chassis with Supermicro BPN-SATA-933 Backplane. Only one hard disk at the moment as I'm testing things out. The Grub bootup doesn't go any further after CPU initialization or USB controller initialization. I have tried the following things in the BIOS : 1. Disable the USB2 Controller and boot from CD Installation (with CD-ROM connected to ACHI on-board Sata controller, as the HD disk also) 2. Enable the USB2 Controller and boot from a USB2 Memory stick with Omnios Installation. 3. Disable the Intel Hyperthreading. 4. Disable the onboard 1394 controller Here are 4 different screenshots of the monitor where the installation boot just hangs. https://dl.dropboxusercontent.com/u/16134/omnios_intel_S5520SC_trouble1.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_S5520SC_trouble2.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_S5520SC_trouble3.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_S5520SC_trouble4.JPG My question is, does anyone know what would cause this hangup? Are there any other boot options I can try or other BIOS related settings ? Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? Thanks in advance people. Best regards, Svavar - Reykjavik - Iceland From wonko at 4amlunch.net Tue Mar 8 12:31:35 2016 From: wonko at 4amlunch.net (wonko at 4amlunch.net) Date: Tue, 8 Mar 2016 07:31:35 -0500 Subject: [OmniOS-discuss] Introducing io-lx -- an attempt to port LX Zones to OmniOS In-Reply-To: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> References: <49C794E2-9B2F-4D6F-9240-13A946651243@omniti.com> Message-ID: YOU ARE MY HERO I wish you the best of luck and will do (what little) I can to help. I've wanted this for a while. -brian > On Mar 7, 2016, at 21:47, Dan McDonald wrote: > > Hello! > > 700+ cherrypicked commits later, and at least I haven't obviously regressed lipkg/ipkg zones. :) > > This github repo: > > https://github.com/danmcd/io-lx-public/ > > is the beginning of my attempt to port LX zones over to OmniOS. I KNOW there's a lot more work to be done, but I wanted to make sure this repo is viewable by the public so: > > 1.) People know about it. > > 2.) People who are interested in helping can do so. > > 3.) People can laugh at all of the mismerges and other flub-ups I'm sure I haven't caught yet. :) > > 4.) I will be context-switching to OmniOS r151018 ramp-up very soon, and want to checkpoint state. > > I've Bcc:ed the SmartOS (home of LX zones) and the illumos (where I hope this work can be upstreamed at some point) lists. Discussion on io-lx (Illumos-OmniOS-LX) should take place either on github or on the OmniOS mailing list. > > So far: > > - I've sidepulled 700+ commits, ending with illumos-joyent's 8443e038ef8eb3ca6a95818d6e90b2a1eb4e9cb6. There are some more illumos-joyent commits I'll need to bring over, but my cherrypicking script and setups are primed & tested, so I *should* be able to make short work of it. > > - I've smoke-tested a global-only OmniOS boot, and a single-lipkg zone OmniOS boot. After some mismerge-related consternation, I booted the lipkg zone as well. A rudimentary ppriv(1) comparison between root shells in global and lipkg zones was part of the smoke test. > > - Any files I didn't know where they went I put into the "brand/lx" package. The inotify feature, for example, is in there, and perhaps it shouldn't be. > > - If you ONU to io-lx, note that to install brand/lx, you will need to UNINSTALL the "illumos-gate" consolidation package first. I don't have changes in omnios-build or any of the other repos (like pkg5) yet. I hope to avoid outside-illumos changes until it's time to figure out the admin model. > > After I'm comfortable with no ipkg/lipkg regressions, I will need to spend some design time figuring out how LX zones will look on OmniOS. I will not be porting vmadm(1M) over from SmartOS, so I need to think about how LX will fit in with traditional zone tools. I may discover other problems, but until I start the '018 process, and immediately after '018 ships, I will need to ensure no ipkg/lipkg regressions first and foremost. > > It's not much, but it's a start. > > Thanks for your time & patience, > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From sford123 at ibbr.umd.edu Tue Mar 8 14:30:58 2016 From: sford123 at ibbr.umd.edu (Steven Ford) Date: Tue, 8 Mar 2016 09:30:58 -0500 Subject: [OmniOS-discuss] Where is my ZFS SMB share? (r151016) Message-ID: Hello, I recently reinstalled our backup data server with OmniOS r151016 from OpenIndiana. It was a completely new installation with a different system drive. I was able to import my previous pool and get replication working again. However, it seems no matter what I do, I cannot get the new installation to export the file system though SMB. My server fault post has all of the details: http://serverfault.com/questions/762160/where-is-my-zfs-smb-share Any insight would be greatly appreciated. Thanks, Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Mar 8 15:13:50 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 8 Mar 2016 10:13:50 -0500 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: References: Message-ID: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> > On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson wrote: > > Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? Disable C-states, that's general illumos advice (regardless of distro). Dan From danmcd at omniti.com Tue Mar 8 15:17:19 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 8 Mar 2016 10:17:19 -0500 Subject: [OmniOS-discuss] Where is my ZFS SMB share? (r151016) In-Reply-To: References: Message-ID: > On Mar 8, 2016, at 9:30 AM, Steven Ford wrote: > > Any insight would be greatly appreciated. > Your noticing Kerberos issues may be the key. I'd concentrate on those. The SMB/CIFS wizard for illumos doesn't hang out here much, if at all. You may wish to ask the illumos lists this question. Dan From sford123 at ibbr.umd.edu Tue Mar 8 15:20:36 2016 From: sford123 at ibbr.umd.edu (Steven Ford) Date: Tue, 8 Mar 2016 10:20:36 -0500 Subject: [OmniOS-discuss] Where is my ZFS SMB share? (r151016) In-Reply-To: References: Message-ID: Dan, I will give that a try. Thanks, Steve On Tue, Mar 8, 2016 at 10:17 AM, Dan McDonald wrote: > > > On Mar 8, 2016, at 9:30 AM, Steven Ford wrote: > > > > Any insight would be greatly appreciated. > > > > Your noticing Kerberos issues may be the key. I'd concentrate on those. > > The SMB/CIFS wizard for illumos doesn't hang out here much, if at all. > You may wish to ask the illumos lists this question. > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Tue Mar 8 18:51:32 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Tue, 8 Mar 2016 13:51:32 -0500 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings In-Reply-To: References: Message-ID: Simple example. 1 Server 1 client. Restart service everything is fast. A few hours later from same client (nothing happening concurrently) speed is slow. Restart service again, speed is fast. Its like CIFS starts off fast than somehow for whatever reason if it is not used, the connection for my CIFS drives to the server becomes slow. Also this only happens when the client is downloading. Not when uploading to the server that is always fast. On Tue, Mar 8, 2016 at 1:42 AM, Jim Klimov wrote: > 8 ????? 2016 ?. 6:42:13 CET, Mini Trader ?????: > >Is it possible that CIFS will ignore TCP buffer settings after a while? > > > >I've confirmed my systems max transfer rate using iperf and have tuned > >my > >buffers accordingly. For whatever reason CIFS seems to forget these > >settings after a while as speed drops significantly. Issuing a restart > >of > >the service immediately appears to restore the setting as transfer > >speed > >becomes normal again. > > > >Any ideas why this would happen? > > > > > >------------------------------------------------------------------------ > > > >_______________________________________________ > >OmniOS-discuss mailing list > >OmniOS-discuss at lists.omniti.com > >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > As a random guess from experience with other network stuff - does the > speed-drop happen on a running connection or new ones too? Do you have > concurrent transfers at this time? > > Some other subsystems (no idea if this one too) use best speeds for new or > recently awakened dormant connections, so short-lived bursts are fast - at > expence of long-running active bulk transfers (deemed to be bulk because > they run for a long time). > > HTH, Jim > -- > Typos courtesy of K-9 Mail on my Samsung Android > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Wed Mar 9 00:56:39 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Tue, 8 Mar 2016 19:56:39 -0500 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings In-Reply-To: References: Message-ID: If it helps. This doesn't happen on NFS from the exact same client. How do I file a bug? On Tue, Mar 8, 2016 at 1:51 PM, Mini Trader wrote: > Simple example. > > 1 Server 1 client. > > Restart service everything is fast. A few hours later from same client > (nothing happening concurrently) speed is slow. Restart service again, > speed is fast. > > Its like CIFS starts off fast than somehow for whatever reason if it is > not used, the connection for my CIFS drives to the server becomes slow. > Also this only happens when the client is downloading. Not when uploading > to the server that is always fast. > > On Tue, Mar 8, 2016 at 1:42 AM, Jim Klimov wrote: > >> 8 ????? 2016 ?. 6:42:13 CET, Mini Trader >> ?????: >> >Is it possible that CIFS will ignore TCP buffer settings after a while? >> > >> >I've confirmed my systems max transfer rate using iperf and have tuned >> >my >> >buffers accordingly. For whatever reason CIFS seems to forget these >> >settings after a while as speed drops significantly. Issuing a restart >> >of >> >the service immediately appears to restore the setting as transfer >> >speed >> >becomes normal again. >> > >> >Any ideas why this would happen? >> > >> > >> >------------------------------------------------------------------------ >> > >> >_______________________________________________ >> >OmniOS-discuss mailing list >> >OmniOS-discuss at lists.omniti.com >> >http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> As a random guess from experience with other network stuff - does the >> speed-drop happen on a running connection or new ones too? Do you have >> concurrent transfers at this time? >> >> Some other subsystems (no idea if this one too) use best speeds for new >> or recently awakened dormant connections, so short-lived bursts are fast - >> at expence of long-running active bulk transfers (deemed to be bulk because >> they run for a long time). >> >> HTH, Jim >> -- >> Typos courtesy of K-9 Mail on my Samsung Android >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Wed Mar 9 01:20:05 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Tue, 8 Mar 2016 20:20:05 -0500 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings In-Reply-To: References: Message-ID: Running the following dtrace. #!/usr/sbin/dtrace -s #pragma D option quiet tcp:::send / (args[4]->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / { @unacked["unacked(bytes)", args[2]->ip_daddr, args[4]->tcp_sport] = quantize(args[3]->tcps_snxt - args[3]->tcps_suna); } tcp:::receive / (args[4]->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / { @swnd["SWND(bytes)", args[2]->ip_saddr, args[4]->tcp_dport] = quantize((args[4]->tcp_window)*(1 << args[3]->tcps_snd_ws)); } Is showing that the windows sizes are not going above 64k when things are not working properly. On Tue, Mar 8, 2016 at 7:56 PM, Mini Trader wrote: > If it helps. This doesn't happen on NFS from the exact same client. How > do I file a bug? > > On Tue, Mar 8, 2016 at 1:51 PM, Mini Trader > wrote: > >> Simple example. >> >> 1 Server 1 client. >> >> Restart service everything is fast. A few hours later from same client >> (nothing happening concurrently) speed is slow. Restart service again, >> speed is fast. >> >> Its like CIFS starts off fast than somehow for whatever reason if it is >> not used, the connection for my CIFS drives to the server becomes slow. >> Also this only happens when the client is downloading. Not when uploading >> to the server that is always fast. >> >> On Tue, Mar 8, 2016 at 1:42 AM, Jim Klimov wrote: >> >>> 8 ????? 2016 ?. 6:42:13 CET, Mini Trader >>> ?????: >>> >Is it possible that CIFS will ignore TCP buffer settings after a while? >>> > >>> >I've confirmed my systems max transfer rate using iperf and have tuned >>> >my >>> >buffers accordingly. For whatever reason CIFS seems to forget these >>> >settings after a while as speed drops significantly. Issuing a restart >>> >of >>> >the service immediately appears to restore the setting as transfer >>> >speed >>> >becomes normal again. >>> > >>> >Any ideas why this would happen? >>> > >>> > >>> >------------------------------------------------------------------------ >>> > >>> >_______________________________________________ >>> >OmniOS-discuss mailing list >>> >OmniOS-discuss at lists.omniti.com >>> >http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >>> As a random guess from experience with other network stuff - does the >>> speed-drop happen on a running connection or new ones too? Do you have >>> concurrent transfers at this time? >>> >>> Some other subsystems (no idea if this one too) use best speeds for new >>> or recently awakened dormant connections, so short-lived bursts are fast - >>> at expence of long-running active bulk transfers (deemed to be bulk because >>> they run for a long time). >>> >>> HTH, Jim >>> -- >>> Typos courtesy of K-9 Mail on my Samsung Android >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From miniflowtrader at gmail.com Wed Mar 9 01:33:47 2016 From: miniflowtrader at gmail.com (Mini Trader) Date: Tue, 8 Mar 2016 20:33:47 -0500 Subject: [OmniOS-discuss] CIFS ignores TCP Buffer Settings In-Reply-To: References: Message-ID: Bad root at storage1:/root# dtrace -s tcp_tput.d ^C unacked(bytes) 10.250.0.3 2049 value ------------- Distribution ------------- count 32 | 0 64 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 11 128 | 0 unacked(bytes) 10.250.0.2 2049 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@ 63 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 0 64 |@ 5 128 |@@@@@@@@@@@@@@@@@@@@@@@ 195 256 |@@@@@@ 54 512 |@@ 13 1024 |@ 5 2048 | 0 unacked(bytes) 10.255.0.55 445 value ------------- Distribution ------------- count -1 | 0 0 | 7 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 10 64 | 48 128 | 9 256 | 0 512 | 0 1024 | 1 2048 | 0 4096 | 0 8192 | 0 16384 | 2 32768 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 46626 65536 | 6 131072 | 0 SWND(bytes) 10.255.0.55 22 value ------------- Distribution ------------- count 16384 | 0 32768 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 65536 | 0 SWND(bytes) 10.250.0.3 2049 value ------------- Distribution ------------- count 32768 | 0 65536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 17 131072 | 0 SWND(bytes) 10.250.0.2 2049 value ------------- Distribution ------------- count 131072 | 0 262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 369 524288 | 0 SWND(bytes) 10.255.0.55 445 value ------------- Distribution ------------- count 16384 | 0 32768 | 367 65536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 49052 131072 | 0 Good root at storage1:/root# svcadm restart smb/server root at storage1:/root# dtrace -s tcp_tput.d ^C unacked(bytes) 10.255.0.55 22 value ------------- Distribution ------------- count 32 | 0 64 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 128 | 0 unacked(bytes) 10.250.0.3 2049 value ------------- Distribution ------------- count 32 | 0 64 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 11 128 | 0 unacked(bytes) 10.250.0.2 2049 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@ 23 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 0 64 |@ 4 128 |@@@@@@@@@@@@@@@@@@@@@@@@@ 84 256 |@@@@ 12 512 |@ 4 1024 |@@ 6 2048 | 0 unacked(bytes) 10.255.0.55 445 value ------------- Distribution ------------- count -1 | 0 0 | 9 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 10 64 | 50 128 | 9 256 | 0 512 | 0 1024 | 1 2048 | 0 4096 | 0 8192 | 0 16384 | 0 32768 | 1 65536 | 2 131072 | 260 262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 27135 524288 | 0 SWND(bytes) 10.255.0.55 22 value ------------- Distribution ------------- count 16384 | 0 32768 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 65536 | 0 SWND(bytes) 10.250.0.3 2049 value ------------- Distribution ------------- count 32768 | 0 65536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 17 131072 | 0 SWND(bytes) 10.250.0.2 2049 value ------------- Distribution ------------- count 131072 | 0 262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 155 524288 | 0 SWND(bytes) 10.255.0.55 445 value ------------- Distribution ------------- count 16384 | 0 32768 | 54 65536 | 63 131072 | 306 262144 | 330 524288 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 210636 1048576 |@@@@@ 28037 2097152 | 0 On Tue, Mar 8, 2016 at 8:20 PM, Mini Trader wrote: > Running the following dtrace. > > #!/usr/sbin/dtrace -s > > #pragma D option quiet > > tcp:::send > / (args[4]->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / > { > @unacked["unacked(bytes)", args[2]->ip_daddr, args[4]->tcp_sport] = > quantize(args[3]->tcps_snxt - args[3]->tcps_suna); > } > > tcp:::receive > / (args[4]->tcp_flags & (TH_SYN|TH_RST|TH_FIN)) == 0 / > { > @swnd["SWND(bytes)", args[2]->ip_saddr, args[4]->tcp_dport] = > quantize((args[4]->tcp_window)*(1 << args[3]->tcps_snd_ws)); > > } > > > Is showing that the windows sizes are not going above 64k when things are > not working properly. > > On Tue, Mar 8, 2016 at 7:56 PM, Mini Trader > wrote: > >> If it helps. This doesn't happen on NFS from the exact same client. How >> do I file a bug? >> >> On Tue, Mar 8, 2016 at 1:51 PM, Mini Trader >> wrote: >> >>> Simple example. >>> >>> 1 Server 1 client. >>> >>> Restart service everything is fast. A few hours later from same client >>> (nothing happening concurrently) speed is slow. Restart service again, >>> speed is fast. >>> >>> Its like CIFS starts off fast than somehow for whatever reason if it is >>> not used, the connection for my CIFS drives to the server becomes slow. >>> Also this only happens when the client is downloading. Not when uploading >>> to the server that is always fast. >>> >>> On Tue, Mar 8, 2016 at 1:42 AM, Jim Klimov wrote: >>> >>>> 8 ????? 2016 ?. 6:42:13 CET, Mini Trader >>>> ?????: >>>> >Is it possible that CIFS will ignore TCP buffer settings after a while? >>>> > >>>> >I've confirmed my systems max transfer rate using iperf and have tuned >>>> >my >>>> >buffers accordingly. For whatever reason CIFS seems to forget these >>>> >settings after a while as speed drops significantly. Issuing a restart >>>> >of >>>> >the service immediately appears to restore the setting as transfer >>>> >speed >>>> >becomes normal again. >>>> > >>>> >Any ideas why this would happen? >>>> > >>>> > >>>> >>>> >------------------------------------------------------------------------ >>>> > >>>> >_______________________________________________ >>>> >OmniOS-discuss mailing list >>>> >OmniOS-discuss at lists.omniti.com >>>> >http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> As a random guess from experience with other network stuff - does the >>>> speed-drop happen on a running connection or new ones too? Do you have >>>> concurrent transfers at this time? >>>> >>>> Some other subsystems (no idea if this one too) use best speeds for new >>>> or recently awakened dormant connections, so short-lived bursts are fast - >>>> at expence of long-running active bulk transfers (deemed to be bulk because >>>> they run for a long time). >>>> >>>> HTH, Jim >>>> -- >>>> Typos courtesy of K-9 Mail on my Samsung Android >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From svavar at pipar-tbwa.is Wed Mar 9 10:00:46 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Wed, 9 Mar 2016 10:00:46 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> Message-ID: <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> Hi Dan. Thanks for the reply. I disabled all the C-states and it's the same story. If I boot from the SATA CD-ROM installation it hangs on boot right after : cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz cpu7: initialization complete - online cpu6: initialization complete - online See here : https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG If I boot from USB stick, with USB 2.0 enabled and or disabled the last message is always releated to usb stuff right after cpu6: initialization complete - online. So my BIOS configuration right now is : Intel QPI Frequency Select [Auto Max] Intel Turbo Boots Technology [Enabled] Enhanced Intel Speedstep Tech [Enabled] Processor C3 [Disabled] Processor C6 [Disabled] Intel Hyper-Threading Tech [Disabled] Core Multi-Processing [All] Execute Disable Bit [Disabled] Intel Virtualization Technology [Disasbled] Intel VT for Directed I/O [Disabled] and some more. My Bios Configuration can been seen here : https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG Best regards, Svavar O > On 8. mar. 2016, at 15:13, Dan McDonald wrote: > > >> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson wrote: >> >> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? > > Disable C-states, that's general illumos advice (regardless of distro). > > Dan > From mir at miras.org Wed Mar 9 19:05:52 2016 From: mir at miras.org (Michael Rasmussen) Date: Wed, 9 Mar 2016 20:05:52 +0100 Subject: [OmniOS-discuss] weird disk behavior Message-ID: <20160309200552.2425958c@sleipner.datanom.net> Hi all, I suddenly noticed one of the disk bays in my storage server going red with this logged in dmesg: Mar 9 19:19:47 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): Mar 9 19:19:47 nas Disconnected command timeout for Target 1 Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): Mar 9 19:19:51 nas Log info 0x31140000 received for target 1. Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): Mar 9 19:19:51 nas Log info 0x31130000 received for target 1. Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): Mar 9 19:19:51 nas Log info 0x31130000 received for target 1. Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc Mar 9 19:20:21 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): Mar 9 19:20:21 nas Command failed to complete...Device is gone Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): Mar 9 19:20:24 nas Command failed to complete...Device is gone Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): Mar 9 19:20:24 nas SYNCHRONIZE CACHE command failed (5) Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): Mar 9 19:20:24 nas drive offline zpool online and smartctl could not talk to the disk. Pulling the disk and reinserting it and the status showed green in which case both smartctl and zpool online could talk to the disk. Resilvering is now taking place. Any idea what has went wrong or should I worry for a disk imminently failing? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: No group of professionals meets except to conspire against the public at large. -- Mark Twain -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From richard.elling at richardelling.com Wed Mar 9 23:53:52 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 9 Mar 2016 15:53:52 -0800 Subject: [OmniOS-discuss] weird disk behavior In-Reply-To: <20160309200552.2425958c@sleipner.datanom.net> References: <20160309200552.2425958c@sleipner.datanom.net> Message-ID: <4D0481B5-CADF-4235-8494-599898E16768@richardelling.com> comment below... > On Mar 9, 2016, at 11:05 AM, Michael Rasmussen wrote: > > Hi all, > > I suddenly noticed one of the disk bays in my storage server going red > with this logged in dmesg: > Mar 9 19:19:47 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): > Mar 9 19:19:47 nas Disconnected command timeout for Target 1 > Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): > Mar 9 19:19:51 nas Log info 0x31140000 received for target 1. > Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc > Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): > Mar 9 19:19:51 nas Log info 0x31130000 received for target 1. > Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc > Mar 9 19:19:51 nas genunix: [ID 365881 kern.info] /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0 (mpt0): > Mar 9 19:19:51 nas Log info 0x31130000 received for target 1. > Mar 9 19:19:51 nas scsi_status=0x0, ioc_status=0x8048, scsi_state=0xc > Mar 9 19:20:21 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): > Mar 9 19:20:21 nas Command failed to complete...Device is gone > Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): > Mar 9 19:20:24 nas Command failed to complete...Device is gone > Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): > Mar 9 19:20:24 nas SYNCHRONIZE CACHE command failed (5) > Mar 9 19:20:24 nas genunix: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci1022,1708 at 3/pci1028,1f0e at 0/sd at 1,0 (sd3): > Mar 9 19:20:24 nas drive offline > > zpool online and smartctl could not talk to the disk. > > Pulling the disk and reinserting it and the status showed green in > which case both smartctl and zpool online could talk to the disk. > > Resilvering is now taking place. > > Any idea what has went wrong or should I worry for a disk imminently > failing? these are symptoms that the drive is not responding and resets are being sent to try (often in vain) to bring the disk online. Since this is mpt, it is likely 3Gbps and if the drive is SATA your tears will flow. Now that the drive is back AND the symptoms cleared after reinstalling the drive, it is very likely that drive is the source of the errors. smartctl might give more info. IMHO you should plan for replacement of that drive. NB, for that SAS fabric generation, it is posaible that the problem drive is not the only drive showing the same errors, but your drive pull test is a reasonable approach. Do not be surprised if smartctl doesn't correctly identify the issue, smart isn't very smart sometimes. -- richard > > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > No group of professionals meets except to conspire against the public > at large. -- Mark Twain > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Thu Mar 10 00:13:35 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 10 Mar 2016 01:13:35 +0100 Subject: [OmniOS-discuss] weird disk behavior In-Reply-To: <4D0481B5-CADF-4235-8494-599898E16768@richardelling.com> References: <20160309200552.2425958c@sleipner.datanom.net> <4D0481B5-CADF-4235-8494-599898E16768@richardelling.com> Message-ID: <20160310011335.37467d45@sleipner.datanom.net> On Wed, 9 Mar 2016 15:53:52 -0800 Richard Elling wrote: > > these are symptoms that the drive is not responding and resets are being sent to try (often in vain) to bring the disk online. Since this is mpt, it is likely 3Gbps and if the drive is SATA your tears will flow. Now that the drive is back AND the symptoms cleared after reinstalling the drive, it is very likely that drive is the source of the errors. smartctl might give more info. IMHO you should plan for replacement of that drive. > > NB, for that SAS fabric generation, it is posaible that the problem drive is not the only drive showing the same errors, but your drive pull test is a reasonable approach. > > Do not be surprised if smartctl doesn't correctly identify the issue, smart isn't very smart sometimes. > I ran an extended offline test of the drive which showed uncorrectable errors so I have ordered a new disk. So your diagnose was correct ;-) PS. This is my second Seagate Barracuda 7200.12 which has gone bad within a month after 28000 hours (approx. 3 years) of service. A lot less than promised by Seagate (MTBF of 0.75 million hours). Glad this is my last Seagate which goes to the bin. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Except for 75% of the women, everyone in the whole world wants to have sex. -- Ellyn Mustard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From svavar at pipar-tbwa.is Thu Mar 10 09:12:15 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Thu, 10 Mar 2016 09:12:15 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> Message-ID: <38406F40-95DE-4E31-891F-2E36CB62178E@pipar-tbwa.is> I have tried both CD-ROM ISO image boot and also the USB stick boot. the USB thumb drive boots just fine on my VMware fusion on my Mac. So it's not the installation that is corrupt. thanks. > On 9. mar. 2016, at 15:56, cristian panci? wrote: > > Hi just as an option try to disable Turbo Boots,enable Intel Virtual Tech, and intel Direct I/O, are you sure that the iso installation works?have you a spare machine with a virtualization software installed?does it boot? > maybe you have to dd the image or try anothere usb key,just a supposition > Good luck. > How much did you paid for that MB? is iPMI installed? > Can some one suggest my a home server MB with skylake design with i7 or xeon processor with IPMI software that will cost around 300Euro? > > On Wed, Mar 9, 2016 at 11:00 AM, Svavar ?rn Eysteinsson > wrote: > Hi Dan. > Thanks for the reply. > > I disabled all the C-states and it's the same story. > > If I boot from the SATA CD-ROM installation it hangs on boot right after : > > cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu7: initialization complete - online > cpu6: initialization complete - online > > See here : https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG > > If I boot from USB stick, with USB 2.0 enabled and or disabled the last message is always releated to usb stuff > right after cpu6: initialization complete - online. > > > So my BIOS configuration right now is : > > Intel QPI Frequency Select [Auto Max] > Intel Turbo Boots Technology [Enabled] > Enhanced Intel Speedstep Tech [Enabled] > Processor C3 [Disabled] > Processor C6 [Disabled] > Intel Hyper-Threading Tech [Disabled] > Core Multi-Processing [All] > Execute Disable Bit [Disabled] > Intel Virtualization Technology [Disasbled] > Intel VT for Directed I/O [Disabled] > > and some more. > > My Bios Configuration can been seen here : > > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG > > > Best regards, > > Svavar O > > > > > On 8. mar. 2016, at 15:13, Dan McDonald > wrote: > > > > > >> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson > wrote: > >> > >> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? > > > > Disable C-states, that's general illumos advice (regardless of distro). > > > > Dan > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at will.to Sun Mar 13 05:50:23 2016 From: doug at will.to (Doug Hughes) Date: Sun, 13 Mar 2016 00:50:23 -0500 Subject: [OmniOS-discuss] misc r16 kernel panics Message-ID: <56E4FF9F.1000106@will.to> I have a couple of misc panic/crashes on an r16 box. But, this is only one box out of ~8. > $C ffffff008cfbb6d0 vpanic() ffffff008cfbb6f0 mutex_panic+0x58(fffffffffb9523c7, ffffff13c9ea5948) ffffff008cfbb760 mutex_vector_enter+0x347(ffffff13c9ea5948) ffffff008cfbb790 exi_rele+0x26(ffffff13c9ea5840) ffffff008cfbb7f0 rfs4_compound_kstat_res+0x113(ffffff13c7b96920) ffffff008cfbb8b0 rfs4_dispatch+0x1b8(ffffffffc01b2010, ffffff008cfbbca0, ffffff13e4f81c00, ffffff008cfbbac0, ffffff008cfbb958) ffffff008cfbbc20 common_dispatch+0xb2c(ffffff008cfbbca0, ffffff13e4f81c00, 2, 4 , fffffffff82d2926, ffffffffc01b2060) ffffff008cfbbc40 rfs_dispatch+0x2d(ffffff008cfbbca0, ffffff13e4f81c00) ffffff008cfbbd20 svc_getreq+0x1c1(ffffff13e4f81c00, ffffff147d975460) ffffff008cfbbd90 svc_run+0xe0(ffffff13c032d708) ffffff008cfbbdd0 svc_do_run+0x8e(1) ffffff008cfbbec0 nfssys+0x111(e, fe710fbc) ffffff008cfbbf10 _sys_sysenter_post_swapgs+0x149() previous crash was also nfssys: root at x4275-3-15-20:/root# mdb -k -w /var/crash/x4275-3-15-20/unix.1 /var/crash/x4275-3-15-20/vmcore.1 Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt_sas sata sd ip hook neti sockfs arp usba uhci stmf stmf_sbd idm crypto random cpc lofs nfs ufs logindmux ptm ] > $C ffffff008b601610 vpanic() ffffff008b601630 mutex_panic+0x58(fffffffffb952427, ffffff15352ae118) ffffff008b601660 mutex_destroy+0xd7(ffffff15352ae118) ffffff008b601690 nfsauth_free_node+0x3c(ffffff15352ae0a8) ffffff008b6016e0 nfsauth_free_clnt_node+0x38(ffffff13a0f392c8) ffffff008b601730 nfsauth_cache_free+0x48(ffffff139baf8080) ffffff008b601760 exportfree+0x56(ffffff139baf8080) ffffff008b601790 exi_rele+0x60(ffffff139baf8080) ffffff008b6017f0 rfs4_compound_kstat_res+0x113(ffffff13afeff078) ffffff008b6018b0 rfs4_dispatch+0x1b8(ffffffffc0213010, ffffff008b601ca0, ffffff1b34a0ea00, ffffff008b601ac0, ffffff008b601958) ffffff008b601c20 common_dispatch+0xb2c(ffffff008b601ca0, ffffff1b34a0ea00, 2, 4 , fffffffff85f3926, ffffffffc0213060) ffffff008b601c40 rfs_dispatch+0x2d(ffffff008b601ca0, ffffff1b34a0ea00) ffffff008b601d20 svc_getreq+0x1c1(ffffff1b34a0ea00, ffffff1fdba582e0) ffffff008b601d90 svc_run+0xe0(ffffff13996d97b0) ffffff008b601dd0 svc_do_run+0x8e(1) ffffff008b601ec0 nfssys+0x111(e, fea30fbc) ffffff008b601f10 _sys_sysenter_post_swapgs+0x149() anybody else seen these? -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.tribble at gmail.com Sun Mar 13 17:59:52 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Sun, 13 Mar 2016 17:59:52 +0000 Subject: [OmniOS-discuss] misc r16 kernel panics In-Reply-To: <56E4FF9F.1000106@will.to> References: <56E4FF9F.1000106@will.to> Message-ID: On Sun, Mar 13, 2016 at 5:50 AM, Doug Hughes wrote: > I have a couple of misc panic/crashes on an r16 box. But, this is only one > box out of ~8. > > > $C > > ffffff008cfbb6d0 vpanic() > > ffffff008cfbb6f0 mutex_panic+0x58(fffffffffb9523c7, ffffff13c9ea5948) > > ffffff008cfbb760 mutex_vector_enter+0x347(ffffff13c9ea5948) > > ffffff008cfbb790 exi_rele+0x26(ffffff13c9ea5840) > > ffffff008cfbb7f0 rfs4_compound_kstat_res+0x113(ffffff13c7b96920) > > ffffff008cfbb8b0 rfs4_dispatch+0x1b8(ffffffffc01b2010, ffffff008cfbbca0, > > ffffff13e4f81c00, ffffff008cfbbac0, ffffff008cfbb958) > > ffffff008cfbbc20 common_dispatch+0xb2c(ffffff008cfbbca0, ffffff13e4f81c00, > 2, 4 > > , fffffffff82d2926, ffffffffc01b2060) > > ffffff008cfbbc40 rfs_dispatch+0x2d(ffffff008cfbbca0, ffffff13e4f81c00) > > ffffff008cfbbd20 svc_getreq+0x1c1(ffffff13e4f81c00, ffffff147d975460) > > ffffff008cfbbd90 svc_run+0xe0(ffffff13c032d708) > > ffffff008cfbbdd0 svc_do_run+0x8e(1) > > ffffff008cfbbec0 nfssys+0x111(e, fe710fbc) > > ffffff008cfbbf10 _sys_sysenter_post_swapgs+0x149( > This looks like 6472 https://www.illumos.org/issues/6472 Which has a pending fix as part of 6696 previous crash was also nfssys: > > root at x4275-3-15-20:/root# mdb -k -w /var/crash/x4275-3-15-20/unix.1 > /var/crash/x4275-3-15-20/vmcore.1 > Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc > pcplusmp scsi_vhci zfs mpt_sas sata sd ip hook neti sockfs arp usba uhci > stmf stmf_sbd idm crypto random cpc lofs nfs ufs logindmux ptm ] > > $C > ffffff008b601610 vpanic() > ffffff008b601630 mutex_panic+0x58(fffffffffb952427, ffffff15352ae118) > ffffff008b601660 mutex_destroy+0xd7(ffffff15352ae118) > ffffff008b601690 nfsauth_free_node+0x3c(ffffff15352ae0a8) > ffffff008b6016e0 nfsauth_free_clnt_node+0x38(ffffff13a0f392c8) > ffffff008b601730 nfsauth_cache_free+0x48(ffffff139baf8080) > ffffff008b601760 exportfree+0x56(ffffff139baf8080) > ffffff008b601790 exi_rele+0x60(ffffff139baf8080) > ffffff008b6017f0 rfs4_compound_kstat_res+0x113(ffffff13afeff078) > ffffff008b6018b0 rfs4_dispatch+0x1b8(ffffffffc0213010, ffffff008b601ca0, > ffffff1b34a0ea00, ffffff008b601ac0, ffffff008b601958) > ffffff008b601c20 common_dispatch+0xb2c(ffffff008b601ca0, ffffff1b34a0ea00, > 2, 4 > , fffffffff85f3926, ffffffffc0213060) > ffffff008b601c40 rfs_dispatch+0x2d(ffffff008b601ca0, ffffff1b34a0ea00) > ffffff008b601d20 svc_getreq+0x1c1(ffffff1b34a0ea00, ffffff1fdba582e0) > ffffff008b601d90 svc_run+0xe0(ffffff13996d97b0) > ffffff008b601dd0 svc_do_run+0x8e(1) > ffffff008b601ec0 nfssys+0x111(e, fea30fbc) > ffffff008b601f10 _sys_sysenter_post_swapgs+0x149() > > anybody else seen these? > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From svavar at pipar-tbwa.is Mon Mar 14 10:19:59 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Mon, 14 Mar 2016 10:19:59 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: <38406F40-95DE-4E31-891F-2E36CB62178E@pipar-tbwa.is> References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> <38406F40-95DE-4E31-891F-2E36CB62178E@pipar-tbwa.is> Message-ID: Solaris 11.3 will boot just fine on USB disk, but OmniOS will not. My boot options are just : -m verbose -v thanks Svavar > On 10. mar. 2016, at 09:12, Svavar ?rn Eysteinsson wrote: > > I have tried both CD-ROM ISO image boot and also the USB stick boot. > the USB thumb drive boots just fine on my VMware fusion on my Mac. > > So it's not the installation that is corrupt. > > thanks. > > >> On 9. mar. 2016, at 15:56, cristian panci? > wrote: >> >> Hi just as an option try to disable Turbo Boots,enable Intel Virtual Tech, and intel Direct I/O, are you sure that the iso installation works?have you a spare machine with a virtualization software installed?does it boot? >> maybe you have to dd the image or try anothere usb key,just a supposition >> Good luck. >> How much did you paid for that MB? is iPMI installed? >> Can some one suggest my a home server MB with skylake design with i7 or xeon processor with IPMI software that will cost around 300Euro? >> >> On Wed, Mar 9, 2016 at 11:00 AM, Svavar ?rn Eysteinsson > wrote: >> Hi Dan. >> Thanks for the reply. >> >> I disabled all the C-states and it's the same story. >> >> If I boot from the SATA CD-ROM installation it hangs on boot right after : >> >> cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu7: initialization complete - online >> cpu6: initialization complete - online >> >> See here : https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG >> >> If I boot from USB stick, with USB 2.0 enabled and or disabled the last message is always releated to usb stuff >> right after cpu6: initialization complete - online. >> >> >> So my BIOS configuration right now is : >> >> Intel QPI Frequency Select [Auto Max] >> Intel Turbo Boots Technology [Enabled] >> Enhanced Intel Speedstep Tech [Enabled] >> Processor C3 [Disabled] >> Processor C6 [Disabled] >> Intel Hyper-Threading Tech [Disabled] >> Core Multi-Processing [All] >> Execute Disable Bit [Disabled] >> Intel Virtualization Technology [Disasbled] >> Intel VT for Directed I/O [Disabled] >> >> and some more. >> >> My Bios Configuration can been seen here : >> >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG >> >> >> Best regards, >> >> Svavar O >> >> >> >> > On 8. mar. 2016, at 15:13, Dan McDonald > wrote: >> > >> > >> >> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson > wrote: >> >> >> >> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? >> > >> > Disable C-states, that's general illumos advice (regardless of distro). >> > >> > Dan >> > >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Mon Mar 14 13:14:31 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Mon, 14 Mar 2016 14:14:31 +0100 Subject: [OmniOS-discuss] RSF-1/ZFS panics node when offlining one iSCSI storage mirror In-Reply-To: <56DDBF8B.6000207@jvm.de> References: <56DC425F.1020906@jvm.de> <5760FDEB-C954-497A-B434-FB5AEA871146@omniti.com> <56DDBF8B.6000207@jvm.de> Message-ID: <56E6B937.90008@hfg-gmuend.de> about iscsiadm modify initiator-node -T conn-login-max=60 You can set lower values but tuning settings depend on rsp-timeout and login-delay so you must set seqentially or together. http://docs.oracle.com/cd/E36784_01/html/E36836/iscsi-19.html ps If you use napp-it pro (16.03 dev), I have added initiator tuning as a menu under Comstar > Initiator > Settings Gea Am 07.03.2016 um 18:51 schrieb Stephan Budach: > Hi Dan, > > Am 07.03.16 um 15:41 schrieb Dan McDonald: >>> On Mar 6, 2016, at 9:44 AM, Stephan Budach >>> wrote: >>> >>> when I noted that one node would panic, >> AS A RULE -- if you have an OmniOS box panic, you should save off the >> corefile (vmdump.N) and be able to share it with the list. I >> understand this may be an RSF-1 panic, BUT if it's not, it'd be nice >> to know. >> >> You can upload it to uploads.omniti.com if you wish, just request an >> upload token. >> >> Dan >> > thanks - I will keep that in mind and I actually had a core dump > available, but since I was testing around, I didn't mean to occupy > anyone's time more than absolutely necessary and so I dumoed them. > Speaking of that incident, I have lowered the iSCSI connection timeout > to 60s, which seems to be the lowest value supported by issueing a > > iscsiadm modify initiator-node -T conn-login-max=60 > > and afterwards I used stmfadm offline target on the storage node to > cut the target off. This time, the initiator timed out after 60s and > that particular zpool changed it's status to degraded without anything > happening. I still have to test that under load, but I will probably > push that to next weekend. > > Thanks, > Stephan > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- H f G Hochschule f?r Gestaltung university of design Schw?bisch Gm?nd Rektor-Klaus Str. 100 73525 Schw?bisch Gm?nd Guenther Alka, Dipl.-Ing. (FH) Leiter des Rechenzentrums head of computer center Tel 07171 602 627 Fax 07171 69259 guenther.alka at hfg-gmuend.de http://rz.hfg-gmuend.de From peter.tribble at gmail.com Mon Mar 14 16:46:08 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Mon, 14 Mar 2016 16:46:08 +0000 Subject: [OmniOS-discuss] 151014 versions Message-ID: Within a given release, say r151014, the ISO image may be updated to reflect changes. Snag is, it's always called OmniOS_Text_r151014.iso So, how do I get hold of one of the earlier ISO images? The reason here is that if I put together a development or build server, it cannot be newer than the oldest system in my production environment. Being older is fine, binary compatibility ensures that anything I build will run fine on a newer release; being newer is bad, as there may be (and there are) new interfaces added to libc during the lifetime of a release. Ideally, I would be able to go to the r151014 release notes and download the ISO corresponding to any given update. Is that possible? Thanks, -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Mar 14 17:30:41 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 14 Mar 2016 13:30:41 -0400 Subject: [OmniOS-discuss] 151014 versions In-Reply-To: References: Message-ID: <3C5F8E0D-6066-43B8-B6A1-A472A08355FE@omniti.com> > On Mar 14, 2016, at 12:46 PM, Peter Tribble wrote: > > So, how do I get hold of one of the earlier ISO images? I rename them, but apparently using http://omnios.omniti.com/media/ by itself doesn't show you the directory: -rw-r--r-- 1 danmcd users 405942272 Feb 4 15:51 OmniOS_Text_r151014.iso -rw-r--r-- 1 danmcd users 406155264 Dec 16 20:16 OmniOS_Text_r151014-Dec16.iso -rw-r--r-- 1 danmcd users 405233664 Nov 13 11:36 OmniOS_Text_r151014-Nov12.iso -rw-r--r-- 1 danmcd users 405069824 Sep 14 2015 OmniOS_Text_r151014-Sep14.iso -rw-r--r-- 1 danmcd users 404805632 Jul 27 2015 OmniOS_Text_r151014-Jul27.iso -rw-r--r-- 1 danmcd users 403738624 Apr 20 2015 OmniOS_Text_r151014-Apr20.iso One of those should do it, appended to http://omnios.omniti.com/media/ Dan From cj.keist at colostate.edu Mon Mar 14 17:41:38 2016 From: cj.keist at colostate.edu (CJ Keist) Date: Mon, 14 Mar 2016 11:41:38 -0600 Subject: [OmniOS-discuss] recovering deleted files from ZFS? Message-ID: <56E6F7D2.10009@colostate.edu> All, Thought I try asking this question on this forum. In light of no snapshots, is there a way in ZFS to recover a recently deleted file? We do nightly backups, but this would be a file that was deleted before the coming daily backup. Does anyone know if there is a service that can do file recoveries from ZFS? -- C. J. Keist Email: cj.keist at colostate.edu Systems Group Manager Solaris 10 OS (SAI) Engineering Network Services Phone: 970-491-0630 College of Engineering, CSU Fax: 970-491-5569 Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness' From lkateley at kateley.com Mon Mar 14 17:47:14 2016 From: lkateley at kateley.com (Linda Kateley) Date: Mon, 14 Mar 2016 12:47:14 -0500 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E6F7D2.10009@colostate.edu> References: <56E6F7D2.10009@colostate.edu> Message-ID: <56E6F922.2000309@kateley.com> drive savers do some recovery. linda On 3/14/16 12:41 PM, CJ Keist wrote: > All, > Thought I try asking this question on this forum. In light of no > snapshots, is there a way in ZFS to recover a recently deleted file? > We do nightly backups, but this would be a file that was deleted > before the coming daily backup. Does anyone know if there is a > service that can do file recoveries from ZFS? > > From bfriesen at simple.dallas.tx.us Mon Mar 14 18:12:13 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 14 Mar 2016 13:12:13 -0500 (CDT) Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E6F7D2.10009@colostate.edu> References: <56E6F7D2.10009@colostate.edu> Message-ID: On Mon, 14 Mar 2016, CJ Keist wrote: > All, > Thought I try asking this question on this forum. In light of no > snapshots, is there a way in ZFS to recover a recently deleted file? We do > nightly backups, but this would be a file that was deleted before the coming > daily backup. Does anyone know if there is a service that can do file > recoveries from ZFS? Hopefully this is not for a problem which has already occured. A solution would be to do zfs filesystem snapshots whenever important data has changed or at a high enough frequency that the probability of loss before backup is marginalized. Otherwise there is no assurance that the deleted data remains. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Mon Mar 14 18:28:46 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 14 Mar 2016 14:28:46 -0400 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E6F7D2.10009@colostate.edu> References: <56E6F7D2.10009@colostate.edu> Message-ID: The very long-shot chance is to export the pool (after backing it recent changes), use zdb to see what its current TXG number is, and re-import the pool with an earlier TXG number using -T. You'll lose files or changes created after the old TXG, AND if it's been a long enough time or a full enough pool, you won't get back to that TXG. It's an extraordinary measure, one I wouldn't take unless I was truly desperate. Dan Sent from my iPhone (typos, autocorrect, and all) > On Mar 14, 2016, at 1:41 PM, CJ Keist wrote: > > All, > Thought I try asking this question on this forum. In light of no snapshots, is there a way in ZFS to recover a recently deleted file? We do nightly backups, but this would be a file that was deleted before the coming daily backup. Does anyone know if there is a service that can do file recoveries from ZFS? > > > -- > C. J. Keist Email: cj.keist at colostate.edu > Systems Group Manager Solaris 10 OS (SAI) > Engineering Network Services Phone: 970-491-0630 > College of Engineering, CSU Fax: 970-491-5569 > Ft. Collins, CO 80523-1301 > > All I want is a chance to prove 'Money can't buy happiness' > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From cj.keist at colostate.edu Mon Mar 14 18:37:44 2016 From: cj.keist at colostate.edu (CJ Keist) Date: Mon, 14 Mar 2016 12:37:44 -0600 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: References: <56E6F7D2.10009@colostate.edu> Message-ID: <56E704F8.3070200@colostate.edu> Thank you!. In this case I think I can do this as the server is basically retired now. On 3/14/16 12:28 PM, Dan McDonald wrote: > The very long-shot chance is to export the pool (after backing it recent changes), use zdb to see what its current TXG number is, and re-import the pool with an earlier TXG number using -T. You'll lose files or changes created after the old TXG, AND if it's been a long enough time or a full enough pool, you won't get back to that TXG. > > It's an extraordinary measure, one I wouldn't take unless I was truly desperate. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > >> On Mar 14, 2016, at 1:41 PM, CJ Keist wrote: >> >> All, >> Thought I try asking this question on this forum. In light of no snapshots, is there a way in ZFS to recover a recently deleted file? We do nightly backups, but this would be a file that was deleted before the coming daily backup. Does anyone know if there is a service that can do file recoveries from ZFS? >> >> >> -- >> C. J. Keist Email: cj.keist at colostate.edu >> Systems Group Manager Solaris 10 OS (SAI) >> Engineering Network Services Phone: 970-491-0630 >> College of Engineering, CSU Fax: 970-491-5569 >> Ft. Collins, CO 80523-1301 >> >> All I want is a chance to prove 'Money can't buy happiness' >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss -- C. J. Keist Email: cj.keist at colostate.edu Systems Group Manager Solaris 10 OS (SAI) Engineering Network Services Phone: 970-491-0630 College of Engineering, CSU Fax: 970-491-5569 Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness' From cj.keist at colostate.edu Mon Mar 14 18:42:06 2016 From: cj.keist at colostate.edu (CJ Keist) Date: Mon, 14 Mar 2016 12:42:06 -0600 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: References: <56E6F7D2.10009@colostate.edu> Message-ID: <56E705FE.3030807@colostate.edu> Dan, You know if this is a just one shot attempt? Meaning if I choose the wrong TXG to import, can I export and try again with a different TXG? On 3/14/16 12:28 PM, Dan McDonald wrote: > The very long-shot chance is to export the pool (after backing it recent changes), use zdb to see what its current TXG number is, and re-import the pool with an earlier TXG number using -T. You'll lose files or changes created after the old TXG, AND if it's been a long enough time or a full enough pool, you won't get back to that TXG. > > It's an extraordinary measure, one I wouldn't take unless I was truly desperate. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > >> On Mar 14, 2016, at 1:41 PM, CJ Keist wrote: >> >> All, >> Thought I try asking this question on this forum. In light of no snapshots, is there a way in ZFS to recover a recently deleted file? We do nightly backups, but this would be a file that was deleted before the coming daily backup. Does anyone know if there is a service that can do file recoveries from ZFS? >> >> >> -- >> C. J. Keist Email: cj.keist at colostate.edu >> Systems Group Manager Solaris 10 OS (SAI) >> Engineering Network Services Phone: 970-491-0630 >> College of Engineering, CSU Fax: 970-491-5569 >> Ft. Collins, CO 80523-1301 >> >> All I want is a chance to prove 'Money can't buy happiness' >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss -- C. J. Keist Email: cj.keist at colostate.edu Systems Group Manager Solaris 10 OS (SAI) Engineering Network Services Phone: 970-491-0630 College of Engineering, CSU Fax: 970-491-5569 Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness' From bfriesen at simple.dallas.tx.us Mon Mar 14 18:45:37 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 14 Mar 2016 13:45:37 -0500 (CDT) Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E704F8.3070200@colostate.edu> References: <56E6F7D2.10009@colostate.edu> <56E704F8.3070200@colostate.edu> Message-ID: On Mon, 14 Mar 2016, CJ Keist wrote: > Thank you!. In this case I think I can do this as the server is basically > retired now. Take care since this would toast the disks for any other recovery attempt. The data may very well be available, especially if the file is small (since zfs typically writes 128k at a time). Recovery is easiest if the pool was a single disk or mirrored rather than raidzN. There are only something like 20 transaction groups available to go back to and this might span only a few seconds to a 100 seconds worth of time. If the file was destroyed long before the problem was noticed, then this approach won't work. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Mon Mar 14 18:53:36 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 14 Mar 2016 14:53:36 -0400 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E705FE.3030807@colostate.edu> References: <56E6F7D2.10009@colostate.edu> <56E705FE.3030807@colostate.edu> Message-ID: <682931D8-F541-4FBC-985F-4ECED0011393@omniti.com> > On Mar 14, 2016, at 2:42 PM, CJ Keist wrote: > > Dan, > You know if this is a just one shot attempt? Meaning if I choose the wrong TXG to import, can I export and try again with a different TXG? Import it read-only and you can probably attempt multiple TXGs. Dan From jdg117 at elvis.arl.psu.edu Mon Mar 14 19:01:26 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Mon, 14 Mar 2016 15:01:26 -0400 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: Your message of "Mon, 14 Mar 2016 10:19:59 -0000." References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> <38406F40-95DE-4E31-891F-2E36CB62178E@pipar-tbwa.is> Message-ID: <201603141901.u2EJ1QOi014500@elvis.arl.psu.edu> In message , =?UTF-8?q?Svav ar_=C3=96rn_Eysteinsson?= writes: >Solaris 11.3 will boot just fine on USB disk, but OmniOS will not. Do any of the *BSDs boot on your mainboard? John groenveld at acm.org From richard.elling at richardelling.com Mon Mar 14 19:04:23 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Mon, 14 Mar 2016 12:04:23 -0700 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: <56E705FE.3030807@colostate.edu> References: <56E6F7D2.10009@colostate.edu> <56E705FE.3030807@colostate.edu> Message-ID: <6C095C11-4E03-43A8-818A-55F5BB18AB13@richardelling.com> > On Mar 14, 2016, at 11:42 AM, CJ Keist wrote: > > Dan, > You know if this is a just one shot attempt? Meaning if I choose the wrong TXG to import, can I export and try again with a different TXG? > pro tip: try retro import with readonly option -- richard > >> On 3/14/16 12:28 PM, Dan McDonald wrote: >> The very long-shot chance is to export the pool (after backing it recent changes), use zdb to see what its current TXG number is, and re-import the pool with an earlier TXG number using -T. You'll lose files or changes created after the old TXG, AND if it's been a long enough time or a full enough pool, you won't get back to that TXG. >> >> It's an extraordinary measure, one I wouldn't take unless I was truly desperate. >> >> Dan >> >> Sent from my iPhone (typos, autocorrect, and all) >> >>> On Mar 14, 2016, at 1:41 PM, CJ Keist wrote: >>> >>> All, >>> Thought I try asking this question on this forum. In light of no snapshots, is there a way in ZFS to recover a recently deleted file? We do nightly backups, but this would be a file that was deleted before the coming daily backup. Does anyone know if there is a service that can do file recoveries from ZFS? >>> >>> >>> -- >>> C. J. Keist Email: cj.keist at colostate.edu >>> Systems Group Manager Solaris 10 OS (SAI) >>> Engineering Network Services Phone: 970-491-0630 >>> College of Engineering, CSU Fax: 970-491-5569 >>> Ft. Collins, CO 80523-1301 >>> >>> All I want is a chance to prove 'Money can't buy happiness' >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- > C. J. Keist Email: cj.keist at colostate.edu > Systems Group Manager Solaris 10 OS (SAI) > Engineering Network Services Phone: 970-491-0630 > College of Engineering, CSU Fax: 970-491-5569 > Ft. Collins, CO 80523-1301 > > All I want is a chance to prove 'Money can't buy happiness' > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From cj.keist at colostate.edu Mon Mar 14 19:13:32 2016 From: cj.keist at colostate.edu (CJ Keist) Date: Mon, 14 Mar 2016 13:13:32 -0600 Subject: [OmniOS-discuss] recovering deleted files from ZFS? In-Reply-To: References: <56E6F7D2.10009@colostate.edu> <56E704F8.3070200@colostate.edu> Message-ID: <56E70D5C.2020701@colostate.edu> Thanks. Sounds like it's already too late then. It has been about day and half since the deletion occurred. On 3/14/16 12:45 PM, Bob Friesenhahn wrote: > On Mon, 14 Mar 2016, CJ Keist wrote: > >> Thank you!. In this case I think I can do this as the server is >> basically retired now. > > Take care since this would toast the disks for any other recovery > attempt. The data may very well be available, especially if the file > is small (since zfs typically writes 128k at a time). Recovery is > easiest if the pool was a single disk or mirrored rather than raidzN. > > There are only something like 20 transaction groups available to go > back to and this might span only a few seconds to a 100 seconds worth > of time. If the file was destroyed long before the problem was > noticed, then this approach won't work. > > Bob -- C. J. Keist Email: cj.keist at colostate.edu Systems Group Manager Solaris 10 OS (SAI) Engineering Network Services Phone: 970-491-0630 College of Engineering, CSU Fax: 970-491-5569 Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness' From peter.tribble at gmail.com Mon Mar 14 19:55:49 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Mon, 14 Mar 2016 19:55:49 +0000 Subject: [OmniOS-discuss] 151014 versions In-Reply-To: <3C5F8E0D-6066-43B8-B6A1-A472A08355FE@omniti.com> References: <3C5F8E0D-6066-43B8-B6A1-A472A08355FE@omniti.com> Message-ID: On Mon, Mar 14, 2016 at 5:30 PM, Dan McDonald wrote: > > > On Mar 14, 2016, at 12:46 PM, Peter Tribble > wrote: > > > > So, how do I get hold of one of the earlier ISO images? > > I rename them, but apparently using http://omnios.omniti.com/media/ by > itself doesn't show you the directory: > > -rw-r--r-- 1 danmcd users 405942272 Feb 4 15:51 > OmniOS_Text_r151014.iso > -rw-r--r-- 1 danmcd users 406155264 Dec 16 20:16 > OmniOS_Text_r151014-Dec16.iso > -rw-r--r-- 1 danmcd users 405233664 Nov 13 11:36 > OmniOS_Text_r151014-Nov12.iso > -rw-r--r-- 1 danmcd users 405069824 Sep 14 2015 > OmniOS_Text_r151014-Sep14.iso > -rw-r--r-- 1 danmcd users 404805632 Jul 27 2015 > OmniOS_Text_r151014-Jul27.iso > -rw-r--r-- 1 danmcd users 403738624 Apr 20 2015 > OmniOS_Text_r151014-Apr20.iso > > One of those should do it, appended to http://omnios.omniti.com/media/ > Brilliant! Thanks Dan. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ml+omnios-discuss at valo.at Mon Mar 14 21:28:30 2016 From: ml+omnios-discuss at valo.at (Christian Kivalo) Date: Mon, 14 Mar 2016 22:28:30 +0100 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> Message-ID: On 2016-03-09 11:00, Svavar ?rn Eysteinsson wrote: > Hi Dan. > Thanks for the reply. > > I disabled all the C-states and it's the same story. > > If I boot from the SATA CD-ROM installation it hangs on boot right > after : > > cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu7: initialization complete - online > cpu6: initialization complete - online > > See here : > https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG > > If I boot from USB stick, with USB 2.0 enabled and or disabled the > last message is always releated to usb stuff > right after cpu6: initialization complete - online. > > > So my BIOS configuration right now is : > > Intel QPI Frequency Select [Auto Max] > Intel Turbo Boots Technology [Enabled] > Enhanced Intel Speedstep Tech [Enabled] > Processor C3 [Disabled] > Processor C6 [Disabled] > Intel Hyper-Threading Tech [Disabled] > Core Multi-Processing [All] > Execute Disable Bit [Disabled] > Intel Virtualization Technology [Disasbled] > Intel VT for Directed I/O [Disabled] > > and some more. > > My Bios Configuration can been seen here : > > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG > Maybe something for you there was a thread here some months ago... It starts here https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03194.html and some messages into the the thread X2APIC is mentioned https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03200.html https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03201.html and a link a blogpost by jeff sipek http://blahg.josefsipek.net/?p=488 > Best regards, > > Svavar O > > > >> On 8. mar. 2016, at 15:13, Dan McDonald wrote: >> >> >>> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson >>> wrote: >>> >>> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or >>> is that not relevant? >> >> Disable C-states, that's general illumos advice (regardless of >> distro). >> >> Dan >> > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Christian Kivalo From danmcd at omniti.com Tue Mar 15 02:21:20 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 14 Mar 2016 22:21:20 -0400 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? Message-ID: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> I have OpenSSH 7.2p2 compiled, thanks again to Joyent's Alex Wilson and his set of patches (only one of which I remove, because we have our own historical sshd_config patch). I'd like a few people with bloody to test it out, please. It will be on the bloody repo server in about 30 minutes from now (11pm US/Eastern) so you can "pkg update" to get it. Thanks, Dan From lotheac at iki.fi Tue Mar 15 05:24:13 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Tue, 15 Mar 2016 07:24:13 +0200 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? In-Reply-To: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> References: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> Message-ID: <20160315052413.GA1479@kekkonen.niksula.hut.fi> On Mon, Mar 14 2016 22:21:20 -0400, Dan McDonald wrote: > I have OpenSSH 7.2p2 compiled, thanks again to Joyent's Alex Wilson and his set of patches (only one of which I remove, because we have our own historical sshd_config patch). > > I'd like a few people with bloody to test it out, please. It will be on the bloody repo server in about 30 minutes from now (11pm US/Eastern) so you can "pkg update" to get it. Works fine here. omnios-build PR #82 doesn't seem to be applied to this build though? -- Lauri Tirkkonen | lotheac @ IRCnet From danmcd at omniti.com Tue Mar 15 14:42:50 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 15 Mar 2016 10:42:50 -0400 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? In-Reply-To: <20160315052413.GA1479@kekkonen.niksula.hut.fi> References: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> <20160315052413.GA1479@kekkonen.niksula.hut.fi> Message-ID: > On Mar 15, 2016, at 1:24 AM, Lauri Tirkkonen wrote: > > On Mon, Mar 14 2016 22:21:20 -0400, Dan McDonald wrote: >> I have OpenSSH 7.2p2 compiled, thanks again to Joyent's Alex Wilson and his set of patches (only one of which I remove, because we have our own historical sshd_config patch). >> >> I'd like a few people with bloody to test it out, please. It will be on the bloody repo server in about 30 minutes from now (11pm US/Eastern) so you can "pkg update" to get it. > > Works fine here. omnios-build PR #82 doesn't seem to be applied to this > build though? AHA! I knew I forgot something. This is why I haven't yet pushed the changes back into omnios-build itself yet. Will fix and republish v. soon. Thanks, Dan From danmcd at omniti.com Tue Mar 15 14:47:05 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 15 Mar 2016 10:47:05 -0400 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? In-Reply-To: References: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> <20160315052413.GA1479@kekkonen.niksula.hut.fi> Message-ID: <4A74A0AE-5C85-44BC-8BA3-64D2CA81E3E2@omniti.com> > On Mar 15, 2016, at 10:42 AM, Dan McDonald wrote: > > Will fix and republish v. soon. Try "pkg update openssh openssh-server" now, and you should see DSA support gone again (i.e. PR #82). Thanks, Dan From lotheac at iki.fi Tue Mar 15 15:11:10 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Tue, 15 Mar 2016 17:11:10 +0200 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? In-Reply-To: <4A74A0AE-5C85-44BC-8BA3-64D2CA81E3E2@omniti.com> References: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> <20160315052413.GA1479@kekkonen.niksula.hut.fi> <4A74A0AE-5C85-44BC-8BA3-64D2CA81E3E2@omniti.com> Message-ID: <20160315151110.GC12035@kekkonen.niksula.hut.fi> On Tue, Mar 15 2016 10:47:05 -0400, Dan McDonald wrote: > > > On Mar 15, 2016, at 10:42 AM, Dan McDonald wrote: > > > > Will fix and republish v. soon. > > Try "pkg update openssh openssh-server" now, and you should see DSA support gone again (i.e. PR #82). Yep, it works, thanks. You mentioned on the PR that you'd backport it to supported releases as well; is that happening? https://github.com/omniti-labs/omnios-build/pull/82#issuecomment-192332860 -- Lauri Tirkkonen | lotheac @ IRCnet From danmcd at omniti.com Tue Mar 15 15:16:55 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 15 Mar 2016 11:16:55 -0400 Subject: [OmniOS-discuss] Volunteers for OpenSSH 7.2p2 testing? In-Reply-To: <20160315151110.GC12035@kekkonen.niksula.hut.fi> References: <67D5D49F-CD64-414D-B23E-5E0310BAEFCC@omniti.com> <20160315052413.GA1479@kekkonen.niksula.hut.fi> <4A74A0AE-5C85-44BC-8BA3-64D2CA81E3E2@omniti.com> <20160315151110.GC12035@kekkonen.niksula.hut.fi> Message-ID: <84F058F7-70C6-4B7A-A849-20810ADA6B57@omniti.com> > On Mar 15, 2016, at 11:11 AM, Lauri Tirkkonen wrote: > > On Tue, Mar 15 2016 10:47:05 -0400, Dan McDonald wrote: >> >>> On Mar 15, 2016, at 10:42 AM, Dan McDonald wrote: >>> >>> Will fix and republish v. soon. >> >> Try "pkg update openssh openssh-server" now, and you should see DSA support gone again (i.e. PR #82). > > Yep, it works, thanks. You mentioned on the PR that you'd backport it to > supported releases as well; is that happening? > https://github.com/omniti-labs/omnios-build/pull/82#issuecomment-192332860 It's coming along for the ride with a 7.2p2 today or tomorrow. :) Dan From peter.tribble at gmail.com Tue Mar 15 15:40:44 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Tue, 15 Mar 2016 15:40:44 +0000 Subject: [OmniOS-discuss] header/library mismatch Message-ID: Currently having a minor issue building software. As mentioned yesterday, I'm building on a slightly older image, so that I know that any binaries I generate will run on all systems (which are newer). The snag is that the install media doesn't have the headers. And if I pkg install system/header then it installs the latest version in the repo. It doesn't look like there are any versioned constraints to tie the version of the header package to the installed libraries. Specifically, the glob fix #6409 causes problems. Because I end up with a set of headers that have the fix and a copy of libc that doesn't. But I'm sure there might be other things lurking. My workaround is to manually look at the versions of system/library and specify the matching version of system/header to install by hand. Is there a better way? This isn't helped by the way that pkg(5) goes and does an update if you utter 'pkg install', rather than just saying 'oh, you're already installed, we're done'. At least with the header package I can uninstall it and put the correct one back, with some other packages I have to roll the system back. Thanks for any thoughts! -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at bens.me.uk Tue Mar 15 15:48:51 2016 From: ben at bens.me.uk (Ben Summers) Date: Tue, 15 Mar 2016 15:48:51 +0000 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: References: Message-ID: > On 15 Mar 2016, at 15:40, Peter Tribble wrote: > > Currently having a minor issue building software. > > As mentioned yesterday, I'm building on a slightly older image, so that I > know that any binaries I generate will run on all systems (which are newer). > > The snag is that the install media doesn't have the headers. And if I > > pkg install system/header > > then it installs the latest version in the repo. It doesn't look like there > are any versioned constraints to tie the version of the header package > to the installed libraries. What if you choose the latest version as of the release date of the iso, and include the full version number in the pkg install command? http://pkg.omniti.com/omnios/r151014/en/advanced_search.shtml?token=system%2Fheader&show=p&sav=1&rpp=100&v=11%252C11-0.151014 (scroll down to see the interesting packages) pkg install system/header at 0.5.11,5.11-0.151014:20150818T161043Z All a bit manual of course. Ben From ben at bens.me.uk Tue Mar 15 15:51:30 2016 From: ben at bens.me.uk (Ben Summers) Date: Tue, 15 Mar 2016 15:51:30 +0000 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: References: Message-ID: <9158C26A-A17B-468B-9027-6ACF9949E788@bens.me.uk> > On 15 Mar 2016, at 15:48, Ben Summers wrote: > > >> On 15 Mar 2016, at 15:40, Peter Tribble wrote: >> >> Currently having a minor issue building software. >> >> As mentioned yesterday, I'm building on a slightly older image, so that I >> know that any binaries I generate will run on all systems (which are newer). >> >> The snag is that the install media doesn't have the headers. And if I >> >> pkg install system/header >> >> then it installs the latest version in the repo. It doesn't look like there >> are any versioned constraints to tie the version of the header package >> to the installed libraries. > > What if you choose the latest version as of the release date of the iso, and include the full version number in the pkg install command? > > http://pkg.omniti.com/omnios/r151014/en/advanced_search.shtml?token=system%2Fheader&show=p&sav=1&rpp=100&v=11%252C11-0.151014 > (scroll down to see the interesting packages) > > pkg install system/header at 0.5.11,5.11-0.151014:20150818T161043Z > > All a bit manual of course. Actually I have a sneaking suspicion that this is what you meant by "manual". I'll go away now, after making two annoying noises on the list. :-( Ben From svavar at pipar-tbwa.is Tue Mar 15 16:17:41 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Tue, 15 Mar 2016 16:17:41 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> Message-ID: <483D6A7C-F2FE-4706-A19E-F8C4D774BBDC@pipar-tbwa.is> FreeBSD 10.2 and Solaris 11.3 boots and installs just fine. I have read the mailing list thread below. thanks for that. I have no floppy support on this motherboard. I have also disabled the C3 and C6 state in the BIOS. I have also tried to boot up with : -m verbose -v -B iommu-enable=false -B immu-enable=false -B immu-intrmap-enable=false Also tried out -B acpi-user-options=0x2 to completly disable ACPI, and it's the same story. The kernel complains about no ACPI but will hang just on the same spot as always. Thank you kindly. > On 14. mar. 2016, at 21:28, Christian Kivalo wrote: > > > > On 2016-03-09 11:00, Svavar ?rn Eysteinsson wrote: >> Hi Dan. >> Thanks for the reply. >> I disabled all the C-states and it's the same story. >> If I boot from the SATA CD-ROM installation it hangs on boot right after : >> cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu7: initialization complete - online >> cpu6: initialization complete - online >> See here : >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG >> If I boot from USB stick, with USB 2.0 enabled and or disabled the >> last message is always releated to usb stuff >> right after cpu6: initialization complete - online. >> So my BIOS configuration right now is : >> Intel QPI Frequency Select [Auto Max] >> Intel Turbo Boots Technology [Enabled] >> Enhanced Intel Speedstep Tech [Enabled] >> Processor C3 [Disabled] >> Processor C6 [Disabled] >> Intel Hyper-Threading Tech [Disabled] >> Core Multi-Processing [All] >> Execute Disable Bit [Disabled] >> Intel Virtualization Technology [Disasbled] >> Intel VT for Directed I/O [Disabled] >> and some more. >> My Bios Configuration can been seen here : >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG > Maybe something for you there was a thread here some months ago... > It starts here https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03194.html > > and some messages into the the thread X2APIC is mentioned > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03200.html > > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03201.html > > and a link a blogpost by jeff sipek > http://blahg.josefsipek.net/?p=488 > > >> Best regards, >> Svavar O >>> On 8. mar. 2016, at 15:13, Dan McDonald wrote: >>>> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson wrote: >>>> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? >>> Disable C-states, that's general illumos advice (regardless of distro). >>> Dan >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- > Christian Kivalo > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Tue Mar 15 22:06:34 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 15 Mar 2016 18:06:34 -0400 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available Message-ID: I've updated git to 2.7.3 and OpenSSH to 7.2p2. Updates are already pushed out to r151014 (LTS), r151016 (Stable), and bloody repo servers. Additionaly, the r151014 and r151016 release media have been updated. (Replaced ones are now r15101[56]-Feb04...). The release media update for ISO and USB-DD includes a Caiman installer fix that I'd thought was already pushed out (timezones with Unicode crashed the installer), but wasn't. Thank you, and happy updating! Dan From phil.harman at gmail.com Wed Mar 16 06:54:01 2016 From: phil.harman at gmail.com (Phil Harman) Date: Wed, 16 Mar 2016 06:54:01 +0000 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: References: Message-ID: Dan, Thanks for this. Apologies if I?ve missed something, but I?m not seeing the update with ?pkg update entire? on r151016. However, I do see the update if I use ?pkg update openssh? and ?pkg update openssh-server?? root at omnios2:/root# pkg update entire No updates available for this image. root at omnios2:/root# uname -a SunOS omnios2 5.11 omnios-33c53a8 i86pc i386 i86pc root at omnios2:/root# cat /etc/release OmniOS v11 r151016 Copyright 2015 OmniTI Computer Consulting, Inc. All rights reserved. Use is subject to license terms. root at omnios2:/root# pkg publisher PUBLISHER TYPE STATUS P LOCATION omnios origin online F http://pkg.omniti.com/omnios/r151016/ root at omnios2:/root# pkg list | grep -i openssh network/openssh 7.1.2-0.151016 i-- network/openssh-server 7.1.2-0.151016 i-- root at omnios2:/root# pkg update entire No updates available for this image. root at omnios2:/root# pkg update openssh Packages to update: 2 Create boot environment: No Create backup boot environment: Yes DOWNLOAD PKGS FILES XFER (MB) SPEED Completed 2/2 414/414 8.6/8.6 1.9M/s PHASE ITEMS Removing old actions 6/6 Installing new actions 31/31 Updating modified actions 416/416 Updating package state database Done Updating package cache 2/2 Updating image state Done Creating fast lookup database Done root at omnios2:/root# pkg update openssh-server Packages to update: 1 Services to change: 2 Create boot environment: No Create backup boot environment: Yes DOWNLOAD PKGS FILES XFER (MB) SPEED Completed 1/1 9/9 0.4/0.4 376k/s PHASE ITEMS Removing old actions 3/3 Installing new actions 7/7 Updating modified actions 9/9 Updating package state database Done Updating package cache 1/1 Updating image state Done Creating fast lookup database Done root at omnios2:/root# pkg list | grep -i openssh network/openssh 7.2.2-0.151016 i-- network/openssh-server 7.2.2-0.151016 i-- root at omnios2:/root# pkg list | grep -i ssh network/openssh 7.2.2-0.151016 i-- network/openssh-server 7.2.2-0.151016 i-- root at omnios2:/root# I?m just a little puzzled because this was a ?default? install, yet ?openssh? is not part of ?entire?. Things are different on bloody, brings a ?default? install brings in SunSSH (?network/ssh?, ?network/ssh/ssh-key?, ?service/network/ssh?, and ?service/network/ssh-common?), which are part of ?entire". Phil > On 15 Mar 2016, at 22:06, Dan McDonald wrote: > > I've updated git to 2.7.3 and OpenSSH to 7.2p2. Updates are already pushed out to r151014 (LTS), r151016 (Stable), and bloody repo servers. Additionaly, the r151014 and r151016 release media have been updated. (Replaced ones are now r15101[56]-Feb04...). The release media update for ISO and USB-DD includes a Caiman installer fix that I'd thought was already pushed out (timezones with Unicode crashed the installer), but wasn't. > > Thank you, and happy updating! > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From ml+omnios-discuss at valo.at Wed Mar 16 11:00:18 2016 From: ml+omnios-discuss at valo.at (Christian Kivalo) Date: Wed, 16 Mar 2016 12:00:18 +0100 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: References: Message-ID: On 2016-03-16 07:54, Phil Harman wrote: > Dan, > > Thanks for this. > [ ... ] > > I?m just a little puzzled because this was a ?default? install, > yet ?openssh? is not part of ?entire?. > > Things are different on bloody, brings a ?default? install brings > in SunSSH (?network/ssh?, ?network/ssh/ssh-key?, > ?service/network/ssh?, and ?service/network/ssh-common?), > which are part of ?entire". Take a look at the release notes of r151016 and the description on how to change sunssh to openssh http://omnios.omniti.com/wiki.php/ReleaseNotes/r151016#r151016release > Phil > >> On 15 Mar 2016, at 22:06, Dan McDonald wrote: >> >> I've updated git to 2.7.3 and OpenSSH to 7.2p2. Updates are already >> pushed out to r151014 (LTS), r151016 (Stable), and bloody repo >> servers. Additionaly, the r151014 and r151016 release media have >> been updated. (Replaced ones are now r15101[56]-Feb04...). The >> release media update for ISO and USB-DD includes a Caiman installer >> fix that I'd thought was already pushed out (timezones with Unicode >> crashed the installer), but wasn't. >> >> Thank you, and happy updating! >> Dan >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Christian Kivalo From phil.harman at gmail.com Wed Mar 16 11:26:29 2016 From: phil.harman at gmail.com (Phil Harman) Date: Wed, 16 Mar 2016 11:26:29 +0000 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: References: Message-ID: > On 16 Mar 2016, at 11:00, Christian Kivalo wrote: > > >> On 2016-03-16 07:54, Phil Harman wrote: >> Dan, >> Thanks for this. > > [ ... ] > >> I?m just a little puzzled because this was a ?default? install, >> yet ?openssh? is not part of ?entire?. >> Things are different on bloody, brings a ?default? install brings >> in SunSSH (?network/ssh?, ?network/ssh/ssh-key?, >> ?service/network/ssh?, and ?service/network/ssh-common?), >> which are part of ?entire". > > Take a look at the release notes of r151016 and the description on how to change sunssh to openssh > > http://omnios.omniti.com/wiki.php/ReleaseNotes/r151016#r151016release Thanks, though I'm still puzzled as to why the default install doesn't upgrade properly with "pkg update entire", and as to why bloody hasn't also switched to openssh as the default. From danmcd at omniti.com Wed Mar 16 12:23:17 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 16 Mar 2016 08:23:17 -0400 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: References: Message-ID: <5E542AC2-CE66-4B4E-AA3C-916D4C544F59@omniti.com> > On Mar 16, 2016, at 7:26 AM, Phil Harman wrote: > > Thanks, though I'm still puzzled as to why the default install doesn't upgrade properly with "pkg update entire", and as to why bloody hasn't also switched to openssh as the default. A fresh install of recently-minted 014 or later WILL install openssh by default. If you have an existing install with SunSSH, that'll stick. As for "entire", don't... just utter: pkg update and the right thing will happen. Dan From peter.tribble at gmail.com Wed Mar 16 13:45:00 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Wed, 16 Mar 2016 13:45:00 +0000 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: <9158C26A-A17B-468B-9027-6ACF9949E788@bens.me.uk> References: <9158C26A-A17B-468B-9027-6ACF9949E788@bens.me.uk> Message-ID: On Tue, Mar 15, 2016 at 3:51 PM, Ben Summers wrote: > > > On 15 Mar 2016, at 15:48, Ben Summers wrote: > > > > > >> On 15 Mar 2016, at 15:40, Peter Tribble > wrote: > >> > >> Currently having a minor issue building software. > >> > >> As mentioned yesterday, I'm building on a slightly older image, so that > I > >> know that any binaries I generate will run on all systems (which are > newer). > >> > >> The snag is that the install media doesn't have the headers. And if I > >> > >> pkg install system/header > >> > >> then it installs the latest version in the repo. It doesn't look like > there > >> are any versioned constraints to tie the version of the header package > >> to the installed libraries. > > > > What if you choose the latest version as of the release date of the iso, > and include the full version number in the pkg install command? > > > > > http://pkg.omniti.com/omnios/r151014/en/advanced_search.shtml?token=system%2Fheader&show=p&sav=1&rpp=100&v=11%252C11-0.151014 > > (scroll down to see the interesting packages) > > > > pkg install system/header at 0.5.11,5.11-0.151014:20150818T161043Z > > > > All a bit manual of course. > > > Actually I have a sneaking suspicion that this is what you meant by > "manual". > So what I'm actually using is the following, using a ksh associative array to map the installed version of the system/library package to the desired version of the system/header package. (It's a timestamp, so you can see that the two packages are only a few seconds apart.) typeset -A headermap headermap[20150402T175242Z]=20150402T175234Z headermap[20150417T182440Z]=20150417T182431Z headermap[20150727T054701Z]=20150727T054659Z headermap[20150818T161046Z]=20150818T161043Z headermap[20150913T201600Z]=20150913T201558Z headermap[20150914T195009Z]=20150914T195007Z headermap[20150929T225339Z]=20150929T225336Z headermap[20151112T210053Z]=20151112T210050Z if [ ! -f /usr/include/stdio.h ]; then SLVER=`pkg info system/library | grep FMRI: | awk -F: '{print $NF}'` RELVER=`pkg info system/library | grep FMRI: | awk -F@ '{print $2}' | awk -F: '{print $1}'` SHVER=${headermap[$SLVER]} if [ -n "$SHVER" ]; then pfexec pkg install pkg:/system/header@${RELVER}:${SHVER} else pfexec pkg install pkg:/system/header fi fi -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From miamikelvin at gmail.com Wed Mar 16 17:14:26 2016 From: miamikelvin at gmail.com (Miami Kelvin) Date: Wed, 16 Mar 2016 20:14:26 +0300 Subject: [OmniOS-discuss] Windows 7 Guest KVM Message-ID: Hi I am trying to setup a windows 7 VM with KVM on OmniOS. I followed instructions from http://omnios.omniti.com/wiki.php/VirtualMachinesKVM and the script works fine, but my isue is with vga support on VNC. When I try to conect to the VM on IP:PORT i get no output, the vnc window just exits. How do i add video/VGA support to my KVM in the startup script? *Miami *E. Kelvin IT Specialist, Software & Web Developer Mobile : (254) 710 430060 P.O Box: 54481-00200 Nairobi, Kenya The future belongs to those who believe in the beauty of their dreams. 0 viruses found. www.avast.com <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Mar 16 17:54:09 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 16 Mar 2016 13:54:09 -0400 Subject: [OmniOS-discuss] Windows 7 Guest KVM In-Reply-To: References: Message-ID: <8509AEF9-0330-474F-9249-8E2A951DD712@omniti.com> Make sure you're connecting to the right port for VNC. The script adds 5900 to $VNC (whatever that is). The pfiles(1) command is helpful here: pfiles `pgrep qemu` And look for an open AF_INET socket and see what port it's bound to. Dan From sford123 at ibbr.umd.edu Wed Mar 16 18:57:36 2016 From: sford123 at ibbr.umd.edu (Steven Ford) Date: Wed, 16 Mar 2016 14:57:36 -0400 Subject: [OmniOS-discuss] File Permissions and ACLs Message-ID: Hello everybody, I was recently working through a problem where my CIFS share was unseen after moving a ZFS file system from OpenIndiana to OmniOS. I found the issue was with the file permissions. My ZFS file system is called data, in a pool called zpool. On the old server, the permissions on /zpool/data were: d---------+821 Administrators at BUILTIN 2147483650 823 Mar 4 14:52 data group:Domain Users at DOMAIN:r-x---a-R-c--s:-------:allow This was sufficient to allow users to see the share, however, on OmniOS I had to chmod 555 before the share was visible. So I'm left wondering why this worked in OpenIndiana, but not in OmniOS. Any thoughts? Thanks, Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Mar 16 19:08:53 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 16 Mar 2016 15:08:53 -0400 Subject: [OmniOS-discuss] File Permissions and ACLs In-Reply-To: References: Message-ID: <319E0AC8-468A-4AC9-892E-0C713C2ABA6C@omniti.com> > On Mar 16, 2016, at 2:57 PM, Steven Ford wrote: > > This was sufficient to allow users to see the share, however, on OmniOS I had to chmod 555 before the share was visible. > > So I'm left wondering why this worked in OpenIndiana, but not in OmniOS. Which versions of OI and OmniOS? No useful answer without knowing that. Dan From sford123 at ibbr.umd.edu Wed Mar 16 19:12:11 2016 From: sford123 at ibbr.umd.edu (Steven Ford) Date: Wed, 16 Mar 2016 15:12:11 -0400 Subject: [OmniOS-discuss] File Permissions and ACLs In-Reply-To: <319E0AC8-468A-4AC9-892E-0C713C2ABA6C@omniti.com> References: <319E0AC8-468A-4AC9-892E-0C713C2ABA6C@omniti.com> Message-ID: oi_151a7 and OmniOS r151016 On Wed, Mar 16, 2016 at 3:08 PM, Dan McDonald wrote: > > > On Mar 16, 2016, at 2:57 PM, Steven Ford wrote: > > > > This was sufficient to allow users to see the share, however, on OmniOS > I had to chmod 555 before the share was visible. > > > > So I'm left wondering why this worked in OpenIndiana, but not in OmniOS. > > Which versions of OI and OmniOS? No useful answer without knowing that. > > Dan > > -- Steven Ford IT Infrastructure Specialist Institute for Bioscience and Biotechnology Research University of Maryland (240)314-6405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Mar 16 19:13:59 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 16 Mar 2016 15:13:59 -0400 Subject: [OmniOS-discuss] File Permissions and ACLs In-Reply-To: References: <319E0AC8-468A-4AC9-892E-0C713C2ABA6C@omniti.com> Message-ID: > On Mar 16, 2016, at 3:12 PM, Steven Ford wrote: > > oi_151a7 and OmniOS r151016 EEEESH. That's a big jump. We're talking YEARS of changes to illumos-gate, including some nontrivial changes in SMB sharing. Your original question is best asked to the illumos list as: I moved a pool from oi_151a7 to a modern-ish illumos (OmniOS r151016), why did I have to chmod my SMB share directory? Dan From sford123 at ibbr.umd.edu Wed Mar 16 19:17:38 2016 From: sford123 at ibbr.umd.edu (Steven Ford) Date: Wed, 16 Mar 2016 15:17:38 -0400 Subject: [OmniOS-discuss] File Permissions and ACLs In-Reply-To: References: <319E0AC8-468A-4AC9-892E-0C713C2ABA6C@omniti.com> Message-ID: Will do. Thanks, Dan. On Wed, Mar 16, 2016 at 3:13 PM, Dan McDonald wrote: > > > On Mar 16, 2016, at 3:12 PM, Steven Ford wrote: > > > > oi_151a7 and OmniOS r151016 > > EEEESH. That's a big jump. We're talking YEARS of changes to > illumos-gate, including some nontrivial changes in SMB sharing. > > Your original question is best asked to the illumos list as: > > I moved a pool from oi_151a7 to a modern-ish illumos (OmniOS > r151016), why did I have to chmod my SMB share directory? > > Dan > > -- Steven Ford IT Infrastructure Specialist Institute for Bioscience and Biotechnology Research University of Maryland (240)314-6405 -------------- next part -------------- An HTML attachment was scrubbed... URL: From miamikelvin at gmail.com Thu Mar 17 05:11:06 2016 From: miamikelvin at gmail.com (Miami Kelvin) Date: Thu, 17 Mar 2016 08:11:06 +0300 Subject: [OmniOS-discuss] Windows 7 Guest KVM In-Reply-To: <8509AEF9-0330-474F-9249-8E2A951DD712@omniti.com> References: <8509AEF9-0330-474F-9249-8E2A951DD712@omniti.com> Message-ID: I am connecting to the right port, in my case 5965, if I attempt to connect to any other port VNC would give an error message. But when I connect to IP:5965 VNC makes a connection the the window just disappears, so I thought it would be a problem with VGA support. Scanned for viruses. www.avast.com <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> *Miami *E. Kelvin IT Specialist, Software & Web Developer Mobile : (254) 710 430060 P.O Box: 54481-00200 Nairobi, Kenya The future belongs to those who believe in the beauty of their dreams. On Wed, Mar 16, 2016 at 8:54 PM, Dan McDonald wrote: > Make sure you're connecting to the right port for VNC. The script adds > 5900 to $VNC (whatever that is). > > The pfiles(1) command is helpful here: > > pfiles `pgrep qemu` > > And look for an open AF_INET socket and see what port it's bound to. > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Thu Mar 17 17:19:22 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 17 Mar 2016 12:19:22 -0500 (CDT) Subject: [OmniOS-discuss] Windows 7 Guest KVM In-Reply-To: References: <8509AEF9-0330-474F-9249-8E2A951DD712@omniti.com> Message-ID: On Thu, 17 Mar 2016, Miami Kelvin wrote: > I am connecting to the right port, in my case 5965, if I attempt to connect to any other port VNC would give an error message. But > when I connect to IP:5965 VNC makes a connection the the window just disappears, so I thought it would be a problem with VGA > support.? What VNC client are you using? There seem to be quite a lot of them now. Some VNC clients might not interoperate correctly since they rely on RFB protocol extensions, or the server might require an extension. I suggest doing 'telnet hostname 5965' and see if you get the expected RFB challenge prompt (see https://en.wikipedia.org/wiki/RFB_protocol). Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From miamikelvin at gmail.com Thu Mar 17 18:13:19 2016 From: miamikelvin at gmail.com (Miami Kelvin) Date: Thu, 17 Mar 2016 21:13:19 +0300 Subject: [OmniOS-discuss] Windows 7 Guest KVM In-Reply-To: References: <8509AEF9-0330-474F-9249-8E2A951DD712@omniti.com> Message-ID: Tanks BOB, but I managed to figure it out. true what you say. I was using TrueVNC Viewer which doesn't work, tightVNC viewer worked for me. 0 viruses found. www.avast.com <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> *Miami *E. Kelvin IT Specialist, Software & Web Developer Mobile : (254) 710 430060 P.O Box: 54481-00200 Nairobi, Kenya The future belongs to those who believe in the beauty of their dreams. On Thu, Mar 17, 2016 at 8:19 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Thu, 17 Mar 2016, Miami Kelvin wrote: > > I am connecting to the right port, in my case 5965, if I attempt to >> connect to any other port VNC would give an error message. But >> when I connect to IP:5965 VNC makes a connection the the window just >> disappears, so I thought it would be a problem with VGA >> support. >> > > What VNC client are you using? There seem to be quite a lot of them now. > Some VNC clients might not interoperate correctly since they rely on RFB > protocol extensions, or the server might require an extension. > > I suggest doing 'telnet hostname 5965' and see if you get the expected RFB > challenge prompt (see https://en.wikipedia.org/wiki/RFB_protocol). > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeffpc at josefsipek.net Fri Mar 18 15:12:38 2016 From: jeffpc at josefsipek.net (Josef 'Jeff' Sipek) Date: Fri, 18 Mar 2016 11:12:38 -0400 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: References: Message-ID: <20160318151237.GC1426@meili.valhalla.31bits.net> On Tue, Mar 15, 2016 at 15:40:44 +0000, Peter Tribble wrote: > Currently having a minor issue building software. > > As mentioned yesterday, I'm building on a slightly older image, so that I > know that any binaries I generate will run on all systems (which are newer). > > The snag is that the install media doesn't have the headers. And if I > > pkg install system/header > > then it installs the latest version in the repo. It doesn't look like there > are any versioned constraints to tie the version of the header package > to the installed libraries. I think it'd be fair to include a dependency in pkg:/system/library restrict which combo of headers & libc can be installed. E.g., something like: depend type=incorporate fmri=pkg:/system/header at 0.5.11-0.151016:2016... Dan & other IPS gurus: any thoughts about this? Jeff. > Specifically, the glob fix #6409 causes problems. Because I end up with > a set of headers that have the fix and a copy of libc that doesn't. But I'm > sure there might be other things lurking. > > My workaround is to manually look at the versions of system/library and > specify > the matching version of system/header to install by hand. Is there a better > way? > > This isn't helped by the way that pkg(5) goes and does an update if you > utter > 'pkg install', rather than just saying 'oh, you're already installed, we're > done'. > At least with the header package I can uninstall it and put the correct one > back, with some other packages I have to roll the system back. > > Thanks for any thoughts! > > -- > -Peter Tribble > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Once you have their hardware. Never give it back. (The First Rule of Hardware Acquisition) From danmcd at omniti.com Fri Mar 18 15:23:38 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 18 Mar 2016 11:23:38 -0400 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: <20160318151237.GC1426@meili.valhalla.31bits.net> References: <20160318151237.GC1426@meili.valhalla.31bits.net> Message-ID: <1D16D69F-366E-49F6-A5DE-A64866F40E2B@omniti.com> > On Mar 18, 2016, at 11:12 AM, Josef 'Jeff' Sipek wrote: > > I think it'd be fair to include a dependency in > pkg:/system/library restrict which combo of headers & libc can be installed. > E.g., something like: > > depend type=incorporate fmri=pkg:/system/header at 0.5.11-0.151016:2016... > > Dan & other IPS gurus: any thoughts about this? I'm wary of incorporate dependencies because of how they've been abused in non-gate IPS packages. Often those incorporate dependencies block upgrades. OTOH, system/library and system/header are peas in a pod, and live in the same build space, so it's possible this would be a good idea. Someone should experiment with that. I more than a coinflip's certain, however, it won't be making it into r151018. (i.e. Someone will have to strongly convince me otherwise.) Dan From jeffpc at josefsipek.net Fri Mar 18 16:00:23 2016 From: jeffpc at josefsipek.net (Josef 'Jeff' Sipek) Date: Fri, 18 Mar 2016 12:00:23 -0400 Subject: [OmniOS-discuss] header/library mismatch In-Reply-To: <1D16D69F-366E-49F6-A5DE-A64866F40E2B@omniti.com> References: <20160318151237.GC1426@meili.valhalla.31bits.net> <1D16D69F-366E-49F6-A5DE-A64866F40E2B@omniti.com> Message-ID: <20160318160023.GE1426@meili.valhalla.31bits.net> On Fri, Mar 18, 2016 at 11:23:38 -0400, Dan McDonald wrote: > > > On Mar 18, 2016, at 11:12 AM, Josef 'Jeff' Sipek wrote: > > > > I think it'd be fair to include a dependency in > > pkg:/system/library restrict which combo of headers & libc can be installed. > > E.g., something like: > > > > depend type=incorporate fmri=pkg:/system/header at 0.5.11-0.151016:2016... > > > > Dan & other IPS gurus: any thoughts about this? > > I'm wary of incorporate dependencies because of how they've been abused in > non-gate IPS packages. Often those incorporate dependencies block > upgrades. Yeah, incorporate dependencies have been abused by userland/etc. For this libc+header case specifically, of all the types of dependencies incorporate is the right one - this specific version, no more, no less. (Other types allow the depend target to be newer than the specified version which is what this would try to avoid.) > OTOH, system/library and system/header are peas in a pod, and live in the > same build space, so it's possible this would be a good idea. I actually think it would be a very good idea otherwise it's quite possible to end up with a mismatch between libc & the headers. If you are lucky, the user that runs into it figures it out. If you are unlucky, you'll get sucked in and likely waste a bit of time. Either way a simple dependency would have made it impossible to end up with the wrong package combo. Or did I misunderstand and mismatched libc & headers are supported by OmniOS? ;) > Someone should experiment with that. I more than a coinflip's certain, > however, it won't be making it into r151018. (i.e. Someone will have to > strongly convince me otherwise.) That's fine. I could see this becoming a thrill-seeker^W^Wbite-sized bug. Jeff. -- Humans were created by water to transport it upward. From jimklimov at cos.ru Fri Mar 18 16:44:36 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 18 Mar 2016 17:44:36 +0100 Subject: [OmniOS-discuss] User/group accounts for packaged daemons Message-ID: <82B56A13-D365-4BBA-8C3D-6BE8B77A6CED@cos.ru> Hello all, With turtle speed I'm progressing to recipe the open-source stacks I used in sysadmin work, such as antispam relays. I'm working at the oi-userland in Hipster, and hopefully the good results can end up anywhere ;) A solution of this sort involves running a number of services, such as a stack of milters, an antivirus engine, a sniffer (p0f), etc. - some with special privileges and constraints, and thus preferably different accounts, so possible security issues with one project do not let break into others. While some services might be generalized as 'mail' or 'antivir' accounts, it is not always good and safe to do so. The illumos default UIDs and GIDs generally reserve numbers under 100 and above somewhere around 60000. While there are Wiki pages for illumos and OI to list the well-known and occupied "system" account numbers and names, I'm not sure there is a procedure to claim and reserve the number so as to avoid conflicts. So I was advised to ask around the community: are there any established ways to proceed here? What can be done to cook those packages well in practice? Grabbing random free uids/gids does not seem good, especially since (in my tests) the package got numbers under 100 that might clash with later installs of software that has valid fixed ID numbers. On a side note, how do we uninstall or update IPS packages where software can create files, and we have no 'preremove' script goodness? :-) For example, clamav downloads virus databases; when I try to uninstall it, there are 'lost+found' objects owned by its uid/gid, and so it can not undefine the uid/gid and remove the delivered working directory actions - so bam, pkg(5) fails either now or upon subsequent re-installation! Also, in case of updates at least, i'd prefer to keep the bulky downloaded files in place, to save both on traffic and storage... Sorry for the cluttered question, but I hope you get its point ;) Thanks in advance, Jim Klimov -- Typos courtesy of K-9 Mail on my Samsung Android From bfriesen at simple.dallas.tx.us Fri Mar 18 17:24:46 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 18 Mar 2016 12:24:46 -0500 (CDT) Subject: [OmniOS-discuss] User/group accounts for packaged daemons In-Reply-To: <82B56A13-D365-4BBA-8C3D-6BE8B77A6CED@cos.ru> References: <82B56A13-D365-4BBA-8C3D-6BE8B77A6CED@cos.ru> Message-ID: On Fri, 18 Mar 2016, Jim Klimov wrote: > A solution of this sort involves running a number of services, such as a stack of milters, an antivirus engine, a sniffer (p0f), etc. - some with special privileges and constraints, and thus preferably different accounts, so possible security issues with one project do not let break into others. While some services might be generalized as 'mail' or 'antivir' accounts, it is not always good and safe to do so. > > The illumos default UIDs and GIDs generally reserve numbers under > 100 and above somewhere around 60000. While there are Wiki pages for > illumos and OI to list the well-known and occupied "system" account > numbers and names, I'm not sure there is a procedure to claim and > reserve the number so as to avoid conflicts. I already encountered a conflict when OmniOS introduced OpenSSH and used the user id used by another add-on package for it. Due to this, I investigated the user id used by the SFE version of the package and used that. The SFE versions should at least not conflict with user ids used by Oracle Solaris 11 packages. > On a side note, how do we uninstall or update IPS packages where software can create files, and we have no 'preremove' script goodness? :-) >From what I have read, while there is no script goodness associated with IPS packages, there is the ability to run a script when a service manifest is installed or removed. As long as each package provides its own service manifest, then it should be possible to remove the junk when the associated service manifest is removed. It would indeed be useful if there was a UID/GID registery for add-on software and managed by the Illumos project (even if just in a Git repository). These should try not to conflict with what Oracle Solaris 10/11 and stable OpenIndiana are already using for similar packages. Guidance should be taken from SFE, which has already needed to deal with conflicts. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From matej at zunaj.si Fri Mar 18 20:28:21 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Fri, 18 Mar 2016 21:28:21 +0100 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: <65DC5816D4BEE043885A89FD54E273FC6CF8DD35@MAIL101.Ellipseinc.com> References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> <4C9DDF4C-9E05-4027-A6A7-56658D8F5E6E@omniti.com> <65DC5816D4BEE043885A89FD54E273FC6CF8DD35@MAIL101.Ellipseinc.com> Message-ID: <94FAA70A-A460-4BBB-A8B2-C33E82BBE8CB@zunaj.si> I tried diskmap.py on a SAS3 LSI card today and it works, I just neede to change the path to sas ircu utils so it runs sas3ircu instead of sas2ircu. Also, sas3ircu from lsi/avago page doesn?t work, you need an older version (5.00). You can find it on nexenta?s github: https://github.com/Nexenta/nza-userland/tree/master/nza-userland/components/sas3ircu/usr/sbin lp, Matej > On 03 Mar 2016, at 19:56, Richard Jahnel wrote: > > Speaking of which, does anyone have a diskmap.py that works with the sas3 lsi cards? > > Richard Jahnel > Network Engineer > On-Site.com | Ellipse Design > 866-266-7483 ext. 4408 > Direct: 669-800-6270 > > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Dan McDonald > Sent: Thursday, March 03, 2016 12:54 PM > To: F?bio Rabelo ; Dan McDonald > Cc: omnios-discuss > Subject: Re: [OmniOS-discuss] LSI3108 > > >> On Mar 3, 2016, at 1:46 PM, F?bio Rabelo wrote: >> >> >> I have a customer who wants to build a pure SSD Storage with Sandisk >> 12 Gb/s SAS interface ! > > Get an LSI 3008-based card. The 9300, for example. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From rjahnel at ellipseinc.com Fri Mar 18 21:19:04 2016 From: rjahnel at ellipseinc.com (Richard Jahnel) Date: Fri, 18 Mar 2016 21:19:04 +0000 Subject: [OmniOS-discuss] LSI3108 In-Reply-To: <94FAA70A-A460-4BBB-A8B2-C33E82BBE8CB@zunaj.si> References: <56D84761.1010808@dojcak.sk> <6B52063F-1F72-4DCF-9494-99E40DAC758E@omniti.com> <4C9DDF4C-9E05-4027-A6A7-56658D8F5E6E@omniti.com> <65DC5816D4BEE043885A89FD54E273FC6CF8DD35@MAIL101.Ellipseinc.com> <94FAA70A-A460-4BBB-A8B2-C33E82BBE8CB@zunaj.si> Message-ID: <65DC5816D4BEE043885A89FD54E273FC71CA7186@MAIL101.Ellipseinc.com> You?re the best :) Richard Jahnel Network Engineer On-Site.com | Ellipse Design 866-266-7483 ext. 4408 Direct: 669-800-6270 -----Original Message----- From: Matej ?erovnik [mailto:matej at zunaj.si] Sent: Friday, March 18, 2016 3:28 PM To: Richard Jahnel Cc: Dan McDonald ; F?bio Rabelo ; omnios-discuss Subject: Re: [OmniOS-discuss] LSI3108 I tried diskmap.py on a SAS3 LSI card today and it works, I just neede to change the path to sas ircu utils so it runs sas3ircu instead of sas2ircu. Also, sas3ircu from lsi/avago page doesn?t work, you need an older version (5.00). You can find it on nexenta?s github: https://github.com/Nexenta/nza-userland/tree/master/nza-userland/components/sas3ircu/usr/sbin lp, Matej > On 03 Mar 2016, at 19:56, Richard Jahnel wrote: > > Speaking of which, does anyone have a diskmap.py that works with the sas3 lsi cards? > > Richard Jahnel > Network Engineer > On-Site.com | Ellipse Design > 866-266-7483 ext. 4408 > Direct: 669-800-6270 > > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Dan McDonald > Sent: Thursday, March 03, 2016 12:54 PM > To: F?bio Rabelo ; Dan McDonald > Cc: omnios-discuss > Subject: Re: [OmniOS-discuss] LSI3108 > > >> On Mar 3, 2016, at 1:46 PM, F?bio Rabelo wrote: >> >> >> I have a customer who wants to build a pure SSD Storage with Sandisk >> 12 Gb/s SAS interface ! > > Get an LSI 3008-based card. The 9300, for example. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From geoffn at gnaa.net Fri Mar 18 23:12:38 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Fri, 18 Mar 2016 16:12:38 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system Message-ID: <56EC8B66.8050609@gnaa.net> Hi. I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? I will be installing the latest version of Ominos. thanks, Geoff From jstockett at molalla.com Fri Mar 18 23:30:45 2016 From: jstockett at molalla.com (Jeff Stockett) Date: Fri, 18 Mar 2016 23:30:45 +0000 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56EC8B66.8050609@gnaa.net> References: <56EC8B66.8050609@gnaa.net> Message-ID: <136C13E89D22BB468B2A7025993639733EBAB920@EXMCCMB.molalla.com> We have been using the bigger version of that - the 5048R-E1CR36L that also has the 3008 on the mobo with 4TB WD enterprise SAS drives for over a year with no problems to report other than once when a drive failed, the system hung up rather than handling it gracefully. We are not running the latest version of OmniOS yet but I'd assume it should work fine. We are about to build a couple more with the 8TB HGST SAS drives. -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Geoff Nordli Sent: Friday, March 18, 2016 4:13 PM To: omnios-discuss Subject: [OmniOS-discuss] supermicro 3U all-in one storage system Hi. I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? I will be installing the latest version of Ominos. thanks, Geoff _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From matej at zunaj.si Sat Mar 19 11:21:21 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Sat, 19 Mar 2016 12:21:21 +0100 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56EC8B66.8050609@gnaa.net> References: <56EC8B66.8050609@gnaa.net> Message-ID: <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> Hey there, I?m just about to expand my SAN with LSI 3008 HBA. As far as I know, support for 3008 has been in Illumos for about a year if not more, so it should be stable. I did have problems with getting sas3ircu and sas3flash utils from LSI page working, but in the end, I had to get an older version (5.00) for it to work (I got it from nexenta github repo). As far as HDD goes, I?m currently running around 100 Seagate SAS3 drives (ST4000NM0034) and none of them died in 4 months that I have them. I know it?s not a long era, but so far so good:) I also have around 180 Seagate Constellation 4TB SATA drives (ST4000NM0033) for over 3 years and only one or two drives died. Matej > On 19 Mar 2016, at 00:12, Geoff Nordli wrote: > > Hi. > > I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. > > Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. > > I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? > > I will be installing the latest version of Ominos. > > thanks, > > Geoff > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From geoffn at gnaa.net Sat Mar 19 18:19:13 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Sat, 19 Mar 2016 11:19:13 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> References: <56EC8B66.8050609@gnaa.net> <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> Message-ID: <56ED9821.5080409@gnaa.net> I have had really good luck with the Seagate drives as well. I see you are running the ST4000NM0034, any reason why you didn't go with the ST4000NM0014 which is 4Kn? thanks, Geoff On 16-03-19 04:21 AM, Matej ?erovnik wrote: > Hey there, > > I?m just about to expand my SAN with LSI 3008 HBA. As far as I know, support for 3008 has been in Illumos for about a year if not more, so it should be stable. I did have problems with getting sas3ircu and sas3flash utils from LSI page working, but in the end, I had to get an older version (5.00) for it to work (I got it from nexenta github repo). > > As far as HDD goes, I?m currently running around 100 Seagate SAS3 drives (ST4000NM0034) and none of them died in 4 months that I have them. I know it?s not a long era, but so far so good:) I also have around 180 Seagate Constellation 4TB SATA drives (ST4000NM0033) for over 3 years and only one or two drives died. > > Matej > >> On 19 Mar 2016, at 00:12, Geoff Nordli wrote: >> >> Hi. >> >> I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. >> >> Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. >> >> I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? >> >> I will be installing the latest version of Ominos. >> >> thanks, >> >> Geoff >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Sat Mar 19 20:28:38 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Sat, 19 Mar 2016 13:28:38 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56EC8B66.8050609@gnaa.net> References: <56EC8B66.8050609@gnaa.net> Message-ID: <801B1A61-1774-4D0B-9472-12AAAC499752@richardelling.com> > On Mar 18, 2016, at 4:12 PM, Geoff Nordli wrote: > > Hi. > > I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. We have several similar servers. The X10DRH is fine. For a non-HA system, single expander backplane is ok (BPN-SAS3-836EL1). > > Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. We have many variants of these parts. All work fine. > > I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? HGST or Seagate 4TB both seem to work fine. For Seagate, you'll want firmware 0004 or later, but that should be all that is in the supply chain for the past year. I avoid WD. -- richard > > I will be installing the latest version of Ominos. > > thanks, > > Geoff > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From matej at zunaj.si Sun Mar 20 08:15:54 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Sun, 20 Mar 2016 09:15:54 +0100 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56ED9821.5080409@gnaa.net> References: <56EC8B66.8050609@gnaa.net> <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> <56ED9821.5080409@gnaa.net> Message-ID: <01B4C7A0-D46C-4C60-888B-0D3488E52B11@zunaj.si> As far as I see, there are no 4Kn 4TB SAS drives, only 6TB. From performance view, is there any different between 4Kn or 512e that is formated with ashift=12? Matej > On 19 Mar 2016, at 19:19, Geoff Nordli wrote: > > I have had really good luck with the Seagate drives as well. > > I see you are running the ST4000NM0034, any reason why you didn't go with the ST4000NM0014 which is 4Kn? > > thanks, > > Geoff > > > On 16-03-19 04:21 AM, Matej ?erovnik wrote: >> Hey there, >> >> I?m just about to expand my SAN with LSI 3008 HBA. As far as I know, support for 3008 has been in Illumos for about a year if not more, so it should be stable. I did have problems with getting sas3ircu and sas3flash utils from LSI page working, but in the end, I had to get an older version (5.00) for it to work (I got it from nexenta github repo). >> >> As far as HDD goes, I?m currently running around 100 Seagate SAS3 drives (ST4000NM0034) and none of them died in 4 months that I have them. I know it?s not a long era, but so far so good:) I also have around 180 Seagate Constellation 4TB SATA drives (ST4000NM0033) for over 3 years and only one or two drives died. >> >> Matej >> >>> On 19 Mar 2016, at 00:12, Geoff Nordli wrote: >>> >>> Hi. >>> >>> I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. >>> >>> Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. >>> >>> I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? >>> >>> I will be installing the latest version of Ominos. >>> >>> thanks, >>> >>> Geoff >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From geoffn at gnaa.net Sun Mar 20 20:33:54 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Sun, 20 Mar 2016 13:33:54 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 Message-ID: <56EF0932.2030100@gnaa.net> Quick, question. Any performance differences between 4Kn and 512e with ashift=12? I am thinking there should not be, since they will both write the full 4K block size. The workload will be virtual machines using a zvol. thanks, Geoff From geoffn at gnaa.net Sun Mar 20 20:41:10 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Sun, 20 Mar 2016 13:41:10 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <01B4C7A0-D46C-4C60-888B-0D3488E52B11@zunaj.si> References: <56EC8B66.8050609@gnaa.net> <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> <56ED9821.5080409@gnaa.net> <01B4C7A0-D46C-4C60-888B-0D3488E52B11@zunaj.si> Message-ID: <56EF0AE6.5070504@gnaa.net> http://www.newegg.com/Product/Product.aspx?Item=9SIA5EM2KP1657 Also, I created a new thread about the differences with 4kn and 512e. Logically, it would make sense the performance would be the same. I think it would then depend on which one is cheaper. thanks, Geoff On 16-03-20 01:15 AM, Matej ?erovnik wrote: > As far as I see, there are no 4Kn 4TB SAS drives, only 6TB. > > From performance view, is there any different between 4Kn or 512e that is formated with ashift=12? > > Matej > >> On 19 Mar 2016, at 19:19, Geoff Nordli wrote: >> >> I have had really good luck with the Seagate drives as well. >> >> I see you are running the ST4000NM0034, any reason why you didn't go with the ST4000NM0014 which is 4Kn? >> >> thanks, >> >> Geoff >> >> >> On 16-03-19 04:21 AM, Matej ?erovnik wrote: >>> Hey there, >>> >>> I?m just about to expand my SAN with LSI 3008 HBA. As far as I know, support for 3008 has been in Illumos for about a year if not more, so it should be stable. I did have problems with getting sas3ircu and sas3flash utils from LSI page working, but in the end, I had to get an older version (5.00) for it to work (I got it from nexenta github repo). >>> >>> As far as HDD goes, I?m currently running around 100 Seagate SAS3 drives (ST4000NM0034) and none of them died in 4 months that I have them. I know it?s not a long era, but so far so good:) I also have around 180 Seagate Constellation 4TB SATA drives (ST4000NM0033) for over 3 years and only one or two drives died. >>> >>> Matej >>> >>>> On 19 Mar 2016, at 00:12, Geoff Nordli wrote: >>>> >>>> Hi. >>>> >>>> I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. >>>> >>>> Thoughts on the http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. It has the LSI 3008 HBA and Intel x540 network cards. >>>> >>>> I want to get at least 4TB 3.5" SAS drives. Any suggestions on those? >>>> >>>> I will be installing the latest version of Ominos. >>>> >>>> thanks, >>>> >>>> Geoff >>>> >>>> From Fred_Liu at issi.com Mon Mar 21 00:21:41 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Sun, 20 Mar 2016 17:21:41 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <56EF0932.2030100@gnaa.net> References: <56EF0932.2030100@gnaa.net> Message-ID: <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> The ashift parameter doesn't apply in illumos if my memory serves me well. It was introduced by ZoL. Thanks. Fred On Sun, Mar 20, 2016 at 1:37 PM -0700, "Geoff Nordli" > wrote: Quick, question. Any performance differences between 4Kn and 512e with ashift=12? I am thinking there should not be, since they will both write the full 4K block size. The workload will be virtual machines using a zvol. thanks, Geoff _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoffn at gnaa.net Mon Mar 21 02:59:19 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Sun, 20 Mar 2016 19:59:19 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> Message-ID: <56EF6387.5080404@gnaa.net> Hi Fred. Definitely, the ashift parameter has been around on the illumos for a couple of years. thanks, Geoff On 16-03-20 05:21 PM, Fred Liu wrote: > The ashift parameter doesn't apply in illumos if my memory serves me > well. It was introduced by ZoL. > > Thanks. > > Fred > > > > > > > > On Sun, Mar 20, 2016 at 1:37 PM -0700, "Geoff Nordli" > wrote: > > Quick, question. > > Any performance differences between 4Kn and 512e with ashift=12? > > I am thinking there should not be, since they will both write the full > 4K block size. > > The workload will be virtual machines using a zvol. > > thanks, > > Geoff > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From Fred_Liu at issi.com Mon Mar 21 05:00:13 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Sun, 20 Mar 2016 22:00:13 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <56EF6387.5080404@gnaa.net> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> Message-ID: So, you can ?zpool create ?o ashift=12? in illumos? I can?t do that in smartos at least?. From: Geoff Nordli [mailto:geoff.nordli at gmail.com] On Behalf Of Geoff Nordli Sent: ???, ?? 21, 2016 10:59 To: Fred Liu; omnios-discuss Cc: zfs-discuss at list.zfsonlinux.org Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 Hi Fred. Definitely, the ashift parameter has been around on the illumos for a couple of years. thanks, Geoff On 16-03-20 05:21 PM, Fred Liu wrote: The ashift parameter doesn't apply in illumos if my memory serves me well. It was introduced by ZoL. Thanks. Fred On Sun, Mar 20, 2016 at 1:37 PM -0700, "Geoff Nordli" > wrote: Quick, question. Any performance differences between 4Kn and 512e with ashift=12? I am thinking there should not be, since they will both write the full 4K block size. The workload will be virtual machines using a zvol. thanks, Geoff _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From matej at zunaj.si Mon Mar 21 06:19:38 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Mon, 21 Mar 2016 07:19:38 +0100 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> Message-ID: <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> You can?t do that, but you can force system to recognize hard drive as if it was 4Kn. Just like we do it for SSDs: http://wiki.illumos.org/display/illumos/List+of+sd-config-list+entries+for+Advanced-Format+drives lp, Matej > On 21 Mar 2016, at 06:00, Fred Liu wrote: > > So, you can ?zpool create ?o ashift=12? in illumos? I can?t do that in smartos at least?. > > From: Geoff Nordli [mailto:geoff.nordli at gmail.com ] On Behalf Of Geoff Nordli > Sent: ???, ?? 21, 2016 10:59 > To: Fred Liu; omnios-discuss > Cc: zfs-discuss at list.zfsonlinux.org > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > Hi Fred. > > Definitely, the ashift parameter has been around on the illumos for a couple of years. > > thanks, > > Geoff > > On 16-03-20 05:21 PM, Fred Liu wrote: > The ashift parameter doesn't apply in illumos if my memory serves me well. It was introduced by ZoL. > > Thanks. > > Fred > > > > > > > > On Sun, Mar 20, 2016 at 1:37 PM -0700, "Geoff Nordli" > wrote: > > Quick, question. > > Any performance differences between 4Kn and 512e with ashift=12? > > I am thinking there should not be, since they will both write the full > 4K block size. > > The workload will be virtual machines using a zvol. > > thanks, > > Geoff > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3468 bytes Desc: not available URL: From Fred_Liu at issi.com Mon Mar 21 07:00:59 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Mon, 21 Mar 2016 00:00:59 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> Message-ID: So that means illumos can handle 512n and 4kn automatically and properly? Thanks. Fred From: Matej ?erovnik [mailto:matej at zunaj.si] Sent: ???, ?? 21, 2016 14:20 To: Fred Liu Cc: geoffn at gnaa.net; omnios-discuss; zfs-discuss at list.zfsonlinux.org Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 You can?t do that, but you can force system to recognize hard drive as if it was 4Kn. Just like we do it for SSDs: http://wiki.illumos.org/display/illumos/List+of+sd-config-list+entries+for+Advanced-Format+drives lp, Matej On 21 Mar 2016, at 06:00, Fred Liu > wrote: So, you can ?zpool create ?o ashift=12? in illumos? I can?t do that in smartos at least?. From: Geoff Nordli [mailto:geoff.nordli at gmail.com] On Behalf Of Geoff Nordli Sent: ???, ?? 21, 2016 10:59 To: Fred Liu; omnios-discuss Cc: zfs-discuss at list.zfsonlinux.org Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 Hi Fred. Definitely, the ashift parameter has been around on the illumos for a couple of years. thanks, Geoff On 16-03-20 05:21 PM, Fred Liu wrote: The ashift parameter doesn't apply in illumos if my memory serves me well. It was introduced by ZoL. Thanks. Fred On Sun, Mar 20, 2016 at 1:37 PM -0700, "Geoff Nordli" > wrote: Quick, question. Any performance differences between 4Kn and 512e with ashift=12? I am thinking there should not be, since they will both write the full 4K block size. The workload will be virtual machines using a zvol. thanks, Geoff _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimklimov at cos.ru Mon Mar 21 08:39:43 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Mon, 21 Mar 2016 09:39:43 +0100 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56EF0AE6.5070504@gnaa.net> References: <56EC8B66.8050609@gnaa.net> <4C64B9B0-3F0A-4A2C-BD7B-78BE72E434F9@zunaj.si> <56ED9821.5080409@gnaa.net> <01B4C7A0-D46C-4C60-888B-0D3488E52B11@zunaj.si> <56EF0AE6.5070504@gnaa.net> Message-ID: <141A0068-7F78-40AB-A853-A775D52303A4@cos.ru> 20 ????? 2016??. 21:41:10 CET, Geoff Nordli ?????: > >http://www.newegg.com/Product/Product.aspx?Item=9SIA5EM2KP1657 > >Also, I created a new thread about the differences with 4kn and 512e. >Logically, it would make sense the performance would be the same. I >think it would then depend on which one is cheaper. > >thanks, > >Geoff > > >On 16-03-20 01:15 AM, Matej ?erovnik wrote: >> As far as I see, there are no 4Kn 4TB SAS drives, only 6TB. >> >> From performance view, is there any different between 4Kn or 512e >that is formated with ashift=12? >> >> Matej >> >>> On 19 Mar 2016, at 19:19, Geoff Nordli wrote: >>> >>> I have had really good luck with the Seagate drives as well. >>> >>> I see you are running the ST4000NM0034, any reason why you didn't go >with the ST4000NM0014 which is 4Kn? >>> >>> thanks, >>> >>> Geoff >>> >>> >>> On 16-03-19 04:21 AM, Matej ?erovnik wrote: >>>> Hey there, >>>> >>>> I?m just about to expand my SAN with LSI 3008 HBA. As far as I >know, support for 3008 has been in Illumos for about a year if not >more, so it should be stable. I did have problems with getting sas3ircu >and sas3flash utils from LSI page working, but in the end, I had to get >an older version (5.00) for it to work (I got it from nexenta github >repo). >>>> >>>> As far as HDD goes, I?m currently running around 100 Seagate SAS3 >drives (ST4000NM0034) and none of them died in 4 months that I have >them. I know it?s not a long era, but so far so good:) I also have >around 180 Seagate Constellation 4TB SATA drives (ST4000NM0033) for >over 3 years and only one or two drives died. >>>> >>>> Matej >>>> >>>>> On 19 Mar 2016, at 00:12, Geoff Nordli wrote: >>>>> >>>>> Hi. >>>>> >>>>> I have had good luck with the SuperStorage 6037R-E1R16L chassis >with the LSI 2308 IT mode HBA. >>>>> >>>>> Thoughts on the >http://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm. >It has the LSI 3008 HBA and Intel x540 network cards. >>>>> >>>>> I want to get at least 4TB 3.5" SAS drives. Any suggestions on >those? >>>>> >>>>> I will be installing the latest version of Ominos. >>>>> >>>>> thanks, >>>>> >>>>> Geoff >>>>> >>>>> > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss I'd guess 4kn drives have 4k native sectors and expose them as such. 512e drives have 4kn sectors and expose RME ability to write smaller sectors for backwards compatibility. 512n drives have 512byte sectors (with ecc for each 512b, not 4kb as the two above). Then it depends if alignment etc. do not cause 512e disks to go into rmw - otherwise it "should be" same as 4kn for 4k*N writes. Depends on vendor firmware from here onwards ;) Jim -- Typos courtesy of K-9 Mail on my Samsung Android From hannohirschberger at googlemail.com Mon Mar 21 09:02:03 2016 From: hannohirschberger at googlemail.com (Hanno Hirschberger) Date: Mon, 21 Mar 2016 10:02:03 +0100 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> Message-ID: <56EFB88B.9030304@googlemail.com> On 21.03.2016 08:00, Fred Liu wrote: > So that means illumos can handle 512n and 4kn automatically and properly? Not necessarily as far as I know. Sometime drives are emulating 512 blocks and don't properly tell the OS about that and Illumos ZFS is aligning the drives with ashift=9 which leads to enormous performance issues. Also forcing the system to handle drives with a specific sector size with the sd.conf doesn't turn out to be reliable in some cases (at least on my workstations). Here's what I do to ensure ashift=12 values: Reboot the system with a Linux live disk of your choice and install ZoL in the live session. Then create the ZFS pool, export it and reboot the machine. OmniOS / Illumos can import the new pool without problems and the ashift value is correctly set. There was a fixed zpool binary (Solaris 11 binary) flying around the internet which can handle the "-o shift=12" parameter and works with OmniOS but unfortunately I can't find it again right now. This would make the reboot into a live session obsolete. Does anyone know if the "ashift" parameter will be implemented in the OmniOS / Illumos zpool binary in the near future? Best regards Hanno From jimklimov at cos.ru Mon Mar 21 18:11:06 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Mon, 21 Mar 2016 19:11:06 +0100 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <56EFB88B.9030304@googlemail.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> Message-ID: <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> 21 ????? 2016??. 10:02:03 CET, Hanno Hirschberger ?????: >On 21.03.2016 08:00, Fred Liu wrote: >> So that means illumos can handle 512n and 4kn automatically and >properly? > >Not necessarily as far as I know. Sometime drives are emulating 512 >blocks and don't properly tell the OS about that and Illumos ZFS is >aligning the drives with ashift=9 which leads to enormous performance >issues. Also forcing the system to handle drives with a specific sector > >size with the sd.conf doesn't turn out to be reliable in some cases (at > >least on my workstations). Here's what I do to ensure ashift=12 values: > >Reboot the system with a Linux live disk of your choice and install ZoL > >in the live session. Then create the ZFS pool, export it and reboot the > >machine. OmniOS / Illumos can import the new pool without problems and >the ashift value is correctly set. There was a fixed zpool binary >(Solaris 11 binary) flying around the internet which can handle the "-o > >shift=12" parameter and works with OmniOS but unfortunately I can't >find >it again right now. This would make the reboot into a live session >obsolete. > >Does anyone know if the "ashift" parameter will be implemented in the >OmniOS / Illumos zpool binary in the near future? > >Best regards > >Hanno >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Adding the ashift argument to zpool was discussed every few years and so far was always deemed not enterprisey enough for the Solaris heritage, so the setup to tweak sd driver reports and properly rely on that layer was pushed instead. That said, the old tweaked binary came with a blog post detailing the source changes; you're welcome to try a d port and rti it (I'd say there is enough user demand to back the non-enterprisey fix to be on par with other OpenZFS siblings). At worst, you can publish the modernized binary as the original blogger did ;) Jim -- Typos courtesy of K-9 Mail on my Samsung Android From richard.elling at richardelling.com Mon Mar 21 18:53:46 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Mon, 21 Mar 2016 11:53:46 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> Message-ID: <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> > On Mar 21, 2016, at 11:11 AM, Jim Klimov wrote: > > 21 ????? 2016 ?. 10:02:03 CET, Hanno Hirschberger ?????: >> On 21.03.2016 08:00, Fred Liu wrote: >>> So that means illumos can handle 512n and 4kn automatically and >> properly? >> >> Not necessarily as far as I know. Sometime drives are emulating 512 >> blocks and don't properly tell the OS about that and Illumos ZFS is >> aligning the drives with ashift=9 which leads to enormous performance >> issues. Also forcing the system to handle drives with a specific sector >> >> size with the sd.conf doesn't turn out to be reliable in some cases (at >> >> least on my workstations). Here's what I do to ensure ashift=12 values: >> >> Reboot the system with a Linux live disk of your choice and install ZoL >> >> in the live session. Then create the ZFS pool, export it and reboot the >> >> machine. OmniOS / Illumos can import the new pool without problems and >> the ashift value is correctly set. There was a fixed zpool binary >> (Solaris 11 binary) flying around the internet which can handle the "-o >> >> shift=12" parameter and works with OmniOS but unfortunately I can't >> find >> it again right now. This would make the reboot into a live session >> obsolete. >> >> Does anyone know if the "ashift" parameter will be implemented in the >> OmniOS / Illumos zpool binary in the near future? >> >> Best regards >> >> Hanno >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > Adding the ashift argument to zpool was discussed every few years and so far was always deemed not enterprisey enough for the Solaris heritage, so the setup to tweak sd driver reports and properly rely on that layer was pushed instead. The issue is that once a drive model lies, then the Solaris approach is to encode the lie into a whitelist, so that the lie is always handled properly. The whitelist is in the sd.conf file. By contrast, the ZFSonLinux approach requires that the sysadmin knows there is a lie and manually corrects for every invocation. This is unfortunately a very error-prone approach. -- richard > > That said, the old tweaked binary came with a blog post detailing the source changes; you're welcome to try a d port and rti it (I'd say there is enough user demand to back the non-enterprisey fix to be on par with other OpenZFS siblings). At worst, you can publish the modernized binary as the original blogger did ;) > > Jim > -- > Typos courtesy of K-9 Mail on my Samsung Android > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From bfriesen at simple.dallas.tx.us Mon Mar 21 19:19:06 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Mon, 21 Mar 2016 14:19:06 -0500 (CDT) Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> Message-ID: On Mon, 21 Mar 2016, Richard Elling wrote: >> >> Adding the ashift argument to zpool was discussed every few years and so far was always deemed not enterprisey enough for the Solaris heritage, so the setup to tweak sd driver reports and properly rely on that layer was pushed instead. > > The issue is that once a drive model lies, then the Solaris approach is to encode > the lie into a whitelist, so that the lie is always handled properly. The whitelist is in the > sd.conf file. Does this approach require that Illumos users only use drive hardware much older than the version of Illumos they happen running since outdated whitelist won't know about the new lies? What if a user is using classic drives but wants to be prepared to install newer drives which require ashift=12? Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From richard.elling at richardelling.com Mon Mar 21 19:19:19 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Mon, 21 Mar 2016 12:19:19 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <65DC5816D4BEE043885A89FD54E273FC71CA96A3@MAIL101.Ellipseinc.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> <65DC5816D4BEE043885A89FD54E273FC71CA96A3@MAIL101.Ellipseinc.com> Message-ID: <30FBDF2F-E7B7-4302-8B09-1ED8E53027B4@richardelling.com> > On Mar 21, 2016, at 12:00 PM, Richard Jahnel wrote: > > Both approaches have their error points. > > FWIW I would very very much like to be able to force my new pools into ashift=12. It would make drive purchasing and replacement a lot more straight forward and future resistant. The fundamental problem is that this is a vdev-settable, not a pool settable. Today, it is very common for pools to have multiple ashifts active. Recently, per-vdev ZAP objects have been proposed and that code is working its way through the review and integration process. Meanwhile, use one or more of the dozens of methods documented for doing this. FWIW, most people with HDDs want space efficiency, because HDDs lost the performance race to SSDs long ago. In general, forcing everything to 4k reduces space efficiency, so it is unlikely to be a good default. -- richard > > Regards > > Richard Jahnel > Network Engineer > On-Site.com | Ellipse Design > 866-266-7483 ext. 4408 > Direct: 669-800-6270 > > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Richard Elling > Sent: Monday, March 21, 2016 1:54 PM > To: Jim Klimov > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > >> On Mar 21, 2016, at 11:11 AM, Jim Klimov wrote: >> >> 21 ????? 2016 ?. 10:02:03 CET, Hanno Hirschberger ?????: >>> On 21.03.2016 08:00, Fred Liu wrote: >>>> So that means illumos can handle 512n and 4kn automatically and >>> properly? >>> >>> Not necessarily as far as I know. Sometime drives are emulating 512 >>> blocks and don't properly tell the OS about that and Illumos ZFS is >>> aligning the drives with ashift=9 which leads to enormous performance >>> issues. Also forcing the system to handle drives with a specific >>> sector >>> >>> size with the sd.conf doesn't turn out to be reliable in some cases >>> (at >>> >>> least on my workstations). Here's what I do to ensure ashift=12 values: >>> >>> Reboot the system with a Linux live disk of your choice and install >>> ZoL >>> >>> in the live session. Then create the ZFS pool, export it and reboot >>> the >>> >>> machine. OmniOS / Illumos can import the new pool without problems >>> and the ashift value is correctly set. There was a fixed zpool binary >>> (Solaris 11 binary) flying around the internet which can handle the >>> "-o >>> >>> shift=12" parameter and works with OmniOS but unfortunately I can't >>> find it again right now. This would make the reboot into a live >>> session obsolete. >>> >>> Does anyone know if the "ashift" parameter will be implemented in the >>> OmniOS / Illumos zpool binary in the near future? >>> >>> Best regards >>> >>> Hanno >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> Adding the ashift argument to zpool was discussed every few years and so far was always deemed not enterprisey enough for the Solaris heritage, so the setup to tweak sd driver reports and properly rely on that layer was pushed instead. > > The issue is that once a drive model lies, then the Solaris approach is to encode the lie into a whitelist, so that the lie is always handled properly. The whitelist is in the sd.conf file. > > By contrast, the ZFSonLinux approach requires that the sysadmin knows there is a lie and manually corrects for every invocation. This is unfortunately a very error-prone approach. > -- richard > >> >> That said, the old tweaked binary came with a blog post detailing the >> source changes; you're welcome to try a d port and rti it (I'd say >> there is enough user demand to back the non-enterprisey fix to be on >> par with other OpenZFS siblings). At worst, you can publish the >> modernized binary as the original blogger did ;) >> >> Jim >> -- >> Typos courtesy of K-9 Mail on my Samsung Android >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Mon Mar 21 19:21:03 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Mon, 21 Mar 2016 12:21:03 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> Message-ID: > On Mar 21, 2016, at 12:19 PM, Bob Friesenhahn wrote: > > On Mon, 21 Mar 2016, Richard Elling wrote: >>> >>> Adding the ashift argument to zpool was discussed every few years and so far was always deemed not enterprisey enough for the Solaris heritage, so the setup to tweak sd driver reports and properly rely on that layer was pushed instead. >> >> The issue is that once a drive model lies, then the Solaris approach is to encode >> the lie into a whitelist, so that the lie is always handled properly. The whitelist is in the >> sd.conf file. > > Does this approach require that Illumos users only use drive hardware much older than the version of Illumos they happen running since outdated whitelist won't know about the new lies? Forunately, lies are becoming less common. But this raises a good point: if your drive doesn't lie, then you don't need to workaround. > > What if a user is using classic drives but wants to be prepared to install newer drives which require ashift=12? See the bazillion other posts on this topic. -- richard > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From cks at cs.toronto.edu Mon Mar 21 19:26:56 2016 From: cks at cs.toronto.edu (Chris Siebenmann) Date: Mon, 21 Mar 2016 15:26:56 -0400 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: richard.elling's message of Mon, 21 Mar 2016 11:53:46 -0700. <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> Message-ID: <20160321192656.31FC97A059A@apps0.cs.toronto.edu> > > Adding the ashift argument to zpool was discussed every few years > > and so far was always deemed not enterprisey enough for the Solaris > > heritage, so the setup to tweak sd driver reports and properly rely > > on that layer was pushed instead. > > The issue is that once a drive model lies, then the Solaris approach > is to encode the lie into a whitelist, so that the lie is always > handled properly. The whitelist is in the sd.conf file. > > By contrast, the ZFSonLinux approach requires that the sysadmin knows > there is a lie and manually corrects for every invocation. This is > unfortunately a very error-prone approach. This implicitly assumes that the only reason to set ashift=12 is if you are currently using one or more drives that require it. I strongly disagree with this view. Since ZFS cannot currently replace a 512n drive with a 512e one, I feel that you really want to force-create all pools with ashift=12 even on 512n drives unless you're absolutely confident that you will be able to obtain replacement 512n drives over the lifetime of your pool (or unless the impact of using ashift=12 is so drastic on your usage case that you will need to totally rethink your pool if you become unable to get 512n replacement drives). For many usage cases, somewhat more space usage and perhaps somewhat slower pools are vastly preferable to a loss of pool redundancy over time. I feel that OmniOS should at least give you the option here (in a less crude way than simply telling it that absolutely all of your drives are 4k drives, partly because such general lies are problematic in various situations). - cks From geoffn at gnaa.net Mon Mar 21 22:07:42 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Mon, 21 Mar 2016 15:07:42 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <801B1A61-1774-4D0B-9472-12AAAC499752@richardelling.com> References: <56EC8B66.8050609@gnaa.net> <801B1A61-1774-4D0B-9472-12AAAC499752@richardelling.com> Message-ID: <56F070AE.201@gnaa.net> On 16-03-19 01:28 PM, Richard Elling wrote: >> On Mar 18, 2016, at 4:12 PM, Geoff Nordli wrote: >> >> Hi. >> >> I have had good luck with the SuperStorage 6037R-E1R16L chassis with the LSI 2308 IT mode HBA. > We have several similar servers. The X10DRH is fine. For a non-HA system, single expander backplane > is ok (BPN-SAS3-836EL1). Any thoughts on how to put 4 SATA drives in that chassis? 2 are easy, since they fit in the back of the server, but the other two would need to go in the front. Is the nvme driver stable enough yet to use as l2arc/slog? thanks, Geoff From Kevin.Swab at ColoState.EDU Mon Mar 21 23:14:43 2016 From: Kevin.Swab at ColoState.EDU (Kevin Swab) Date: Mon, 21 Mar 2016 17:14:43 -0600 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56F070AE.201@gnaa.net> References: <56EC8B66.8050609@gnaa.net> <801B1A61-1774-4D0B-9472-12AAAC499752@richardelling.com> <56F070AE.201@gnaa.net> Message-ID: <56F08063.5040805@ColoState.EDU> These things work with the ahci driver: http://www.addonics.com/products/ad4mspx2-a.php Performance is marginal for use as an l2arc or slog, but they're fine for installing the OS. Hot-swapping a failed device is tricky but possible - I use nylon screws in case I drop one on the MB... Kevin On 03/21/2016 04:07 PM, Geoff Nordli wrote: > On 16-03-19 01:28 PM, Richard Elling wrote: >>> On Mar 18, 2016, at 4:12 PM, Geoff Nordli wrote: >>> >>> Hi. >>> >>> I have had good luck with the SuperStorage 6037R-E1R16L chassis with >>> the LSI 2308 IT mode HBA. >> We have several similar servers. The X10DRH is fine. For a non-HA >> system, single expander backplane >> is ok (BPN-SAS3-836EL1). > > Any thoughts on how to put 4 SATA drives in that chassis? 2 are easy, > since they fit in the back of the server, but the other two would need > to go in the front. > > Is the nvme driver stable enough yet to use as l2arc/slog? > > thanks, > > Geoff > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- ------------------------------------------------------------------- Kevin Swab UNIX Systems Administrator ACNS Colorado State University Phone: (970)491-6572 Email: Kevin.Swab at ColoState.EDU GPG Fingerprint: 7026 3F66 A970 67BD 6F17 8EB8 8A7D 142F 2392 791C From Fred_Liu at issi.com Tue Mar 22 10:00:50 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Tue, 22 Mar 2016 03:00:50 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <20160321192656.31FC97A059A@apps0.cs.toronto.edu> References: richard.elling's message of Mon, 21 Mar 2016 11:53:46 -0700. <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> <20160321192656.31FC97A059A@apps0.cs.toronto.edu> Message-ID: > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On > Behalf Of Chris Siebenmann > Sent: ???, ?? 22, 2016 3:27 > To: Richard Elling > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > Adding the ashift argument to zpool was discussed every few years > > > and so far was always deemed not enterprisey enough for the Solaris > > > heritage, so the setup to tweak sd driver reports and properly rely > > > on that layer was pushed instead. > > > > The issue is that once a drive model lies, then the Solaris approach > > is to encode the lie into a whitelist, so that the lie is always > > handled properly. The whitelist is in the sd.conf file. > > > > By contrast, the ZFSonLinux approach requires that the sysadmin knows > > there is a lie and manually corrects for every invocation. This is > > unfortunately a very error-prone approach. > > This implicitly assumes that the only reason to set ashift=12 is if you are > currently using one or more drives that require it. I strongly disagree with this > view. Since ZFS cannot currently replace a 512n drive with a 512e one, I feel *In theory* this replacement should work well if the lie works *correctly*. In ZoL, for the "-o ashift" is supported in "zpool replace", the replacement should also work in mixed sector sizes. And in illumos the whitelist will do the same. What errors have you ever seen? > that you really want to force-create all pools with ashift=12 even on 512n drives > unless you're absolutely confident that you will be able to obtain replacement > 512n drives over the lifetime of your pool (or unless the impact of using > ashift=12 is so drastic on your usage case that you will need to totally rethink > your pool if you become unable to get 512n replacement drives). Same as my comments above. > > For many usage cases, somewhat more space usage and perhaps somewhat > slower pools are vastly preferable to a loss of pool redundancy over time. I feel > that OmniOS should at least give you the option here (in a less crude way than > simply telling it that absolutely all of your drives are 4k drives, partly because > such general lies are problematic in various situations). > The whitelist (sd.conf) should fit into this consideration. But not sure how mixed sector sizes impact the performance. Thanks. Fred From Fred_Liu at issi.com Tue Mar 22 10:13:29 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Tue, 22 Mar 2016 03:13:29 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <56EFB88B.9030304@googlemail.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> Message-ID: > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On > Behalf Of Hanno Hirschberger > Sent: ???, ?? 21, 2016 17:02 > To: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > On 21.03.2016 08:00, Fred Liu wrote: > > So that means illumos can handle 512n and 4kn automatically and properly? > > Not necessarily as far as I know. Sometime drives are emulating 512 blocks and > don't properly tell the OS about that and Illumos ZFS is aligning the drives with > ashift=9 which leads to enormous performance issues. Also forcing the system That is the goal(ashift=9) that 512e wants to realize. Isn't it? Emulation is to guarantee compatiblity not performace, right? > to handle drives with a specific sector size with the sd.conf doesn't turn out to > be reliable in some cases (at least on my workstations). Here's what I do to What errors have you ever seen? > ensure ashift=12 values: > > Reboot the system with a Linux live disk of your choice and install ZoL in the live > session. Then create the ZFS pool, export it and reboot the machine. OmniOS / > Illumos can import the new pool without problems and the ashift value is This operation is driver-dependant. I have the similar workaround. It works for avago HBAs. But it doesn't apply in NVMe drivers. The zpool comprised of NVMe SSDs created under ZoL can't be imported under Illumos in my tests. > correctly set. There was a fixed zpool binary (Solaris 11 binary) flying around the > internet which can handle the "-o shift=12" parameter and works with OmniOS > but unfortunately I can't find it again right now. This would make the reboot > into a live session obsolete. > > Does anyone know if the "ashift" parameter will be implemented in the OmniOS > / Illumos zpool binary in the near future? I think the whitelist(sd.conf) is the choice by Illumos. But we may try build it ourselves if the source is available. Thanks. Fred From Fred_Liu at issi.com Tue Mar 22 10:14:38 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Tue, 22 Mar 2016 03:14:38 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> Message-ID: > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On > Behalf Of Jim Klimov > Sent: ???, ?? 22, 2016 2:11 > To: Hanno Hirschberger; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > 21 ????? 2016??. 10:02:03 CET, Hanno Hirschberger > ?????: > >On 21.03.2016 08:00, Fred Liu wrote: > >> So that means illumos can handle 512n and 4kn automatically and > >properly? > > > >Not necessarily as far as I know. Sometime drives are emulating 512 > >blocks and don't properly tell the OS about that and Illumos ZFS is > >aligning the drives with ashift=9 which leads to enormous performance > >issues. Also forcing the system to handle drives with a specific sector > > > >size with the sd.conf doesn't turn out to be reliable in some cases (at > > > >least on my workstations). Here's what I do to ensure ashift=12 values: > > > >Reboot the system with a Linux live disk of your choice and install ZoL > > > >in the live session. Then create the ZFS pool, export it and reboot the > > > >machine. OmniOS / Illumos can import the new pool without problems and > >the ashift value is correctly set. There was a fixed zpool binary > >(Solaris 11 binary) flying around the internet which can handle the "-o > > > >shift=12" parameter and works with OmniOS but unfortunately I can't > >find it again right now. This would make the reboot into a live session > >obsolete. > > > >Does anyone know if the "ashift" parameter will be implemented in the > >OmniOS / Illumos zpool binary in the near future? > > > >Best regards > > > >Hanno > >_______________________________________________ > >OmniOS-discuss mailing list > >OmniOS-discuss at lists.omniti.com > >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > Adding the ashift argument to zpool was discussed every few years and so far > was always deemed not enterprisey enough for the Solaris heritage, so the > setup to tweak sd driver reports and properly rely on that layer was pushed > instead. > > That said, the old tweaked binary came with a blog post detailing the source > changes; you're welcome to try a d port and rti it (I'd say there is enough user > demand to back the non-enterprisey fix to be on par with other OpenZFS > siblings). At worst, you can publish the modernized binary as the original > blogger did ;) Is the post/source still accessible? Thanks. Fred From Fred_Liu at issi.com Tue Mar 22 10:16:40 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Tue, 22 Mar 2016 03:16:40 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> Message-ID: > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On > Behalf Of Richard Elling > Sent: ???, ?? 22, 2016 3:21 > To: Bob Friesenhahn > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > On Mar 21, 2016, at 12:19 PM, Bob Friesenhahn > wrote: > > > > On Mon, 21 Mar 2016, Richard Elling wrote: > >>> > >>> Adding the ashift argument to zpool was discussed every few years and so > far was always deemed not enterprisey enough for the Solaris heritage, so the > setup to tweak sd driver reports and properly rely on that layer was pushed > instead. > >> > >> The issue is that once a drive model lies, then the Solaris approach > >> is to encode the lie into a whitelist, so that the lie is always > >> handled properly. The whitelist is in the sd.conf file. > > > > Does this approach require that Illumos users only use drive hardware much > older than the version of Illumos they happen running since outdated whitelist > won't know about the new lies? > > Forunately, lies are becoming less common. But this raises a good point: if your > drive doesn't lie, then you don't need to workaround. That means 512n and 4kn are handled correctly by default, right? Thanks. Fred From Fred_Liu at issi.com Tue Mar 22 10:21:19 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Tue, 22 Mar 2016 03:21:19 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <30FBDF2F-E7B7-4302-8B09-1ED8E53027B4@richardelling.com> References: <56EF0932.2030100@gnaa.net> <41D8AF3D9B6AA27A.795B8F4C-9D18-4DBA-938E-32ED3D6C19F4@mail.outlook.com> <56EF6387.5080404@gnaa.net> <3FD39568-BC1F-48F0-B0D1-74FE46C143AC@zunaj.si> <56EFB88B.9030304@googlemail.com> <31A6FF25-D0E7-4934-A1B3-67DCA528B097@cos.ru> <8DBB570F-CA0A-48A6-9535-3D705C395DB6@richardelling.com> <65DC5816D4BEE043885A89FD54E273FC71CA96A3@MAIL101.Ellipseinc.com> <30FBDF2F-E7B7-4302-8B09-1ED8E53027B4@richardelling.com> Message-ID: > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On > Behalf Of Richard Elling > Sent: ???, ?? 22, 2016 3:19 > To: Richard Jahnel > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > On Mar 21, 2016, at 12:00 PM, Richard Jahnel > wrote: > > > > Both approaches have their error points. > > > > FWIW I would very very much like to be able to force my new pools into > ashift=12. It would make drive purchasing and replacement a lot more straight > forward and future resistant. > > The fundamental problem is that this is a vdev-settable, not a pool settable. > Today, it is very common for pools to have multiple ashifts active. Recently, > per-vdev ZAP objects have been proposed and that code is working its way > through the review and integration process. Not sure how multiple-ashifts impact the performace. > > Meanwhile, use one or more of the dozens of methods documented for doing > this. > > FWIW, most people with HDDs want space efficiency, because HDDs lost the > performance race to SSDs long ago. In general, forcing everything to 4k reduces > space efficiency, so it is unlikely to be a good default. Agree. Thanks. Fred From cks at cs.toronto.edu Tue Mar 22 14:41:26 2016 From: cks at cs.toronto.edu (Chris Siebenmann) Date: Tue, 22 Mar 2016 10:41:26 -0400 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: Fred_Liu's message of Tue, 22 Mar 2016 03:00:50 -0700. Message-ID: <20160322144126.6D9D77A0498@apps0.cs.toronto.edu> > > This implicitly assumes that the only reason to set ashift=12 is > > if you are currently using one or more drives that require it. I > > strongly disagree with this view. Since ZFS cannot currently replace > > a 512n drive with a 512e one, I feel [...] > > *In theory* this replacement should work well if the lie works *correctly*. > In ZoL, for the "-o ashift" is supported in "zpool replace", the > replacement should also work in mixed sector sizes. > And in illumos the whitelist will do the same. > What errors have you ever seen? We have seen devices that changed between (claimed) 512n and (claimed) 512e/4k *within the same model number*; the only thing that distinguished the two was firmware version (which is not something that you can match in sd.conf). This came as a complete surprise to us the first time we needed to replace an old (512n) one of these with a new (512e) one. The sd.conf whitelist also requires a reboot to activate if you need to add a new entry, as far as I know. (Nor do I know what happens if you have some 512n disks and some 512e disks, both correctly recognized and in different pools, and now you need to replace a 512n disk with a spare 512e disk so you change sd.conf to claim that all of the 512e disks are 512n. I'd like to think that ZFS will carry on as normal, but I'm not sure. This makes it somewhat dangerous to change sd.conf on a live system.) > > For many usage cases, somewhat more space usage and perhaps > > somewhat slower pools are vastly preferable to a loss of pool > > redundancy over time. I feel that OmniOS should at least give you > > the option here (in a less crude way than simply telling it that > > absolutely all of your drives are 4k drives, partly because such > > general lies are problematic in various situations). > > The whitelist (sd.conf) should fit into this consideration. But not > sure how mixed sector sizes impact the performance. Oh, 512e disks in a 512n pool will probably have not great performance. ZFS does a lot of unaligned reads and writes, unlike other filesystems; if you say your disks are 512n, it really believes you and behaves accordingly. - cks From geoffn at gnaa.net Tue Mar 22 18:39:14 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Tue, 22 Mar 2016 11:39:14 -0700 Subject: [OmniOS-discuss] supermicro 3U all-in one storage system In-Reply-To: <56F08063.5040805@ColoState.EDU> References: <56EC8B66.8050609@gnaa.net> <801B1A61-1774-4D0B-9472-12AAAC499752@richardelling.com> <56F070AE.201@gnaa.net> <56F08063.5040805@ColoState.EDU> Message-ID: <56F19152.90509@gnaa.net> I found out there is an internal cage you can get to add two more disks. Part # MCP-220-82611-0N On 16-03-21 04:14 PM, Kevin Swab wrote: > These things work with the ahci driver: > > http://www.addonics.com/products/ad4mspx2-a.php > > Performance is marginal for use as an l2arc or slog, but they're fine > for installing the OS. Hot-swapping a failed device is tricky but > possible - I use nylon screws in case I drop one on the MB... > > Kevin > > > > On 03/21/2016 04:07 PM, Geoff Nordli wrote: >> On 16-03-19 01:28 PM, Richard Elling wrote: >>>> On Mar 18, 2016, at 4:12 PM, Geoff Nordli wrote: >>>> >>>> Hi. >>>> >>>> I have had good luck with the SuperStorage 6037R-E1R16L chassis with >>>> the LSI 2308 IT mode HBA. >>> We have several similar servers. The X10DRH is fine. For a non-HA >>> system, single expander backplane >>> is ok (BPN-SAS3-836EL1). >> Any thoughts on how to put 4 SATA drives in that chassis? 2 are easy, >> since they fit in the back of the server, but the other two would need >> to go in the front. >> >> Is the nvme driver stable enough yet to use as l2arc/slog? >> >> thanks, >> >> Geoff >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Tue Mar 22 20:53:10 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Tue, 22 Mar 2016 13:53:10 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <20160322144126.6D9D77A0498@apps0.cs.toronto.edu> References: <20160322144126.6D9D77A0498@apps0.cs.toronto.edu> Message-ID: <3EC354AD-2DCD-4CD9-A02E-B7785DE7EE8F@richardelling.com> > On Mar 22, 2016, at 7:41 AM, Chris Siebenmann wrote: > >>> This implicitly assumes that the only reason to set ashift=12 is >>> if you are currently using one or more drives that require it. I >>> strongly disagree with this view. Since ZFS cannot currently replace >>> a 512n drive with a 512e one, I feel [...] >> >> *In theory* this replacement should work well if the lie works *correctly*. >> In ZoL, for the "-o ashift" is supported in "zpool replace", the >> replacement should also work in mixed sector sizes. >> And in illumos the whitelist will do the same. >> What errors have you ever seen? > > We have seen devices that changed between (claimed) 512n and > (claimed) 512e/4k *within the same model number*; the only thing that > distinguished the two was firmware version (which is not something that > you can match in sd.conf). This came as a complete surprise to us the > first time we needed to replace an old (512n) one of these with a new > (512e) one. > > The sd.conf whitelist also requires a reboot to activate if you need > to add a new entry, as far as I know. > > (Nor do I know what happens if you have some 512n disks and some > 512e disks, both correctly recognized and in different pools, and > now you need to replace a 512n disk with a spare 512e disk so you > change sd.conf to claim that all of the 512e disks are 512n. I'd > like to think that ZFS will carry on as normal, but I'm not sure. > This makes it somewhat dangerous to change sd.conf on a live system.) What is missing from http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks is: 1. how to change the un_phy_blocksize for any or all uns 2. how to set a default setting for all drives in sd.conf by setting attributes to the "" of "" (see sd(7d)) I am aware of no new HDDs with 512n, so this problem will go away for HDDs. However, there are many SSDs that work better with un_phy_blocksize = 8192 and some vendors set sd.conf or source appropriately. -- richard > >>> For many usage cases, somewhat more space usage and perhaps >>> somewhat slower pools are vastly preferable to a loss of pool >>> redundancy over time. I feel that OmniOS should at least give you >>> the option here (in a less crude way than simply telling it that >>> absolutely all of your drives are 4k drives, partly because such >>> general lies are problematic in various situations). >> >> The whitelist (sd.conf) should fit into this consideration. But not >> sure how mixed sector sizes impact the performance. > > Oh, 512e disks in a 512n pool will probably have not great performance. > ZFS does a lot of unaligned reads and writes, unlike other filesystems; > if you say your disks are 512n, it really believes you and behaves > accordingly. > > - cks -------------- next part -------------- An HTML attachment was scrubbed... URL: From Fred_Liu at issi.com Wed Mar 23 09:34:19 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Wed, 23 Mar 2016 02:34:19 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <3EC354AD-2DCD-4CD9-A02E-B7785DE7EE8F@richardelling.com> References: <20160322144126.6D9D77A0498@apps0.cs.toronto.edu> <3EC354AD-2DCD-4CD9-A02E-B7785DE7EE8F@richardelling.com> Message-ID: > -----Original Message----- > From: Richard Elling [mailto:richard.elling at richardelling.com] > Sent: ???, ?? 23, 2016 4:53 > To: Chris Siebenmann > Cc: Fred Liu; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > On Mar 22, 2016, at 7:41 AM, Chris Siebenmann > wrote: > > > This implicitly assumes that the only reason to set ashift=12 is > if you are currently using one or more drives that require it. I > strongly disagree with this view. Since ZFS cannot currently > replace > a 512n drive with a 512e one, I feel [...] > > > > *In theory* this replacement should work well if the lie works > *correctly*. > In ZoL, for the "-o ashift" is supported in "zpool replace", the > replacement should also work in mixed sector sizes. > And in illumos the whitelist will do the same. > What errors have you ever seen? > > > > We have seen devices that changed between (claimed) 512n and > (claimed) 512e/4k *within the same model number*; the only thing that > distinguished the two was firmware version (which is not something that > you can match in sd.conf). This came as a complete surprise to us the > first time we needed to replace an old (512n) one of these with a new > (512e) one. > > The sd.conf whitelist also requires a reboot to activate if you need > to add a new entry, as far as I know. > > (Nor do I know what happens if you have some 512n disks and some > 512e disks, both correctly recognized and in different pools, and > now you need to replace a 512n disk with a spare 512e disk so you > change sd.conf to claim that all of the 512e disks are 512n. I'd > like to think that ZFS will carry on as normal, but I'm not sure. > This makes it somewhat dangerous to change sd.conf on a live system.) There are two cases if we don't use the remedy (whitelist in illumos or -o ashift in ZoL) here: a): 512n <---> 512e. This replacement should work *in theory* if the lie works *correctly*. b): 512n <-x-> 4kn. This replacement may not work for the different physical sector sizes. Your surprise may come from case b. > > > > What is missing from > http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks > is: > > 1. how to change the un_phy_blocksize for any or all uns 2. how to set a default > setting for all drives in sd.conf by setting attributes to > the "" of "" (see sd(7d)) > > I am aware of no new HDDs with 512n, so this problem will go away for HDDs. > However, there are many SSDs that work better with un_phy_blocksize = 8192 > and some vendors set sd.conf or source appropriately. > -- richard > > > > For many usage cases, somewhat more space usage and > perhaps > somewhat slower pools are vastly preferable to a loss of pool > redundancy over time. I feel that OmniOS should at least give > you > the option here (in a less crude way than simply telling it that > absolutely all of your drives are 4k drives, partly because such > general lies are problematic in various situations). > > > > The whitelist (sd.conf) should fit into this consideration. But not > sure how mixed sector sizes impact the performance. > > > > Oh, 512e disks in a 512n pool will probably have not great performance. > ZFS does a lot of unaligned reads and writes, unlike other filesystems; > if you say your disks are 512n, it really believes you and behaves > accordingly. I am just curious about if the mixed sector sizes(512n+4kn) will impact performance. Thanks. Fred From cks at cs.toronto.edu Wed Mar 23 14:36:01 2016 From: cks at cs.toronto.edu (Chris Siebenmann) Date: Wed, 23 Mar 2016 10:36:01 -0400 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: Fred_Liu's message of Wed, 23 Mar 2016 02:34:19 -0700. Message-ID: <20160323143602.17F757A0502@apps0.cs.toronto.edu> > > The sd.conf whitelist also requires a reboot to activate if you need > > to add a new entry, as far as I know. > > > > (Nor do I know what happens if you have some 512n disks and > > some 512e disks, both correctly recognized and in different > > pools, and now you need to replace a 512n disk with a spare 512e > > disk so you change sd.conf to claim that all of the 512e disks > > are 512n. I'd like to think that ZFS will carry on as normal, > > but I'm not sure. This makes it somewhat dangerous to change > > sd.conf on a live system.) > > There are two cases if we don't use the remedy (whitelist in illumos > or -o ashift in ZoL) here: > a): 512n <---> 512e. This replacement should work *in theory* if the > lie works *correctly*. This will not work without the sd.conf workaround in Illumos. All 512e disks that I know of correctly report their actual physical disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K physical sector size, ZFS will refuse to allow it into an ashift=9 vdev *regardless* of the fact that it is 512e and will accept reads and writes in 512-byte sectors. In Illumos, you can use sd.conf to lie to the system and claim that this is not a 512e but a 512n disk (ie, it has a 512 byte physical sector size). I don't believe there's an equivalent on ZoL, but I haven't looked. This absolute insistence on ZFS's part is what makes ashift=9 vdevs so dangerous today, because you cannot replace existing 512n disks in them with 512e disks without (significant) hackery. (Perhaps I'm misunderstanding what people mean by '512e' here; I've been assuming it means a disk which reports 512 byte logical sectors and 4k physical sectors. Such disks are what you commonly get today.) - cks From rjahnel at ellipseinc.com Wed Mar 23 14:49:12 2016 From: rjahnel at ellipseinc.com (Richard Jahnel) Date: Wed, 23 Mar 2016 14:49:12 +0000 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <20160323143602.17F757A0502@apps0.cs.toronto.edu> References: Fred_Liu's message of Wed, 23 Mar 2016 02:34:19 -0700. <20160323143602.17F757A0502@apps0.cs.toronto.edu> Message-ID: <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> It should be noted that using a 512e disk as a 512n disk subjects you to a significant risk of silent corruption in the event of power loss. Because 512e disks does a read>modify>write operation to modify 512byte chunk of a 4k sector, zfs won't know about the other 7 corrupted 512e sectors in the event of a power loss during a write operation. So when discards the incomplete txg on reboot, it won't do anything about the other 7 512e sectors it doesn't know were affected. Richard Jahnel Network Engineer On-Site.com | Ellipse Design 866-266-7483 ext. 4408 Direct: 669-800-6270 -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Chris Siebenmann Sent: Wednesday, March 23, 2016 9:36 AM To: Fred Liu Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > The sd.conf whitelist also requires a reboot to activate if you need > > to add a new entry, as far as I know. > > > > (Nor do I know what happens if you have some 512n disks and > > some 512e disks, both correctly recognized and in different > > pools, and now you need to replace a 512n disk with a spare 512e > > disk so you change sd.conf to claim that all of the 512e disks > > are 512n. I'd like to think that ZFS will carry on as normal, > > but I'm not sure. This makes it somewhat dangerous to change > > sd.conf on a live system.) > > There are two cases if we don't use the remedy (whitelist in illumos > or -o ashift in ZoL) here: > a): 512n <---> 512e. This replacement should work *in theory* if the > lie works *correctly*. This will not work without the sd.conf workaround in Illumos. All 512e disks that I know of correctly report their actual physical disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K physical sector size, ZFS will refuse to allow it into an ashift=9 vdev *regardless* of the fact that it is 512e and will accept reads and writes in 512-byte sectors. In Illumos, you can use sd.conf to lie to the system and claim that this is not a 512e but a 512n disk (ie, it has a 512 byte physical sector size). I don't believe there's an equivalent on ZoL, but I haven't looked. This absolute insistence on ZFS's part is what makes ashift=9 vdevs so dangerous today, because you cannot replace existing 512n disks in them with 512e disks without (significant) hackery. (Perhaps I'm misunderstanding what people mean by '512e' here; I've been assuming it means a disk which reports 512 byte logical sectors and 4k physical sectors. Such disks are what you commonly get today.) - cks _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Wed Mar 23 15:23:26 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 23 Mar 2016 08:23:26 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> References: <20160323143602.17F757A0502@apps0.cs.toronto.edu> <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> Message-ID: > On Mar 23, 2016, at 7:49 AM, Richard Jahnel wrote: > > It should be noted that using a 512e disk as a 512n disk subjects you to a significant risk of silent corruption in the event of power loss. Because 512e disks does a read>modify>write operation to modify 512byte chunk of a 4k sector, zfs won't know about the other 7 corrupted 512e sectors in the event of a power loss during a write operation. So when discards the incomplete txg on reboot, it won't do anything about the other 7 512e sectors it doesn't know were affected. Disagree. The risk is no greater than HDDs today with their volatile write caches. -- richard > > Richard Jahnel > Network Engineer > On-Site.com | Ellipse Design > 866-266-7483 ext. 4408 > Direct: 669-800-6270 > > > -----Original Message----- > From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Chris Siebenmann > Sent: Wednesday, March 23, 2016 9:36 AM > To: Fred Liu > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > >>> The sd.conf whitelist also requires a reboot to activate if you need >>> to add a new entry, as far as I know. >>> >>> (Nor do I know what happens if you have some 512n disks and >>> some 512e disks, both correctly recognized and in different >>> pools, and now you need to replace a 512n disk with a spare 512e >>> disk so you change sd.conf to claim that all of the 512e disks >>> are 512n. I'd like to think that ZFS will carry on as normal, >>> but I'm not sure. This makes it somewhat dangerous to change >>> sd.conf on a live system.) >> >> There are two cases if we don't use the remedy (whitelist in illumos >> or -o ashift in ZoL) here: >> a): 512n <---> 512e. This replacement should work *in theory* if the >> lie works *correctly*. > > This will not work without the sd.conf workaround in Illumos. > > All 512e disks that I know of correctly report their actual physical disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K physical sector size, ZFS will refuse to allow it into an ashift=9 vdev *regardless* of the fact that it is 512e and will accept reads and writes in 512-byte sectors. > > In Illumos, you can use sd.conf to lie to the system and claim that this is not a 512e but a 512n disk (ie, it has a 512 byte physical sector size). I don't believe there's an equivalent on ZoL, but I haven't looked. > > This absolute insistence on ZFS's part is what makes ashift=9 vdevs so dangerous today, because you cannot replace existing 512n disks in them with 512e disks without (significant) hackery. > > (Perhaps I'm misunderstanding what people mean by '512e' here; I've been assuming it means a disk which reports 512 byte logical sectors and 4k physical sectors. Such disks are what you commonly get today.) > > - cks > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Wed Mar 23 15:29:48 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 23 Mar 2016 08:29:48 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <20160323143602.17F757A0502@apps0.cs.toronto.edu> References: <20160323143602.17F757A0502@apps0.cs.toronto.edu> Message-ID: <2F19EC61-DE32-4C07-B9A7-F00A0FE4E02E@richardelling.com> > On Mar 23, 2016, at 7:36 AM, Chris Siebenmann wrote: > >>> The sd.conf whitelist also requires a reboot to activate if you need >>> to add a new entry, as far as I know. >>> >>> (Nor do I know what happens if you have some 512n disks and >>> some 512e disks, both correctly recognized and in different >>> pools, and now you need to replace a 512n disk with a spare 512e >>> disk so you change sd.conf to claim that all of the 512e disks >>> are 512n. I'd like to think that ZFS will carry on as normal, >>> but I'm not sure. This makes it somewhat dangerous to change >>> sd.conf on a live system.) >> >> There are two cases if we don't use the remedy (whitelist in illumos >> or -o ashift in ZoL) here: >> a): 512n <---> 512e. This replacement should work *in theory* if the >> lie works *correctly*. > > This will not work without the sd.conf workaround in Illumos. > > All 512e disks that I know of correctly report their actual physical > disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K > physical sector size, ZFS will refuse to allow it into an ashift=9 > vdev *regardless* of the fact that it is 512e and will accept reads > and writes in 512-byte sectors. > > In Illumos, you can use sd.conf to lie to the system and claim that > this is not a 512e but a 512n disk (ie, it has a 512 byte physical > sector size). I don't believe there's an equivalent on ZoL, but I > haven't looked. > > This absolute insistence on ZFS's part is what makes ashift=9 vdevs so > dangerous today, because you cannot replace existing 512n disks in them > with 512e disks without (significant) hackery. > > (Perhaps I'm misunderstanding what people mean by '512e' here; I've > been assuming it means a disk which reports 512 byte logical sectors and > 4k physical sectors. Such disks are what you commonly get today.) Yes. 512e means: un_phy_blocksize = 4096 (or 8192) un_tgt_blocksize = 512 for disks that don't lie. Lying disks claim un_phy_blocksize = 512 when it isn't. At this point, before the discussion degenerates further, remember that George covered this in detail at the OpenZFS conference and in his blog. http://blog.delphix.com/gwilson/2012/11/15/4k-sectors-and-zfs/ http://www.youtube.com/watch?v=TmH3iRLhZ-A&feature=youtu.be -- richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From cks at cs.toronto.edu Wed Mar 23 15:33:00 2016 From: cks at cs.toronto.edu (Chris Siebenmann) Date: Wed, 23 Mar 2016 11:33:00 -0400 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: rjahnel's message of Wed, 23 Mar 2016 14:49:12 -0000. <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> Message-ID: <20160323153300.6972E7A05B6@apps0.cs.toronto.edu> > It should be noted that using a 512e disk as a 512n disk subjects > you to a significant risk of silent corruption in the event of power > loss. Because 512e disks does a read>modify>write operation to > modify 512byte chunk of a 4k sector, zfs won't know about the other > 7 corrupted 512e sectors in the event of a power loss during a write > operation. So when discards the incomplete txg on reboot, it won't do > anything about the other 7 512e sectors it doesn't know were affected. This is true; under normal circumstances you do not want to use a 512e drive in an ashift=9 vdev. However, if you have a dead 512n drive and you have no remaining 512n spares, your choices are to run without redundancy, to wedge in a 512e drive and accept the potential problems on power failure (problems that can likely be fixed by scrubbing the pool afterwards), or obtain enough additional drives (and perhaps server(s)) to entirely rebuild the pool on 512e drives with ashift=12. In this situation, running with a 512e drive and accepting the performance issues and potential exposure to power failures is basically the lesser evil. (I wish ZFS was willing to accept this, but it isn't.) - cks From cks at cs.toronto.edu Wed Mar 23 19:48:58 2016 From: cks at cs.toronto.edu (Chris Siebenmann) Date: Wed, 23 Mar 2016 15:48:58 -0400 Subject: [OmniOS-discuss] Good way to debug DTrace invalid address errors? Message-ID: <20160323194858.C97D07A04E1@apps0.cs.toronto.edu> I have a relatively complicated chunk of dtrace code that reads kernel data structures and chases pointers through them. Some of the time it spits out 'invalid address' errors during execution, for example: dtrace: error on enabled probe ID 8 (ID 75313: fbt:nfssrv:nfs3_fhtovp:return): invalid address (0x2e8) in action #6 at DIF offset 40 dtrace: error on enabled probe ID 8 (ID 75313: fbt:nfssrv:nfs3_fhtovp:return): invalid address (0xbaddcafebaddcc7e) in action #6 at DIF offset 60 I'd like to find out exactly what pointer dereferences or other operations are failing here, so that I can figure out how to work around the issue. However, I have no solid idea how to map things like 'probe ID 8' and 'DIF offset 60' to particular lines in my DTrace source code. I assume that the answer to this involves reading DIF (the DTrace intermediate form). I've looked at 'dtrace -Se' output from this DTrace script, but I can't identify the spot I need to look at. In particular, as far as I can see nothing in the output has instructions with an offset as high as 40 or 60. I can flail around sticking guards in and varying how I do stuff to make the errors go away, but I'd like to understand how to debug this sort of stuff so I can have more confidence in my changes. Thanks in advance for any suggestions, and if people want to see the actual code involved it is this DTrace script: https://github.com/siebenmann/cks-dtrace/blob/master/nfs3-long.d (Look at line 144 for the specific dtrace probe that is probably failing, since it's the only probe on fbt:nfssrv:nfs3_fhtovp:return.) - cks PS: it's entirely possible that there's a better way to do what I'm trying here, too. These DTrace scripts were originally written on Solaris 10 update 8 and haven't been drastically revised since. From bfriesen at simple.dallas.tx.us Thu Mar 24 01:37:19 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 23 Mar 2016 20:37:19 -0500 (CDT) Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <20160323143602.17F757A0502@apps0.cs.toronto.edu> <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> Message-ID: On Wed, 23 Mar 2016, Richard Elling wrote: > >> On Mar 23, 2016, at 7:49 AM, Richard Jahnel wrote: >> >> It should be noted that using a 512e disk as a 512n disk subjects you to a significant risk of silent corruption in the event of power loss. Because 512e disks does a read>modify>write operation to modify 512byte chunk of a 4k sector, zfs won't know about the other 7 corrupted 512e sectors in the event of a power loss during a write operation. So when discards the incomplete txg on reboot, it won't do anything about the other 7 512e sectors it doesn't know were affected. > > Disagree. The risk is no greater than HDDs today with their volatile write caches. If the data unrelated to the current transaction group is read and then partially modifed (possibly with data corruption due to loss of power during write), this would seem to be worse than loss due to a volatile write cache (assuming drives which observe cache sync requests) since data unrelated to the current transaction group may have been modified. The end result would be checksum errors during a scrub. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From dave-oo at pooserville.com Thu Mar 24 04:00:40 2016 From: dave-oo at pooserville.com (Dave Pooser) Date: Wed, 23 Mar 2016 23:00:40 -0500 Subject: [OmniOS-discuss] Badlock SMB bug -- do we know if Illumos kernel CIFS implementation is also affected? Message-ID: See http://badlock.org/ for more information. I'd like to assume that the devs in question would have communicated with other SMB implementations like the Solaris/Illumos and Apple's implementation, but.... -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com From Fred_Liu at issi.com Thu Mar 24 10:26:20 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Thu, 24 Mar 2016 03:26:20 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: <20160323153300.6972E7A05B6@apps0.cs.toronto.edu> References: rjahnel's message of Wed, 23 Mar 2016 14:49:12 -0000. <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> <20160323153300.6972E7A05B6@apps0.cs.toronto.edu> Message-ID: > -----Original Message----- > From: Chris Siebenmann [mailto:cks at cs.toronto.edu] > Sent: ???, ?? 23, 2016 23:33 > To: Richard Jahnel > Cc: Chris Siebenmann; Fred Liu; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > It should be noted that using a 512e disk as a 512n disk subjects you > > to a significant risk of silent corruption in the event of power loss. > > Because 512e disks does a read>modify>write operation to modify > > 512byte chunk of a 4k sector, zfs won't know about the other > > 7 corrupted 512e sectors in the event of a power loss during a write > > operation. So when discards the incomplete txg on reboot, it won't do > > anything about the other 7 512e sectors it doesn't know were affected. > > This is true; under normal circumstances you do not want to use a 512e drive > in an ashift=9 vdev. However, if you have a dead 512n drive and you have no > remaining 512n spares, your choices are to run without redundancy, to wedge > in a 512e drive and accept the potential problems on power failure (problems > that can likely be fixed by scrubbing the pool afterwards), or obtain enough > additional drives (and perhaps > server(s)) to entirely rebuild the pool on 512e drives with ashift=12. > > In this situation, running with a 512e drive and accepting the performance > issues and potential exposure to power failures is basically the lesser evil. (I > wish ZFS was willing to accept this, but it isn't.) > [Fred Liu]: I have a similar test here: [root at 00-25-90-74-f5-04 ~]# zpool status pool: tank state: ONLINE scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c2t45d0 ONLINE 0 0 0 c2t46d0 ONLINE 0 0 0 c2t47d0 ONLINE 0 0 0 c2t48d0 ONLINE 0 0 0 c2t49d0 ONLINE 0 0 0 c2t52d0 ONLINE 0 0 0 c2t53d0 ONLINE 0 0 0 c2t44d0 ONLINE 0 0 0 spares c0t5000CCA6A0C791CBd0 AVAIL errors: No known data errors pool: zones state: ONLINE scan: scrub repaired 0 in 2h45m with 0 errors on Tue Aug 12 20:24:30 2014 config: NAME STATE READ WRITE CKSUM zones ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c0t5000C500584AC07Bd0 ONLINE 0 0 0 c0t5000C500584AC557d0 ONLINE 0 0 0 c0t5000C500584ACB1Fd0 ONLINE 0 0 0 c0t5000C500584AD7B3d0 ONLINE 0 0 0 c0t5000C500584C30DBd0 ONLINE 0 0 0 c0t5000C500586E54A3d0 ONLINE 0 0 0 c0t5000C500586EF0CBd0 ONLINE 0 0 0 c0t5000C50058426A0Fd0 ONLINE 0 0 0 logs c4t0d0 ONLINE 0 0 0 c4t1d0 ONLINE 0 0 0 cache c0t55CD2E404BE9CB7Ed0 ONLINE 0 0 0 errors: No known data errors [root at 00-25-90-74-f5-04 ~]# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t55CD2E404BE9CB7Ed0 /scsi_vhci/disk at g55cd2e404be9cb7e 1. c0t5000C500584AC07Bd0 /scsi_vhci/disk at g5000c500584ac07b 2. c0t5000C500584AC557d0 /scsi_vhci/disk at g5000c500584ac557 3. c0t5000C500584ACB1Fd0 /scsi_vhci/disk at g5000c500584acb1f 4. c0t5000C500584AD7B3d0 /scsi_vhci/disk at g5000c500584ad7b3 5. c0t5000C500584C30DBd0 /scsi_vhci/disk at g5000c500584c30db 6. c0t5000C500586E54A3d0 /scsi_vhci/disk at g5000c500586e54a3 7. c0t5000C500586EF0CBd0 /scsi_vhci/disk at g5000c500586ef0cb 8. c0t5000C50058426A0Fd0 /scsi_vhci/disk at g5000c50058426a0f 9. c0t5000CCA6A0C791CBd0 /scsi_vhci/disk at g5000cca6a0c791cb 10. c0t50000F0056425331d0 /scsi_vhci/disk at g50000f0056425331 11. c2t44d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2c,0 12. c2t45d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2d,0 13. c2t46d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2e,0 14. c2t47d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2f,0 15. c2t48d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 30,0 16. c2t49d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 31,0 17. c2t52d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 34,0 18. c2t53d0 /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 35,0 19. c4t0d0 /pci at 0,0/pci15d9,624 at 1f,2/disk at 0,0 20. c4t1d0 /pci at 0,0/pci15d9,624 at 1f,2/disk at 1,0 [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment But in fact "c2t44d0" and "c0t5000CCA6A0C791CBd0" are in the same model -- ATA-Hitachi HTS54101-A480-931.51GB. That is HTS541010A9E680 (https://www.hgst.com/sites/default/files/resources/TS5K1000_ds.pdf) which is a 512e HDD. The *only* difference is that "c2t44d0" is attached to a LSI 1068 HBA and "c0t5000CCA6A0C791CBd0" is attached to a LSI 2308 HBA. [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift ashift: 9 ashift: 9 ashift: 9 ashift: 9 [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | grep ashift ashift: 12 ashift: 12 ashift: 12 ashift: 12 format> inq Vendor: ATA Product: Hitachi HTS54101 Revision: A480 format> q adding " "ATA Hitachi HTS54101", "physical-block-size:512"," into sd.conf [root at 00-25-90-74-f5-04 ~]# update_drv -vf sd Cannot unload module: sd Will be unloaded upon reboot. Forcing update of sd.conf. sd.conf updated in the kernel. Reboot the server for "cfgadm -c unconfigure" can't work here. [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | grep ashift [root at 00-25-90-74-f5-04 ~]# No ashift in output now. [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift ashift: 9 ashift: 9 ashift: 9 ashift: 9 same like before [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment Remove the spare: [root at 00-25-90-74-f5-04 ~]# zpool remove tank c0t5000CCA6A0C791CBd0 [root at 00-25-90-74-f5-04 ~]# Add it back: [root at 00-25-90-74-f5-04 ~]# zpool add tank spare c0t5000CCA6A0C791CBd0 [root at 00-25-90-74-f5-04 ~]# [root at 00-25-90-74-f5-04 ~]# zpool status tank pool: tank state: ONLINE scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c2t45d0 ONLINE 0 0 0 c2t46d0 ONLINE 0 0 0 c2t47d0 ONLINE 0 0 0 c2t48d0 ONLINE 0 0 0 c2t49d0 ONLINE 0 0 0 c2t52d0 ONLINE 0 0 0 c2t53d0 ONLINE 0 0 0 c2t44d0 ONLINE 0 0 0 spares c0t5000CCA6A0C791CBd0 AVAIL errors: No known data errors Still not working; [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with c0t5000CCA6A0C791CBd0: devices have different sector alignment maybe the sd.conf update is not correct. From richard.elling at richardelling.com Thu Mar 24 13:58:58 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Thu, 24 Mar 2016 06:58:58 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 In-Reply-To: References: <20160323143602.17F757A0502@apps0.cs.toronto.edu> <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> Message-ID: <63AB471B-65C6-4CA6-A2F2-420A4D6E603C@richardelling.com> > On Mar 23, 2016, at 6:37 PM, Bob Friesenhahn wrote: > > On Wed, 23 Mar 2016, Richard Elling wrote: > >> >>> On Mar 23, 2016, at 7:49 AM, Richard Jahnel wrote: >>> >>> It should be noted that using a 512e disk as a 512n disk subjects you to a significant risk of silent corruption in the event of power loss. Because 512e disks does a read>modify>write operation to modify 512byte chunk of a 4k sector, zfs won't know about the other 7 corrupted 512e sectors in the event of a power loss during a write operation. So when discards the incomplete txg on reboot, it won't do anything about the other 7 512e sectors it doesn't know were affected. >> >> Disagree. The risk is no greater than HDDs today with their volatile write caches. > > If the data unrelated to the current transaction group is read and then partially modifed (possibly with data corruption due to loss of power during write), this would seem to be worse than loss due to a volatile write cache (assuming drives which observe cache sync requests) since data unrelated to the current transaction group may have been modified. The end result would be checksum errors during a scrub. The old data is not modified. This is not read-destroy-modify-write. -- richard From richard.elling at richardelling.com Thu Mar 24 14:27:33 2016 From: richard.elling at richardelling.com (Richard Elling) Date: Thu, 24 Mar 2016 07:27:33 -0700 Subject: [OmniOS-discuss] Good way to debug DTrace invalid address errors? In-Reply-To: <20160323194858.C97D07A04E1@apps0.cs.toronto.edu> References: <20160323194858.C97D07A04E1@apps0.cs.toronto.edu> Message-ID: <7382B257-B1E0-4B94-B9E5-4EC277AE9A27@richardelling.com> a few pointers... > On Mar 23, 2016, at 12:48 PM, Chris Siebenmann wrote: > > I have a relatively complicated chunk of dtrace code that reads kernel > data structures and chases pointers through them. Some of the time it > spits out 'invalid address' errors during execution, for example: > > dtrace: error on enabled probe ID 8 (ID 75313: fbt:nfssrv:nfs3_fhtovp:return): invalid address (0x2e8) in action #6 at DIF offset 40 > dtrace: error on enabled probe ID 8 (ID 75313: fbt:nfssrv:nfs3_fhtovp:return): invalid address (0xbaddcafebaddcc7e) in action #6 at DIF offset 60 > > I'd like to find out exactly what pointer dereferences or other > operations are failing here, so that I can figure out how to work > around the issue. However, I have no solid idea how to map things > like 'probe ID 8' and 'DIF offset 60' to particular lines in my > DTrace source code. > > I assume that the answer to this involves reading DIF (the DTrace > intermediate form). I've looked at 'dtrace -Se' output from this > DTrace script, but I can't identify the spot I need to look at. > In particular, as far as I can see nothing in the output has > instructions with an offset as high as 40 or 60. > > I can flail around sticking guards in and varying how I do stuff > to make the errors go away, but I'd like to understand how to debug > this sort of stuff so I can have more confidence in my changes. > > Thanks in advance for any suggestions, and if people want to see > the actual code involved it is this DTrace script: > > https://github.com/siebenmann/cks-dtrace/blob/master/nfs3-long.d We do something similar, however we differ in the method to arrive at pool and dataset. Without knowing your conventions, it is not really possible to make concrete recommendations, but here is how we do it: + each share is unique + each share has a UUID + mount point contains the share UUID + each UUID can be mapped to a pool and dataset + all permutations of (operation, client, share) are collected, including "all" + today, all of this data is pumped into collectd --> influxdb for queries With newer illumos distros, there are Riemann sum kstats for each NFS version, operation, by share. This might or might not help. We find the averages to be less insightful than the distribution. Also, for client IP address, here's what we do: self->remote = (xlate (args[3])).ci_remote; -- richard > > (Look at line 144 for the specific dtrace probe that is probably > failing, since it's the only probe on fbt:nfssrv:nfs3_fhtovp:return.) > > - cks > PS: it's entirely possible that there's a better way to do what I'm > trying here, too. These DTrace scripts were originally written > on Solaris 10 update 8 and haven't been drastically revised > since. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Thu Mar 24 15:11:25 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 24 Mar 2016 11:11:25 -0400 Subject: [OmniOS-discuss] Badlock SMB bug -- do we know if Illumos kernel CIFS implementation is also affected? In-Reply-To: References: Message-ID: <858E2AEA-E25F-45AE-B349-F3B551F4DF2F@omniti.com> > On Mar 24, 2016, at 12:00 AM, Dave Pooser wrote: > > See http://badlock.org/ for more information. I'd like to assume that the > devs in question would have communicated with other SMB implementations > like the Solaris/Illumos and Apple's implementation, but.... The illumos SMB expert has informed me that the kernel SMB is not vulnerable. He can't give me details at the moment, but I know he's in contact with Microsoft. I suspect when he's allowed to speak about it, we'll get some details on one of the illumos lists. Dan From dave-oo at pooserville.com Thu Mar 24 16:21:45 2016 From: dave-oo at pooserville.com (Dave Pooser) Date: Thu, 24 Mar 2016 11:21:45 -0500 Subject: [OmniOS-discuss] Badlock SMB bug -- do we know if Illumos kernel CIFS implementation is also affected? In-Reply-To: <858E2AEA-E25F-45AE-B349-F3B551F4DF2F@omniti.com> References: <858E2AEA-E25F-45AE-B349-F3B551F4DF2F@omniti.com> Message-ID: On 3/24/16, 10:11 AM, "Dan McDonald" wrote: >The illumos SMB expert has informed me that the kernel SMB is not >vulnerable. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com From dave-oo at pooserville.com Fri Mar 25 17:16:52 2016 From: dave-oo at pooserville.com (Dave Pooser) Date: Fri, 25 Mar 2016 12:16:52 -0500 Subject: [OmniOS-discuss] supermicro A1SRM-2558F In-Reply-To: References: <20150801210124.64866f09@sleipner.datanom.net> Message-ID: On 8/1/15, 2:45 PM, "OmniOS-discuss on behalf of Bob Friesenhahn" wrote: >I have a system with one of these motherboards on order but have not >yet heard about specific issues with OmniOS except that the 10Gb-E >interfaces will not be supported right away, and there was mention of >"weirdness with asy port" with early hardware on an Illumos list. >Hopefully I will have news within the next two weeks Any experiences to report? I'm curious about a low-poer box for my own nefarious purposes.... -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com From bfriesen at simple.dallas.tx.us Fri Mar 25 19:17:16 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 25 Mar 2016 14:17:16 -0500 (CDT) Subject: [OmniOS-discuss] supermicro A1SRM-2558F In-Reply-To: References: <20150801210124.64866f09@sleipner.datanom.net> Message-ID: On Fri, 25 Mar 2016, Dave Pooser wrote: > On 8/1/15, 2:45 PM, "OmniOS-discuss on behalf of Bob Friesenhahn" > bfriesen at simple.dallas.tx.us> wrote: > >> I have a system with one of these motherboards on order but have not >> yet heard about specific issues with OmniOS except that the 10Gb-E >> interfaces will not be supported right away, and there was mention of >> "weirdness with asy port" with early hardware on an Illumos list. >> Hopefully I will have news within the next two weeks > > Any experiences to report? I'm curious about a low-poer box for my own > nefarious purposes.... Other than the 10Gbit interfaces still not working, and the need to downgrade USB to USB 2.0, I have experienced nothing but joy from my X10SDV-TLN4F-based system in the past four months after a firmware update was applied. The firmware update was necessary in order for the BIOS to discover and boot from M.2 SATA devices. It is not as fast as systems taking 10X more power but should handle most medium-duty Internet services and file server duties without any problem at all. The only real gripe I have found is that the IPMI Ethernet is bridged onto one of the LAN Ethernet interfaces (unfortunately the one I chose to point to the Internet), which causes a security concern. Apparently this is typical for SuperMicro hardware. I would be happier if the SOC/motherboad supported 8 SATA interfaces rather than 6 since the chassis supports 8. Some Xeon D motherboard vendors add a SAS chip for 8 SAS channels. My system was purchased from SW Technology (https://www.swt.com/) which is off Greenville in Plano, Texas. The system was delivered with OmniOS installed and working according to my specification. The configurator page for the system I ordered (which includes an option to install OmniOS) is at "https://www.swt.com/xd04.php". Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From thomas.wiese at ovis-consulting.de Mon Mar 28 06:44:50 2016 From: thomas.wiese at ovis-consulting.de (T. Wiese, OVIS IT Consulting GmbH) Date: Mon, 28 Mar 2016 06:44:50 +0000 Subject: [OmniOS-discuss] Qlogic HBA Message-ID: <8D6FC022-4F23-485D-BA8C-CB8FFAF1C686@ovis-consulting.de> Hello, does anyone had a working setup with 8GB FC HBA from QLOGIC ( like qle2564 ) or EMULEX (LPE-12002)? Thanks Thomas Mit freundlichen Gr??en Thomas Wiese Gesch?ftsf?hrender Gesellschafter OVIS IT Consulting GmbH Arnulfstra?e 95, 12105 Berlin Tel.: 030 - 2201206 01 Fax: 030 - 2201206 30 https://www.ovis-consulting.de Gesch?ftsf?hrer: Thomas Wiese Handelsregister: Berlin - Charlottenburg, HRB 155424B UST-IdNr.: DE293333139 From Fred_Liu at issi.com Mon Mar 28 08:56:57 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Mon, 28 Mar 2016 01:56:57 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 References: rjahnel's message of Wed, 23 Mar 2016 14:49:12 -0000. <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> <20160323153300.6972E7A05B6@apps0.cs.toronto.edu> Message-ID: > -----Original Message----- > From: Fred Liu > Sent: ???, ?? 24, 2016 18:26 > To: 'Chris Siebenmann'; Richard Jahnel > Cc: omnios-discuss at lists.omniti.com > Subject: RE: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > > -----Original Message----- > > From: Chris Siebenmann [mailto:cks at cs.toronto.edu] > > Sent: ???, ?? 23, 2016 23:33 > > To: Richard Jahnel > > Cc: Chris Siebenmann; Fred Liu; omnios-discuss at lists.omniti.com > > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > > It should be noted that using a 512e disk as a 512n disk subjects > > > you to a significant risk of silent corruption in the event of power loss. > > > Because 512e disks does a read>modify>write operation to modify > > > 512byte chunk of a 4k sector, zfs won't know about the other > > > 7 corrupted 512e sectors in the event of a power loss during a write > > > operation. So when discards the incomplete txg on reboot, it won't > > > do anything about the other 7 512e sectors it doesn't know were affected. > > > > This is true; under normal circumstances you do not want to use a > > 512e drive in an ashift=9 vdev. However, if you have a dead 512n drive > > and you have no remaining 512n spares, your choices are to run without > > redundancy, to wedge in a 512e drive and accept the potential problems > > on power failure (problems that can likely be fixed by scrubbing the > > pool afterwards), or obtain enough additional drives (and perhaps > > server(s)) to entirely rebuild the pool on 512e drives with ashift=12. > > > > In this situation, running with a 512e drive and accepting the > > performance issues and potential exposure to power failures is > > basically the lesser evil. (I wish ZFS was willing to accept this, but > > it isn't.) > > > [Fred Liu]: I have a similar test here: > > [root at 00-25-90-74-f5-04 ~]# zpool status > pool: tank > state: ONLINE > scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > raidz2-0 ONLINE 0 0 0 > c2t45d0 ONLINE 0 0 0 > c2t46d0 ONLINE 0 0 0 > c2t47d0 ONLINE 0 0 0 > c2t48d0 ONLINE 0 0 0 > c2t49d0 ONLINE 0 0 0 > c2t52d0 ONLINE 0 0 0 > c2t53d0 ONLINE 0 0 0 > c2t44d0 ONLINE 0 0 0 > spares > c0t5000CCA6A0C791CBd0 AVAIL > > errors: No known data errors > > pool: zones > state: ONLINE > scan: scrub repaired 0 in 2h45m with 0 errors on Tue Aug 12 20:24:30 2014 > config: > > NAME STATE READ WRITE CKSUM > zones ONLINE 0 0 0 > raidz2-0 ONLINE 0 0 0 > c0t5000C500584AC07Bd0 ONLINE 0 0 0 > c0t5000C500584AC557d0 ONLINE 0 0 0 > c0t5000C500584ACB1Fd0 ONLINE 0 0 0 > c0t5000C500584AD7B3d0 ONLINE 0 0 0 > c0t5000C500584C30DBd0 ONLINE 0 0 0 > c0t5000C500586E54A3d0 ONLINE 0 0 0 > c0t5000C500586EF0CBd0 ONLINE 0 0 0 > c0t5000C50058426A0Fd0 ONLINE 0 0 0 > logs > c4t0d0 ONLINE 0 0 0 > c4t1d0 ONLINE 0 0 0 > cache > c0t55CD2E404BE9CB7Ed0 ONLINE 0 0 0 > > errors: No known data errors > > [root at 00-25-90-74-f5-04 ~]# format > Searching for disks...done > > > AVAILABLE DISK SELECTIONS: > 0. c0t55CD2E404BE9CB7Ed0 SSDSC2BW18-DC32-167.68GB> > /scsi_vhci/disk at g55cd2e404be9cb7e > 1. c0t5000C500584AC07Bd0 > > /scsi_vhci/disk at g5000c500584ac07b > 2. c0t5000C500584AC557d0 > > /scsi_vhci/disk at g5000c500584ac557 > 3. c0t5000C500584ACB1Fd0 > > /scsi_vhci/disk at g5000c500584acb1f > 4. c0t5000C500584AD7B3d0 > > /scsi_vhci/disk at g5000c500584ad7b3 > 5. c0t5000C500584C30DBd0 > > /scsi_vhci/disk at g5000c500584c30db > 6. c0t5000C500586E54A3d0 > > /scsi_vhci/disk at g5000c500586e54a3 > 7. c0t5000C500586EF0CBd0 > > /scsi_vhci/disk at g5000c500586ef0cb > 8. c0t5000C50058426A0Fd0 > > /scsi_vhci/disk at g5000c50058426a0f > 9. c0t5000CCA6A0C791CBd0 > /scsi_vhci/disk at g5000cca6a0c791cb > 10. c0t50000F0056425331d0 MMCRE28G-AS1Q-119.24GB> > /scsi_vhci/disk at g50000f0056425331 > 11. c2t44d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2c,0 > 12. c2t45d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2d,0 > 13. c2t46d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2e,0 > 14. c2t47d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2f,0 > 15. c2t48d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 30,0 > 16. c2t49d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 31,0 > 17. c2t52d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 34,0 > 18. c2t53d0 > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 35,0 > 19. c4t0d0 > /pci at 0,0/pci15d9,624 at 1f,2/disk at 0,0 > 20. c4t1d0 > /pci at 0,0/pci15d9,624 at 1f,2/disk at 1,0 > > > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > But in fact "c2t44d0" and "c0t5000CCA6A0C791CBd0" are in the same model -- > ATA-Hitachi HTS54101-A480-931.51GB. > That is HTS541010A9E680 > (https://www.hgst.com/sites/default/files/resources/TS5K1000_ds.pdf) which > is a 512e HDD. > The *only* difference is that "c2t44d0" is attached to a LSI 1068 HBA and > "c0t5000CCA6A0C791CBd0" is attached to a LSI 2308 HBA. > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift > ashift: 9 > ashift: 9 > ashift: 9 > ashift: 9 > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | > grep ashift > ashift: 12 > ashift: 12 > ashift: 12 > ashift: 12 > format> inq > Vendor: ATA > Product: Hitachi HTS54101 > Revision: A480 > format> q > > adding " "ATA Hitachi HTS54101", "physical-block-size:512"," into sd.conf > > [root at 00-25-90-74-f5-04 ~]# update_drv -vf sd Cannot unload module: sd Will > be unloaded upon reboot. > Forcing update of sd.conf. > sd.conf updated in the kernel. > > Reboot the server for "cfgadm -c unconfigure" can't work here. > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | > grep ashift > [root at 00-25-90-74-f5-04 ~]# > > No ashift in output now. > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift > ashift: 9 > ashift: 9 > ashift: 9 > ashift: 9 > > same like before > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > Remove the spare: > [root at 00-25-90-74-f5-04 ~]# zpool remove tank c0t5000CCA6A0C791CBd0 > [root at 00-25-90-74-f5-04 ~]# > > Add it back: > [root at 00-25-90-74-f5-04 ~]# zpool add tank spare c0t5000CCA6A0C791CBd0 > [root at 00-25-90-74-f5-04 ~]# > > [root at 00-25-90-74-f5-04 ~]# zpool status tank > pool: tank > state: ONLINE > scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 2015 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > raidz2-0 ONLINE 0 0 0 > c2t45d0 ONLINE 0 0 0 > c2t46d0 ONLINE 0 0 0 > c2t47d0 ONLINE 0 0 0 > c2t48d0 ONLINE 0 0 0 > c2t49d0 ONLINE 0 0 0 > c2t52d0 ONLINE 0 0 0 > c2t53d0 ONLINE 0 0 0 > c2t44d0 ONLINE 0 0 0 > spares > c0t5000CCA6A0C791CBd0 AVAIL > > errors: No known data errors > > Still not working; > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > > maybe the sd.conf update is not correct. > > It looks ZoL is a really helpful tool to override ashift when sd.conf doesn't work even with following errors: PANIC: blkptr at ffff8807e4b34000 has invalid CHECKSUM 10 Showing stack for process 11419 Pid: 11419, comm: z_zvol Tainted: P -- ------------ 2.6.32-573.3.1.el6.x86_64 #1 Call Trace: [] ? spl_dumpstack+0x3d/0x40 [spl] [] ? vcmn_err+0x8d/0xf0 [spl] [] ? schedule_timeout+0x19a/0x2e0 [] ? process_timeout+0x0/0x10 [] ? finish_wait+0x67/0x80 [] ? spl_kmem_cache_alloc+0x38f/0x8c0 [spl] [] ? zfs_panic_recover+0x52/0x60 [zfs] [] ? arc_read_done+0x0/0x320 [zfs] [] ? zfs_blkptr_verify+0x83/0x420 [zfs] [] ? autoremove_wake_function+0x0/0x40 [] ? zio_read+0x42/0x100 [zfs] [] ? __kmalloc_node+0x4d/0x60 [] ? arc_read_done+0x0/0x320 [zfs] [] ? arc_read+0x341/0xa70 [zfs] [] ? dbuf_prefetch+0x1f4/0x2e0 [zfs] [] ? dmu_prefetch+0x1da/0x210 [zfs] [] ? alloc_disk_node+0xad/0x110 [] ? zvol_create_minor_impl+0x607/0x630 [zfs] [] ? zvol_create_minors_cb+0x88/0xf0 [zfs] [] ? dmu_objset_find_impl+0x106/0x420 [zfs] [] ? zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? dmu_objset_find_impl+0x1ca/0x420 [zfs] [] ? zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? dmu_objset_find_impl+0x1ca/0x420 [zfs] [] ? zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? dmu_objset_find+0x52/0x80 [zfs] [] ? spl_kmem_alloc+0x96/0x1a0 [spl] [] ? zvol_task_cb+0x392/0x3b0 [zfs] [] ? taskq_thread+0x25f/0x540 [spl] [] ? default_wake_function+0x0/0x20 [] ? taskq_thread+0x0/0x540 [spl] [] ? kthread+0x9e/0xc0 [] ? child_rip+0xa/0x20 [] ? kthread+0x0/0xc0 [] ? child_rip+0x0/0x20 INFO: task z_zvol:11419 blocked for more than 120 seconds. Tainted: P -- ------------ 2.6.32-573.3.1.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. z_zvol D 0000000000000003 0 11419 2 0x00000000 ffff8807e5daf610 0000000000000046 0000000000000000 0000000000000000 0000000000000000 ffff8807e5daf5e8 000000b57f764afb 0000000000020000 ffff8807e5daf5b0 0000000100074cde ffff8807e68bbad8 ffff8 The pool can still be imported by ZoL and processed by zpool replace -o ashift=9: [root at livecd ~]# zpool status pool: tank state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Mar 28 08:36:30 2016 1.95G scanned out of 1.47T at 1.76M/s, 242h30m to go 246M resilvered, 0.13% done config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 ata-Hitachi_HTS541010A9E680_J8400076GJU97C ONLINE 0 0 0 ata-ST1000LM024_HN-M101MBB_S2R8J9BC502817 ONLINE 0 0 0 ata-ST1000LM024_HN-M101MBB_S2R8J9KC505621 ONLINE 0 0 0 ata-WDC_WD10JPVT-08A1YT2_WD-WXD1A4355927 ONLINE 0 0 0 ata-WDC_WD10JPVT-75A1YT0_WXP1EA2KFK12 ONLINE 0 0 0 ata-ST1000LM024_HN-M101MBB_S318J9AF191087 ONLINE 0 0 0 ata-ST1000LM024_HN-M101MBB_S318J9AF191090 ONLINE 0 0 0 spare-7 ONLINE 0 0 0 ata-Hitachi_HTS541010A9E680_J8400076GJ0KZD ONLINE 0 0 0 sdw ONLINE 0 0 0 (resilvering) spares sdw INUSE currently in use It can be good remedy for the old storage systems with 512n HDDs and without 512n spares. Thanks Fred From Fred_Liu at issi.com Mon Mar 28 09:06:46 2016 From: Fred_Liu at issi.com (Fred Liu) Date: Mon, 28 Mar 2016 02:06:46 -0700 Subject: [OmniOS-discuss] 4kn or 512e with ashift=12 References: rjahnel's message of Wed, 23 Mar 2016 14:49:12 -0000. <65DC5816D4BEE043885A89FD54E273FC71CAA82A@MAIL101.Ellipseinc.com> <20160323153300.6972E7A05B6@apps0.cs.toronto.edu> Message-ID: > -----Original Message----- > From: Fred Liu > Sent: ???, ?? 28, 2016 16:57 > To: 'Chris Siebenmann'; 'Richard Jahnel' > Cc: 'omnios-discuss at lists.omniti.com' > Subject: RE: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > > -----Original Message----- > > From: Fred Liu > > Sent: ???, ?? 24, 2016 18:26 > > To: 'Chris Siebenmann'; Richard Jahnel > > Cc: omnios-discuss at lists.omniti.com > > Subject: RE: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > > > > > > -----Original Message----- > > > From: Chris Siebenmann [mailto:cks at cs.toronto.edu] > > > Sent: ???, ?? 23, 2016 23:33 > > > To: Richard Jahnel > > > Cc: Chris Siebenmann; Fred Liu; omnios-discuss at lists.omniti.com > > > Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12 > > > > > > > It should be noted that using a 512e disk as a 512n disk subjects > > > > you to a significant risk of silent corruption in the event of power loss. > > > > Because 512e disks does a read>modify>write operation to modify > > > > 512byte chunk of a 4k sector, zfs won't know about the other > > > > 7 corrupted 512e sectors in the event of a power loss during a > > > > write operation. So when discards the incomplete txg on reboot, it > > > > won't do anything about the other 7 512e sectors it doesn't know were > affected. > > > > > > This is true; under normal circumstances you do not want to use a > > > 512e drive in an ashift=9 vdev. However, if you have a dead 512n > > > drive and you have no remaining 512n spares, your choices are to run > > > without redundancy, to wedge in a 512e drive and accept the > > > potential problems on power failure (problems that can likely be > > > fixed by scrubbing the pool afterwards), or obtain enough additional > > > drives (and perhaps > > > server(s)) to entirely rebuild the pool on 512e drives with ashift=12. > > > > > > In this situation, running with a 512e drive and accepting the > > > performance issues and potential exposure to power failures is > > > basically the lesser evil. (I wish ZFS was willing to accept this, > > > but it isn't.) > > > > > [Fred Liu]: I have a similar test here: > > > > [root at 00-25-90-74-f5-04 ~]# zpool status > > pool: tank > > state: ONLINE > > scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 > > 2015 > > config: > > > > NAME STATE READ WRITE CKSUM > > tank ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > c2t45d0 ONLINE 0 0 0 > > c2t46d0 ONLINE 0 0 0 > > c2t47d0 ONLINE 0 0 0 > > c2t48d0 ONLINE 0 0 0 > > c2t49d0 ONLINE 0 0 0 > > c2t52d0 ONLINE 0 0 0 > > c2t53d0 ONLINE 0 0 0 > > c2t44d0 ONLINE 0 0 0 > > spares > > c0t5000CCA6A0C791CBd0 AVAIL > > > > errors: No known data errors > > > > pool: zones > > state: ONLINE > > scan: scrub repaired 0 in 2h45m with 0 errors on Tue Aug 12 20:24:30 > > 2014 > > config: > > > > NAME STATE READ WRITE > CKSUM > > zones ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > c0t5000C500584AC07Bd0 ONLINE 0 0 0 > > c0t5000C500584AC557d0 ONLINE 0 0 0 > > c0t5000C500584ACB1Fd0 ONLINE 0 0 0 > > c0t5000C500584AD7B3d0 ONLINE 0 0 0 > > c0t5000C500584C30DBd0 ONLINE 0 0 0 > > c0t5000C500586E54A3d0 ONLINE 0 0 0 > > c0t5000C500586EF0CBd0 ONLINE 0 0 0 > > c0t5000C50058426A0Fd0 ONLINE 0 0 0 > > logs > > c4t0d0 ONLINE 0 0 0 > > c4t1d0 ONLINE 0 0 0 > > cache > > c0t55CD2E404BE9CB7Ed0 ONLINE 0 0 0 > > > > errors: No known data errors > > > > [root at 00-25-90-74-f5-04 ~]# format > > Searching for disks...done > > > > > > AVAILABLE DISK SELECTIONS: > > 0. c0t55CD2E404BE9CB7Ed0 SSDSC2BW18-DC32-167.68GB> > > /scsi_vhci/disk at g55cd2e404be9cb7e > > 1. c0t5000C500584AC07Bd0 > > > > /scsi_vhci/disk at g5000c500584ac07b > > 2. c0t5000C500584AC557d0 > > > > /scsi_vhci/disk at g5000c500584ac557 > > 3. c0t5000C500584ACB1Fd0 > > > > /scsi_vhci/disk at g5000c500584acb1f > > 4. c0t5000C500584AD7B3d0 > > > > /scsi_vhci/disk at g5000c500584ad7b3 > > 5. c0t5000C500584C30DBd0 > > > > /scsi_vhci/disk at g5000c500584c30db > > 6. c0t5000C500586E54A3d0 > > > > /scsi_vhci/disk at g5000c500586e54a3 > > 7. c0t5000C500586EF0CBd0 > > > > /scsi_vhci/disk at g5000c500586ef0cb > > 8. c0t5000C50058426A0Fd0 > > > > /scsi_vhci/disk at g5000c50058426a0f > > 9. c0t5000CCA6A0C791CBd0 HTS54101-A480-931.51GB> > > /scsi_vhci/disk at g5000cca6a0c791cb > > 10. c0t50000F0056425331d0 MMCRE28G-AS1Q-119.24GB> > > /scsi_vhci/disk at g50000f0056425331 > > 11. c2t44d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2c,0 > > 12. c2t45d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2d,0 > > 13. c2t46d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2e,0 > > 14. c2t47d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 2f,0 > > 15. c2t48d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 30,0 > > 16. c2t49d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 31,0 > > 17. c2t52d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 34,0 > > 18. c2t53d0 > > /pci at 0,0/pci8086,1c10 at 1c/pci1000,3140 at 0/sd at 35,0 > > 19. c4t0d0 > > /pci at 0,0/pci15d9,624 at 1f,2/disk at 0,0 > > 20. c4t1d0 > > /pci at 0,0/pci15d9,624 at 1f,2/disk at 1,0 > > > > > > > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > > > But in fact "c2t44d0" and "c0t5000CCA6A0C791CBd0" are in the same > > model -- ATA-Hitachi HTS54101-A480-931.51GB. > > That is HTS541010A9E680 > > (https://www.hgst.com/sites/default/files/resources/TS5K1000_ds.pdf) > > which is a 512e HDD. > > The *only* difference is that "c2t44d0" is attached to a LSI 1068 HBA > > and "c0t5000CCA6A0C791CBd0" is attached to a LSI 2308 HBA. > > > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift > > ashift: 9 > > ashift: 9 > > ashift: 9 > > ashift: 9 > > > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | > > grep ashift > > ashift: 12 > > ashift: 12 > > ashift: 12 > > ashift: 12 > > format> inq > > Vendor: ATA > > Product: Hitachi HTS54101 > > Revision: A480 > > format> q > > > > adding " "ATA Hitachi HTS54101", "physical-block-size:512"," into > > sd.conf > > > > [root at 00-25-90-74-f5-04 ~]# update_drv -vf sd Cannot unload module: sd > > Will be unloaded upon reboot. > > Forcing update of sd.conf. > > sd.conf updated in the kernel. > > > > Reboot the server for "cfgadm -c unconfigure" can't work here. > > > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c0t5000CCA6A0C791CBd0s0 | > > grep ashift > > [root at 00-25-90-74-f5-04 ~]# > > > > No ashift in output now. > > > > [root at 00-25-90-74-f5-04 ~]# zdb -l /dev/dsk/c2t44d0s0 | grep ashift > > ashift: 9 > > ashift: 9 > > ashift: 9 > > ashift: 9 > > > > same like before > > > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > > > Remove the spare: > > [root at 00-25-90-74-f5-04 ~]# zpool remove tank c0t5000CCA6A0C791CBd0 > > [root at 00-25-90-74-f5-04 ~]# > > > > Add it back: > > [root at 00-25-90-74-f5-04 ~]# zpool add tank spare c0t5000CCA6A0C791CBd0 > > [root at 00-25-90-74-f5-04 ~]# > > > > [root at 00-25-90-74-f5-04 ~]# zpool status tank > > pool: tank > > state: ONLINE > > scan: resilvered 187G in 21h9m with 0 errors on Thu Jan 15 08:05:16 > > 2015 > > config: > > > > NAME STATE READ WRITE CKSUM > > tank ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > c2t45d0 ONLINE 0 0 0 > > c2t46d0 ONLINE 0 0 0 > > c2t47d0 ONLINE 0 0 0 > > c2t48d0 ONLINE 0 0 0 > > c2t49d0 ONLINE 0 0 0 > > c2t52d0 ONLINE 0 0 0 > > c2t53d0 ONLINE 0 0 0 > > c2t44d0 ONLINE 0 0 0 > > spares > > c0t5000CCA6A0C791CBd0 AVAIL > > > > errors: No known data errors > > > > Still not working; > > > > [root at 00-25-90-74-f5-04 ~]# zpool replace tank c2t44d0 > > c0t5000CCA6A0C791CBd0 cannot replace c2t44d0 with > > c0t5000CCA6A0C791CBd0: devices have different sector alignment > > > > > > maybe the sd.conf update is not correct. > > > > > > It looks ZoL is a really helpful tool to override ashift when sd.conf doesn't work > even with following errors: > > PANIC: blkptr at ffff8807e4b34000 has invalid CHECKSUM 10 Showing stack for > process 11419 > Pid: 11419, comm: z_zvol Tainted: P -- ------------ > 2.6.32-573.3.1.el6.x86_64 #1 > Call Trace: > [] ? spl_dumpstack+0x3d/0x40 [spl] [] ? > vcmn_err+0x8d/0xf0 [spl] [] ? > schedule_timeout+0x19a/0x2e0 [] ? > process_timeout+0x0/0x10 [] ? finish_wait+0x67/0x80 > [] ? spl_kmem_cache_alloc+0x38f/0x8c0 [spl] > [] ? zfs_panic_recover+0x52/0x60 [zfs] [] ? > arc_read_done+0x0/0x320 [zfs] [] ? > zfs_blkptr_verify+0x83/0x420 [zfs] [] ? > autoremove_wake_function+0x0/0x40 [] ? > zio_read+0x42/0x100 [zfs] [] ? __kmalloc_node+0x4d/0x60 > [] ? arc_read_done+0x0/0x320 [zfs] [] ? > arc_read+0x341/0xa70 [zfs] [] ? > dbuf_prefetch+0x1f4/0x2e0 [zfs] [] ? > dmu_prefetch+0x1da/0x210 [zfs] [] ? > alloc_disk_node+0xad/0x110 [] ? > zvol_create_minor_impl+0x607/0x630 [zfs] [] ? > zvol_create_minors_cb+0x88/0xf0 [zfs] [] ? > dmu_objset_find_impl+0x106/0x420 [zfs] [] ? > zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? > dmu_objset_find_impl+0x1ca/0x420 [zfs] [] ? > zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? > dmu_objset_find_impl+0x1ca/0x420 [zfs] [] ? > zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? > zvol_create_minors_cb+0x0/0xf0 [zfs] [] ? > dmu_objset_find+0x52/0x80 [zfs] [] ? > spl_kmem_alloc+0x96/0x1a0 [spl] [] ? > zvol_task_cb+0x392/0x3b0 [zfs] [] ? > taskq_thread+0x25f/0x540 [spl] [] ? > default_wake_function+0x0/0x20 [] ? > taskq_thread+0x0/0x540 [spl] [] ? kthread+0x9e/0xc0 > [] ? child_rip+0xa/0x20 [] ? > kthread+0x0/0xc0 [] ? child_rip+0x0/0x20 > INFO: task z_zvol:11419 blocked for more than 120 seconds. > Tainted: P -- ------------ 2.6.32-573.3.1.el6.x86_64 #1 > "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > z_zvol D 0000000000000003 0 11419 2 0x00000000 > ffff8807e5daf610 0000000000000046 0000000000000000 0000000000000000 > 0000000000000000 ffff8807e5daf5e8 000000b57f764afb 0000000000020000 > ffff8807e5daf5b0 0000000100074cde ffff8807e68bbad8 ffff8 > > The pool can still be imported by ZoL and processed by zpool replace -o > ashift=9: > > [root at livecd ~]# zpool status > pool: tank > state: ONLINE > status: One or more devices is currently being resilvered. The pool will > continue to function, possibly in a degraded state. > action: Wait for the resilver to complete. > scan: resilver in progress since Mon Mar 28 08:36:30 2016 > 1.95G scanned out of 1.47T at 1.76M/s, 242h30m to go > 246M resilvered, 0.13% done > config: > > NAME STATE > READ WRITE CKSUM > tank ONLINE > 0 0 0 > raidz2-0 ONLINE > 0 0 0 > ata-Hitachi_HTS541010A9E680_J8400076GJU97C ONLINE > 0 0 0 > ata-ST1000LM024_HN-M101MBB_S2R8J9BC502817 ONLINE > 0 0 0 > ata-ST1000LM024_HN-M101MBB_S2R8J9KC505621 ONLINE > 0 0 0 > ata-WDC_WD10JPVT-08A1YT2_WD-WXD1A4355927 ONLINE > 0 0 0 > ata-WDC_WD10JPVT-75A1YT0_WXP1EA2KFK12 ONLINE > 0 0 0 > ata-ST1000LM024_HN-M101MBB_S318J9AF191087 ONLINE > 0 0 0 > ata-ST1000LM024_HN-M101MBB_S318J9AF191090 ONLINE > 0 0 0 > spare-7 ONLINE > 0 0 0 > ata-Hitachi_HTS541010A9E680_J8400076GJ0KZD ONLINE > 0 0 0 > sdw ONLINE > 0 0 0 (resilvering) > spares > sdw INUSE > currently in use > > It can be good remedy for the old storage systems with 512n HDDs and > without 512n spares. > > Switching back to illumos: [root at 00-25-90-74-f5-04 ~]# zpool status tank pool: tank state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Mar 28 20:36:30 2016 2.29G scanned out of 1.47T at 1.63M/s, 261h27m to go 288M resilvered, 0.15% done config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c2t45d0 ONLINE 0 0 0 c2t46d0 ONLINE 0 0 0 c2t47d0 ONLINE 0 0 0 c2t48d0 ONLINE 0 0 0 c2t49d0 ONLINE 0 0 0 c2t52d0 ONLINE 0 0 0 c2t53d0 ONLINE 0 0 0 spare-7 ONLINE 0 0 0 c2t44d0 ONLINE 0 0 0 c0t5000CCA6A0C791CBd0 ONLINE 0 0 0 (resilvering) spares c0t5000CCA6A0C791CBd0 INUSE currently in use errors: No known data errors Thanks. Fred From johan.kragsterman at capvert.se Mon Mar 28 09:44:07 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Mon, 28 Mar 2016 11:44:07 +0200 Subject: [OmniOS-discuss] Ang: Qlogic HBA In-Reply-To: <8D6FC022-4F23-485D-BA8C-CB8FFAF1C686@ovis-consulting.de> References: <8D6FC022-4F23-485D-BA8C-CB8FFAF1C686@ovis-consulting.de> Message-ID: Hi! -----"OmniOS-discuss" skrev: ----- Till: "omnios-discuss at lists.omniti.com" Fr?n: "T. Wiese, OVIS IT Consulting GmbH" S?nt av: "OmniOS-discuss" Datum: 2016-03-28 08:51 ?rende: [OmniOS-discuss] Qlogic HBA Hello, does anyone had a working setup with 8GB FC HBA from QLOGIC ( like qle2564 ) or EMULEX (LPE-12002)? Thanks Thomas WHat is it you want to know? Is it on the target side or the client side? I have set up several environments with 8 Gbis FC, all on Qlogic. I also have a showroom with an OmniOS built SAN and several hosts, but there I run 4 Gbit, but I don't really see the difference. There was a discussion here on this list last yr about 8 Gbit FC and problems with multipath/too many paths access to the LU. I don't remember the outcome of that discussion, though. Probably to limit the number of paths to a LU. Rgrds Johan Mit freundlichen Gr??en Thomas Wiese Gesch?ftsf?hrender Gesellschafter OVIS IT Consulting GmbH Arnulfstra?e 95, 12105 Berlin Tel.: ?030 - 2201206 01 Fax: ? 030 - 2201206 30 https://www.ovis-consulting.de Gesch?ftsf?hrer: Thomas Wiese Handelsregister: Berlin - Charlottenburg, HRB 155424B UST-IdNr.: DE293333139 _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Mon Mar 28 18:09:53 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 28 Mar 2016 14:09:53 -0400 Subject: [OmniOS-discuss] illumos-omnios is toxic at the moment Message-ID: <52FB8815-1477-4BAA-925B-E5586DFD44E5@omniti.com> I screwed up illumos-gate, and propagated it into illumos-omnios. PLEASE DO NOT PULL from illumos-omnios until I give an all-clear here. Thanks, Dan From mir at miras.org Mon Mar 28 18:26:53 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 28 Mar 2016 20:26:53 +0200 Subject: [OmniOS-discuss] illumos-omnios is toxic at the moment In-Reply-To: <52FB8815-1477-4BAA-925B-E5586DFD44E5@omniti.com> References: <52FB8815-1477-4BAA-925B-E5586DFD44E5@omniti.com> Message-ID: <37910FF3-610E-4092-AE86-F9EF8B343EDC@miras.org> I updated git on 151014 yesterday, was that a mistake? On March 28, 2016 8:09:53 PM GMT+02:00, Dan McDonald wrote: >I screwed up illumos-gate, and propagated it into illumos-omnios. > >PLEASE DO NOT PULL from illumos-omnios until I give an all-clear here. > >Thanks, >Dan > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ---- This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Mar 28 18:36:42 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 28 Mar 2016 14:36:42 -0400 Subject: [OmniOS-discuss] illumos-omnios is toxic at the moment In-Reply-To: <37910FF3-610E-4092-AE86-F9EF8B343EDC@miras.org> References: <52FB8815-1477-4BAA-925B-E5586DFD44E5@omniti.com> <37910FF3-610E-4092-AE86-F9EF8B343EDC@miras.org> Message-ID: > On Mar 28, 2016, at 2:26 PM, Michael Rasmussen wrote: > > I updated git on 151014 yesterday, was that a mistake? Nope! The damage was literally introduced in the past 90 minutes, and I almost have it cleared up. Expect an all-clear in 30 mins or less. Dan From danmcd at omniti.com Mon Mar 28 18:46:07 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 28 Mar 2016 14:46:07 -0400 Subject: [OmniOS-discuss] ALL CLEAR (was Re: illumos-omnios is toxic at the moment) In-Reply-To: References: <52FB8815-1477-4BAA-925B-E5586DFD44E5@omniti.com> <37910FF3-610E-4092-AE86-F9EF8B343EDC@miras.org> Message-ID: <8D4A20CE-3E89-4734-A50A-BCCA20A4A3C2@omniti.com> > On Mar 28, 2016, at 2:36 PM, Dan McDonald wrote: > > Expect an all-clear in 30 mins or less. For the first time in my life, I had to use "-f" for "git push". I hope I never have to do it again. I broke illumos-gate upstream by pushing in a merge commit, which is verboten. illumos-gate was fixed by Josh Clulow, and that's all sorted out. I compounded my error, however, by: 1.) Pulling that into the "upstream" branch of illumos-omnios (which is a copy of illumos-gate) 2.) MERGING the toxic upstream into master, making master toxic as well. NOTE: No other branches were corrupted during this time. So what I had to do was: 1.) Reset "upstream" to the now-cleaned illumos-gate. 2.) Reset "master" to a commit before this screw-up. 3.) "git push -f" these two branches to both upstream illumos-omnios repos (github and src.omniti.com). I've done all of the above three, and now illumos-omnios is back to normal. If you have a downstream of illumos-omnios, and did a "git pull" between 1:44pm US/Eastern and now, you may need to reset your local repository. Ask here if you need help with that. If you did not "git pull" in that window, you're in good shape, and can proceed as normal. ALSO, if you only had changes to branches not "upstream" or "master", you are also in good shape regardless (but please do a "git pull" now). Thanks, and sorry for the distruption, Dan From peter.tribble at gmail.com Tue Mar 29 15:57:19 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Tue, 29 Mar 2016 16:57:19 +0100 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: <5E542AC2-CE66-4B4E-AA3C-916D4C544F59@omniti.com> References: <5E542AC2-CE66-4B4E-AA3C-916D4C544F59@omniti.com> Message-ID: On Wed, Mar 16, 2016 at 12:23 PM, Dan McDonald wrote: > > A fresh install of recently-minted 014 or later WILL install openssh by > default. If you have an existing install with SunSSH, that'll stick. > Well I'm confused now. I've just installed some 151014 systems from the latest USB image (latest as of last week), and I appear to have: # pkg list | grep ssh network/openssh 7.2.2-0.151014 i-- service/network/ssh 0.5.11-0.151014 i-- In other words, it looks like the openssh client and the sunssh server. Is this intentional? It looks like an odd mix to me, and you end up missing things like /etc/ssh/moduli -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Mar 29 16:52:22 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 29 Mar 2016 12:52:22 -0400 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: References: <5E542AC2-CE66-4B4E-AA3C-916D4C544F59@omniti.com> Message-ID: <5F38FDDB-C31C-4BAD-AB70-FC594F0B4246@omniti.com> > On Mar 29, 2016, at 11:57 AM, Peter Tribble wrote: > > On Wed, Mar 16, 2016 at 12:23 PM, Dan McDonald wrote: > > A fresh install of recently-minted 014 or later WILL install openssh by default. If you have an existing install with SunSSH, that'll stick. > > Well I'm confused now. I've just installed some 151014 systems from the latest > USB image (latest as of last week), and I appear to have: > > # pkg list | grep ssh > network/openssh 7.2.2-0.151014 i-- > service/network/ssh 0.5.11-0.151014 i-- > > In other words, it looks like the openssh client and the sunssh server. > > Is this intentional? It looks like an odd mix to me, and you end up missing > things like /etc/ssh/moduli I am aware of the USB/ISO problem. Something vague in the distro_const script is happening. As I'm ramping up 018, I'll fix it there, and make sure 014 gets the same fix. You should uninstall the SunSSH server and install the openssh server simultaneously. Sorry about that, Dan From dave-oo at pooserville.com Tue Mar 29 22:30:58 2016 From: dave-oo at pooserville.com (Dave Pooser) Date: Tue, 29 Mar 2016 17:30:58 -0500 Subject: [OmniOS-discuss] Moving from Postgres/Solaris/x86_64 to Postgres/OmniOS/x86_64 - gotchas? Message-ID: $DAYJOB is currently running our core database on Postgres/Solaris. We're considering moving to OmniOS for a few reasons: partly the cost of support (since I have to pay for Solaris support on my dev/test machines, where with Omni I'd only go paid on the production box), partly the move to open ZFS (for better compatibility with ZFS on our Linux and Mac boxes), partly because Dan appears to be less evil than Larry. ;-) 1) Any gotchas running Postgres on OmniOS? I'm currently building from source on the Solaris servers, so I'm not particularly concerned about prepackaged options, but if there's some weird incompatibility I'd like to know ahead of time 2) I know Puppet isn't supported on OmniOS, but is there anybody out there using it and having it work for them? I imagine I might have to manually define some providers, but aside from that.... Thanks in advance for any experiences. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com From paul.jochum at nokia.com Wed Mar 30 01:17:55 2016 From: paul.jochum at nokia.com (Paul Jochum) Date: Tue, 29 Mar 2016 20:17:55 -0500 Subject: [OmniOS-discuss] Moving from Postgres/Solaris/x86_64 to Postgres/OmniOS/x86_64 - gotchas? In-Reply-To: References: Message-ID: <56FB2943.6010206@nokia.com> Hi Dave: Sorry, can't answer the questions on Postgres, but here is what I have for Puppet: There is a version of puppet, but you will need to add a new repo from niksula (which can be done with the commands: pkg set-publisher -p http://pkg.niksula.hut.fi/ \ --set-property signature-policy=require-signatures wget https://www.niksula.hut.fi/pkg.niksula.hut.fi.pem pkg set-publisher --approve-ca-cert pkg.niksula.hut.fi.pem niksula.hut.fi and then install puppet: pkg install puppet I also created a manifest file, which I put in /root/puppetagentd.xml ************************************* $ cat /root/puppetagentd.xml ************************************* I install this manifest with the command: svccfg import /root/puppetagentd.xml Instead of the above service, you could also call puppet from a cron job, if you prefer to do that. Hope this helps, Paul On 03/29/2016 05:30 PM, Dave Pooser wrote: > $DAYJOB is currently running our core database on Postgres/Solaris. We're > considering moving to OmniOS for a few reasons: partly the cost of support > (since I have to pay for Solaris support on my dev/test machines, where > with Omni I'd only go paid on the production box), partly the move to open > ZFS (for better compatibility with ZFS on our Linux and Mac boxes), partly > because Dan appears to be less evil than Larry. ;-) > > 1) Any gotchas running Postgres on OmniOS? I'm currently building from > source on the Solaris servers, so I'm not particularly concerned about > prepackaged options, but if there's some weird incompatibility I'd like to > know ahead of time > 2) I know Puppet isn't supported on OmniOS, but is there anybody out there > using it and having it work for them? I imagine I might have to manually > define some providers, but aside from that.... > > Thanks in advance for any experiences. From josh at sysmgr.org Wed Mar 30 01:46:56 2016 From: josh at sysmgr.org (Joshua M. Clulow) Date: Tue, 29 Mar 2016 18:46:56 -0700 Subject: [OmniOS-discuss] Moving from Postgres/Solaris/x86_64 to Postgres/OmniOS/x86_64 - gotchas? In-Reply-To: References: Message-ID: On 29 March 2016 at 15:30, Dave Pooser wrote: > 1) Any gotchas running Postgres on OmniOS? I'm currently building from > source on the Solaris servers, so I'm not particularly concerned about > prepackaged options, but if there's some weird incompatibility I'd like to > know ahead of time We make heavy use of the PostgreSQL packages from pkgsrc[1] at Joyent, but running on SmartOS (another illumos distribution). I should expect it works just as well on OmniOS! [1]: https://pkgsrc.joyent.com/ -- Joshua M. Clulow UNIX Admin/Developer http://blog.sysmgr.org From robert at omniti.com Wed Mar 30 04:43:50 2016 From: robert at omniti.com (Robert Treat) Date: Wed, 30 Mar 2016 00:43:50 -0400 Subject: [OmniOS-discuss] Moving from Postgres/Solaris/x86_64 to Postgres/OmniOS/x86_64 - gotchas? In-Reply-To: References: Message-ID: Yes, Postgres runs fine on OmniOS, we use it fairly extensively in our web operations / consulting work at OmniTI. Robert Treat On Tue, Mar 29, 2016 at 9:46 PM, Joshua M. Clulow wrote: > On 29 March 2016 at 15:30, Dave Pooser wrote: >> 1) Any gotchas running Postgres on OmniOS? I'm currently building from >> source on the Solaris servers, so I'm not particularly concerned about >> prepackaged options, but if there's some weird incompatibility I'd like to >> know ahead of time > > We make heavy use of the PostgreSQL packages from pkgsrc[1] at Joyent, > but running on SmartOS (another illumos distribution). I should > expect it works just as well on OmniOS! > > > [1]: https://pkgsrc.joyent.com/ > > -- > Joshua M. Clulow > UNIX Admin/Developer > http://blog.sysmgr.org > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From johan.kragsterman at capvert.se Wed Mar 30 10:05:26 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Wed, 30 Mar 2016 12:05:26 +0200 Subject: [OmniOS-discuss] isc dhcp server in a zone with crossbow vnic's Message-ID: Hi! I've started a development process where I need a dhcp server in a zone, and I thought I try the new(for omnios) isc dhcp. I found that Alex McWhirter wrote an artikel on Linkedin about the inability of isc dhcp to use the crossbow vnic's...? Is this true? That was in dec 2014... Another thing: Does someone know where to put the interface information? I mean which interfaces that the dhcp server should listen on(and obviously NOT listen on the others). In Linux you don't set that in dhcpd.conf, but in /etc/default/isc-dhcp-server, which is not present in omnios. Or should it be present? Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert From peter.tribble at gmail.com Wed Mar 30 10:40:51 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Wed, 30 Mar 2016 11:40:51 +0100 Subject: [OmniOS-discuss] UPDATES for git and OpenSSH now available In-Reply-To: <5F38FDDB-C31C-4BAD-AB70-FC594F0B4246@omniti.com> References: <5E542AC2-CE66-4B4E-AA3C-916D4C544F59@omniti.com> <5F38FDDB-C31C-4BAD-AB70-FC594F0B4246@omniti.com> Message-ID: Dan, On Tue, Mar 29, 2016 at 5:52 PM, Dan McDonald wrote: > > > On Mar 29, 2016, at 11:57 AM, Peter Tribble > wrote: > > > > On Wed, Mar 16, 2016 at 12:23 PM, Dan McDonald > wrote: > > > > A fresh install of recently-minted 014 or later WILL install openssh by > default. If you have an existing install with SunSSH, that'll stick. > > > > Well I'm confused now. I've just installed some 151014 systems from the > latest > > USB image (latest as of last week), and I appear to have: > > > > # pkg list | grep ssh > > network/openssh 7.2.2-0.151014 > i-- > > service/network/ssh 0.5.11-0.151014 > i-- > > > > In other words, it looks like the openssh client and the sunssh server. > > > > Is this intentional? It looks like an odd mix to me, and you end up > missing > > things like /etc/ssh/moduli > > I am aware of the USB/ISO problem. Something vague in the distro_const > script is happening. As I'm ramping up 018, I'll fix it there, and make > sure 014 gets the same fix. > > You should uninstall the SunSSH server and install the openssh server > simultaneously. > Or go back to SunSSH for everything. Which is consistent, and known to work with the sshd_config we push via puppet (and the rest of our setup). While I would much prefer to use openssh (and there's nothing in terms of functionality that would keep us on sunssh) the differences in packaging and configuration files have been giving me a bit of trouble, so I'm taking the easy way out, at least in the short term. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdg117 at elvis.arl.psu.edu Wed Mar 30 12:00:26 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Wed, 30 Mar 2016 08:00:26 -0400 Subject: [OmniOS-discuss] Moving from Postgres/Solaris/x86_64 to Postgres/OmniOS/x86_64 - gotchas? In-Reply-To: Your message of "Tue, 29 Mar 2016 17:30:58 CDT." References: Message-ID: <201603301200.u2UC0QGn009066@elvis.arl.psu.edu> In message , Dave Pooser writes: >1) Any gotchas running Postgres on OmniOS? I'm currently building from >source on the Solaris servers, so I'm not particularly concerned about >prepackaged options, but if there's some weird incompatibility I'd like to >know ahead of time $ /opt/postgres/bin/pg_config --configure '--prefix=/opt/postgres' '--localstatedir=/var/opt/postgres' 'CC=gcc' 'CFLAGS=-m64 -O3' 'LDFLAGS=-m64' John groenveld at acm.org From danmcd at omniti.com Wed Mar 30 15:19:23 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 30 Mar 2016 11:19:23 -0400 Subject: [OmniOS-discuss] isc dhcp server in a zone with crossbow vnic's In-Reply-To: References: Message-ID: <8A463F42-69D6-44E0-BEE5-966741820019@omniti.com> > On Mar 30, 2016, at 6:05 AM, Johan Kragsterman wrote: > > I found that Alex McWhirter wrote an artikel on Linkedin about the inability of isc dhcp to use the crossbow vnic's...? Is this true? That was in dec 2014... Here's the post to which you refer: https://www.linkedin.com/pulse/20141205201206-339434232-isc-dhcp-dlpi-on-solaris-illumos?trkSplashRedir=true&forceNoSplash=true That patch has not made its way upstream, AND we compile with "./configure --enable-use-sockets --enable-ipv4-pktinfo". If *appears* that the problem isn't with vnics, as it is with multiple instances of a named interface. For example, "e1000g0" and "e1000g1" or "net0" and "net1". > Does someone know where to put the interface information? I mean which interfaces that the dhcp server should listen on(and obviously NOT listen on the others). In Linux you don't set that in dhcpd.conf, but in /etc/default/isc-dhcp-server, which is not present in omnios. Or should it be present? Hmmm, I think I may need to update the man pages, but it's an SMF property: shell(~)[1]% svcprop dhcp:ipv4 | grep application/listen_ifnames application/listen_ifnames astring "" shell(~)[0]% The contents of this string are space-separated interface names. They SHOULD be able to be vNICs. I use dhcpd in a single-vNIC zone without a problem, but I don't use the listen_ifnames property. Since I configure with sockets and ipv4-pktinfo, I'm not sure how the ifnames work in this case. Try setting the listen_ifnames property with svccfg(1M). If the property change worked (and you should be able to tell with "pargs `pgrep dhcpd`" if it has), then see if it behaves like you wish. You could also create a "DHCP server" zone and dedicate a single vnic (configured over a NIC you want) to it as well. Dan From bfriesen at simple.dallas.tx.us Wed Mar 30 15:28:49 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 30 Mar 2016 10:28:49 -0500 (CDT) Subject: [OmniOS-discuss] isc dhcp server in a zone with crossbow vnic's In-Reply-To: <8A463F42-69D6-44E0-BEE5-966741820019@omniti.com> References: <8A463F42-69D6-44E0-BEE5-966741820019@omniti.com> Message-ID: On Wed, 30 Mar 2016, Dan McDonald wrote: > > Try setting the listen_ifnames property with svccfg(1M). If the > property change worked (and you should be able to tell with "pargs > `pgrep dhcpd`" if it has), then see if it behaves like you wish. > You could also create a "DHCP server" zone and dedicate a single > vnic (configured over a NIC you want) to it as well. Are there any zone limitations (networking or zone initialization ordering) which would prevent a "DHCP server" zone from providing DHCP services for other zones on the same system which use dhcp to configure their interfaces? That is my current situation. Some zones are nailed down with static assignments while others (used for more transient purposes) use DHCP to get their configuration. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Mar 30 15:32:41 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 30 Mar 2016 11:32:41 -0400 Subject: [OmniOS-discuss] isc dhcp server in a zone with crossbow vnic's In-Reply-To: References: <8A463F42-69D6-44E0-BEE5-966741820019@omniti.com> Message-ID: <33FD895D-1C5E-4AC0-8D56-E19C75FB9389@omniti.com> > On Mar 30, 2016, at 11:28 AM, Bob Friesenhahn wrote: > > Are there any zone limitations (networking or zone initialization ordering) which would prevent a "DHCP server" zone from providing DHCP services for other zones on the same system which use dhcp to configure their interfaces? That is my current situation. Some zones are nailed down with static assignments while others (used for more transient purposes) use DHCP to get their configuration. So you have zone "D" serving DHCP, and zones "A" and "B" which are DHCP clients, right? Unless you have a bizarre circular dependency ("A" serves something to "D" and vice-versa), I believe you're okay. A dhcp configured interface launches dhcpagent, and networking for that interface is down until it hears from the DHCP server. Other parts of the zone boot keep plowing. The ONLY POSSIBLE gotcha I can think of is if intra-zone broadcast is buggy, and I don't think it is. If it's buggy, we need to fix it. Dan From bfriesen at simple.dallas.tx.us Wed Mar 30 16:15:39 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 30 Mar 2016 11:15:39 -0500 (CDT) Subject: [OmniOS-discuss] Continuing zone shutdown issues with r151016 Message-ID: Previously I reported zone shutdown issues (zoneadm -z foo shutdown) with OmniOS v11 r151016 (now at omnios-33c53a8). I learned that I am not the only one encountering this issue. For my last update cycle I shut the zones down in a loop (to prepare for the update) and observed that a couple of the zones encountered the problem. The loop hung due to the shutdown handing. Does the forthcoming r151018 somehow solve this problem? This problem makes it difficult to administer the system and would seem to be quite perilous for any type of zone which persistent data which needs to be commited to disk (e.g. a database). Thanks, Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From johan.kragsterman at capvert.se Wed Mar 30 16:42:05 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Wed, 30 Mar 2016 18:42:05 +0200 Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: Message-ID: Hi! -----"OmniOS-discuss" skrev: ----- Till: omnios-discuss at lists.omniti.com Fr?n: Bob Friesenhahn S?nt av: "OmniOS-discuss" Datum: 2016-03-30 18:20 ?rende: [OmniOS-discuss] Continuing zone shutdown issues with r151016 Previously I reported zone shutdown issues (zoneadm -z foo shutdown) with OmniOS v11 r151016 (now at omnios-33c53a8). ?I learned that I am not the only one encountering this issue. For my last update cycle I shut the zones down in a loop (to prepare for the update) and observed that a couple of the zones encountered the problem. ?The loop hung due to the shutdown handing. Does the forthcoming r151018 somehow solve this problem? This problem makes it difficult to administer the system and would seem to be quite perilous for any type of zone which persistent data which needs to be commited to disk (e.g. a database). Thanks, Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, ? ?http://www.GraphicsMagick.org/ I've noticed that too(same omnios version) when I try to shut them down from the "outside". Though from the inside(init 0) it works all the time. Rgrds Johan ________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From bfriesen at simple.dallas.tx.us Wed Mar 30 17:27:54 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 30 Mar 2016 12:27:54 -0500 (CDT) Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: Message-ID: > I've noticed that too(same omnios version) when I try to shut them > down from the "outside". Though from the inside(init 0) it works all > the time. Oddly, I have yet to see a problem with 'reboot' or 'halt'. Perhaps 'reboot' skips the normal application shutdown processing? Inside the zone everything is happy. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Mar 30 18:01:18 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 30 Mar 2016 14:01:18 -0400 Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: Message-ID: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> > On Mar 30, 2016, at 1:27 PM, Bob Friesenhahn wrote: > >> I've noticed that too(same omnios version) when I try to shut them down from the "outside". Though from the inside(init 0) it works all >> the time. > > Oddly, I have yet to see a problem with 'reboot' or 'halt'. Perhaps 'reboot' skips the normal application shutdown processing? > > Inside the zone everything is happy. This isn't something that happens reproducibly, is it? I'm dinking with one of my r151016 zones, uttering "zoneadm -z shutdown" and subsequently booting without any hangs. Do you have a sure-fire way to make "zoneadm -z shutdown" happen? Dan From bfriesen at simple.dallas.tx.us Wed Mar 30 19:07:30 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Wed, 30 Mar 2016 14:07:30 -0500 (CDT) Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> References: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> Message-ID: On Wed, 30 Mar 2016, Dan McDonald wrote: >> >> Inside the zone everything is happy. > > This isn't something that happens reproducibly, is it? I'm dinking > with one of my r151016 zones, uttering "zoneadm -z shutdown" > and subsequently booting without any hangs. Do you have a sure-fire > way to make "zoneadm -z shutdown" happen? It seems to happen perhaps 20-25 percent of the time and after it happens the zone is "infected" until system reboot. If it matters, I am using lightly-loaded 8-core, 16-thread Xeon-D hardware. Interestingly, I just noticed that my most recent effort (cribbed from OmniOS system upgrade admin notes) did this: for zone in base pkgbuild ftp www src smtp do zlogin $zone shutdown -i5 -g0 -y done rather than this for zone in base pkgbuild ftp www src smtp do zoneadm -z $zone shutdown done Normally I would use 'zoneadm' to do the shutdown. However, I am pretty sure that the former (based on 'zlogin') encountered the same problem. If so, that is very interesting. In the shutdown loop, two of the zones encountered the problem. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Wed Mar 30 19:16:15 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 30 Mar 2016 15:16:15 -0400 Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> Message-ID: > On Mar 30, 2016, at 3:07 PM, Bob Friesenhahn wrote: > > Normally I would use 'zoneadm' to do the shutdown. However, I am pretty sure that the former (based on 'zlogin') encountered the same problem. If so, that is very interesting. On a zone lock, we need to see a couple of things (from root at global): - pgrep -z - pgrep -f (Should only produce the zoneadmd process.) IF AND ONLY IF THOSE TWO ABOVE consistently produce process lists (you cannot run the following w/o a nonzero process list, otherwise it defaults to ALL the processes): - ptree `pgrep -z ; pgrep -f ` - pstack `pgrep -z ; pgrep -f ` (NOTE: This may spew a lot if you have a lot of processes.) Something's stopping zoneadmd from halting, and the process stacks may help us figure it out. Dan From jimklimov at cos.ru Thu Mar 31 05:06:23 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Thu, 31 Mar 2016 07:06:23 +0200 Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> Message-ID: 30 ????? 2016??. 21:16:15 CEST, Dan McDonald ?????: > >> On Mar 30, 2016, at 3:07 PM, Bob Friesenhahn > wrote: >> >> Normally I would use 'zoneadm' to do the shutdown. However, I am >pretty sure that the former (based on 'zlogin') encountered the same >problem. If so, that is very interesting. > >On a zone lock, we need to see a couple of things (from root at global): > >- pgrep -z > >- pgrep -f (Should only produce the zoneadmd process.) > >IF AND ONLY IF THOSE TWO ABOVE consistently produce process lists (you >cannot run the following w/o a nonzero process list, otherwise it >defaults to ALL the processes): > >- ptree `pgrep -z ; pgrep -f ` > >- pstack `pgrep -z ; pgrep -f ` (NOTE: This may >spew a lot if you have a lot of processes.) > >Something's stopping zoneadmd from halting, and the process stacks may >help us figure it out. > >Dan > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss This is reminiscent of what I saw in other solarish systems, maybe even sol10 and sxce although quite rare (once in hundreds of times). There a zoneadmd would lock up and not die until kill'ed, so blocking zone reboots from outside or within. On newer illumos setups that manage a comples zfs zoneroot hierarchy, this often seemed linked with some process (e.g. a shell or midnight commander or updatedb scans, nowadays maybe pkg integration between gz and lz can play a role? etc.) running from global zone (unkillable by lz shutdown) and having as current a directory from the local zone namespace - thus blocking its unmount. Killing such process or going into another dir allowed zoneadmd teardown to proceed. Alas, this description and method do not quite apply to sol10/sxce that have simpler gz-provided zoneroots, so there may be more sinister beasts of the old lurking in the deep ;) Hope these memories help, Jim Klimov -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Thu Mar 31 05:13:50 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Thu, 31 Mar 2016 07:13:50 +0200 Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> Message-ID: 31 ????? 2016??. 7:06:23 CEST, Jim Klimov ?????: >30 ????? 2016??. 21:16:15 CEST, Dan McDonald ?????: >> >>> On Mar 30, 2016, at 3:07 PM, Bob Friesenhahn >> wrote: >>> >>> Normally I would use 'zoneadm' to do the shutdown. However, I am >>pretty sure that the former (based on 'zlogin') encountered the same >>problem. If so, that is very interesting. >> >>On a zone lock, we need to see a couple of things (from root at global): >> >>- pgrep -z >> >>- pgrep -f (Should only produce the zoneadmd process.) >> >>IF AND ONLY IF THOSE TWO ABOVE consistently produce process lists (you >>cannot run the following w/o a nonzero process list, otherwise it >>defaults to ALL the processes): >> >>- ptree `pgrep -z ; pgrep -f ` >> >>- pstack `pgrep -z ; pgrep -f ` (NOTE: This may >>spew a lot if you have a lot of processes.) >> >>Something's stopping zoneadmd from halting, and the process stacks may >>help us figure it out. >> >>Dan >> >>_______________________________________________ >>OmniOS-discuss mailing list >>OmniOS-discuss at lists.omniti.com >>http://lists.omniti.com/mailman/listinfo/omnios-discuss > >This is reminiscent of what I saw in other solarish systems, maybe even >sol10 and sxce although quite rare (once in hundreds of times). There a >zoneadmd would lock up and not die until kill'ed, so blocking zone >reboots from outside or within. > >On newer illumos setups that manage a comples zfs zoneroot hierarchy, >this often seemed linked with some process (e.g. a shell or midnight >commander or updatedb scans, nowadays maybe pkg integration between gz >and lz can play a role? etc.) running from global zone (unkillable by >lz shutdown) and having as current a directory from the local zone >namespace - thus blocking its unmount. Killing such process or going >into another dir allowed zoneadmd teardown to proceed. > >Alas, this description and method do not quite apply to sol10/sxce that >have simpler gz-provided zoneroots, so there may be more sinister >beasts of the old lurking in the deep ;) > >Hope these memories help, >Jim Klimov >-- >Typos courtesy of K-9 Mail on my Samsung Android Forgot to add: the cases similar to what i described can be inspected with fuser (-c, -m IIRC) to see if any processes block the zoneroot filesystems. Also blockage can be due to mounts underneath and other kernel-provided blockers, that userspace fuser won't show. In particular then look out for nfs+autofs client in a zone (something sol10 documented as a no-no, especiakly to not use nfs from gz of the same server, but which just worked and was cool to e.g. share homedirs or distribs, and so was widely used) - perhaps autofs failed to unmount something from a remote host, then was killed by lz shutdown, and then the remaining sub-fs blocks the zoneroot teardown. Jim -- Typos courtesy of K-9 Mail on my Samsung Android From svavar at pipar-tbwa.is Thu Mar 31 09:59:20 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Thu, 31 Mar 2016 09:59:20 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> Message-ID: <9680905E-E048-4E8E-B24C-239131BEE859@pipar-tbwa.is> So, Solaris 11.3 boots up and installs fine but not OmniOS. I sure would like to use OmniOS on this machine, so I ask are there any other debuging options for me at the installation level ? Could I try to install OmniOS on a HDD/SSD on other machine with ACHI and boot it up on the problematic machine? Or is that just a no-go from the start? Thanks again people. Svavar Reykjavik - Iceland > On 14. mar. 2016, at 21:28, Christian Kivalo wrote: > > > > On 2016-03-09 11:00, Svavar ?rn Eysteinsson wrote: >> Hi Dan. >> Thanks for the reply. >> I disabled all the C-states and it's the same story. >> If I boot from the SATA CD-ROM installation it hangs on boot right after : >> cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >> cpu7: initialization complete - online >> cpu6: initialization complete - online >> See here : >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG >> If I boot from USB stick, with USB 2.0 enabled and or disabled the >> last message is always releated to usb stuff >> right after cpu6: initialization complete - online. >> So my BIOS configuration right now is : >> Intel QPI Frequency Select [Auto Max] >> Intel Turbo Boots Technology [Enabled] >> Enhanced Intel Speedstep Tech [Enabled] >> Processor C3 [Disabled] >> Processor C6 [Disabled] >> Intel Hyper-Threading Tech [Disabled] >> Core Multi-Processing [All] >> Execute Disable Bit [Disabled] >> Intel Virtualization Technology [Disasbled] >> Intel VT for Directed I/O [Disabled] >> and some more. >> My Bios Configuration can been seen here : >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG >> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG > Maybe something for you there was a thread here some months ago... > It starts here https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03194.html > > and some messages into the the thread X2APIC is mentioned > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03200.html > > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03201.html > > and a link a blogpost by jeff sipek > http://blahg.josefsipek.net/?p=488 > > >> Best regards, >> Svavar O >>> On 8. mar. 2016, at 15:13, Dan McDonald wrote: >>>> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson wrote: >>>> Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? >>> Disable C-states, that's general illumos advice (regardless of distro). >>> Dan >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- > Christian Kivalo > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Thu Mar 31 11:16:55 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 31 Mar 2016 13:16:55 +0200 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: <9680905E-E048-4E8E-B24C-239131BEE859@pipar-tbwa.is> References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> <9680905E-E048-4E8E-B24C-239131BEE859@pipar-tbwa.is> Message-ID: <9D7D3346-F46A-40FB-A7B7-8D00746373AD@miras.org> Try to disable usb in bios before installing. On March 31, 2016 11:59:20 AM GMT+02:00, "Svavar ?rn Eysteinsson" wrote: >So, Solaris 11.3 boots up and installs fine but not OmniOS. >I sure would like to use OmniOS on this machine, so I ask are there any >other debuging >options for me at the installation level ? > >Could I try to install OmniOS on a HDD/SSD on other machine with ACHI >and boot it up on the problematic machine? >Or is that just a no-go from the start? > >Thanks again people. > >Svavar >Reykjavik - Iceland > > > >> On 14. mar. 2016, at 21:28, Christian Kivalo > wrote: >> >> >> >> On 2016-03-09 11:00, Svavar ?rn Eysteinsson wrote: >>> Hi Dan. >>> Thanks for the reply. >>> I disabled all the C-states and it's the same story. >>> If I boot from the SATA CD-ROM installation it hangs on boot right >after : >>> cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >>> cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz >>> cpu7: initialization complete - online >>> cpu6: initialization complete - online >>> See here : >>> >https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG >>> If I boot from USB stick, with USB 2.0 enabled and or disabled the >>> last message is always releated to usb stuff >>> right after cpu6: initialization complete - online. >>> So my BIOS configuration right now is : >>> Intel QPI Frequency Select [Auto Max] >>> Intel Turbo Boots Technology [Enabled] >>> Enhanced Intel Speedstep Tech [Enabled] >>> Processor C3 [Disabled] >>> Processor C6 [Disabled] >>> Intel Hyper-Threading Tech [Disabled] >>> Core Multi-Processing [All] >>> Execute Disable Bit [Disabled] >>> Intel Virtualization Technology [Disasbled] >>> Intel VT for Directed I/O [Disabled] >>> and some more. >>> My Bios Configuration can been seen here : >>> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG >>> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG >>> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG >>> https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG >> Maybe something for you there was a thread here some months ago... >> It starts here >https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03194.html >> >> and some messages into the the thread X2APIC is mentioned >> >https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03200.html >> >> >https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03201.html >> >> and a link a blogpost by jeff sipek >> http://blahg.josefsipek.net/?p=488 >> >> >>> Best regards, >>> Svavar O >>>> On 8. mar. 2016, at 15:13, Dan McDonald wrote: >>>>> On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson > wrote: >>>>> Should I disable other CPU ? Should a enable C2, C3 state in BIOS >or is that not relevant? >>>> Disable C-states, that's general illumos advice (regardless of >distro). >>>> Dan >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> -- >> Christian Kivalo >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ---- This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Thu Mar 31 13:11:15 2016 From: chip at innovates.com (Schweiss, Chip) Date: Thu, 31 Mar 2016 08:11:15 -0500 Subject: [OmniOS-discuss] sshd logging Message-ID: I'm trying to get sshd logging to work on OmniOS with OpenSSH installed. Nothing I try seems to produce any logging. In sshd_config I have: # Syslog facility and level SyslogFacility AUTH LogLevel VERBOSE In /etc/syslog.conf: *.err;kern.notice;auth.notice /dev/sysmsg *.err;kern.debug;daemon.notice;mail.crit /var/adm/messages *.alert;kern.err;daemon.err operator *.alert root *.emerg * # if a non-loghost machine chooses to have authentication messages # sent to the loghost machine, un-comment out the following line: auth.notice ifdef(`LOGHOST', /var/log/authlog, @loghost) mail.debug ifdef(`LOGHOST', /var/log/syslog, @loghost) # # non-loghost machines will use the following lines to cause "user" # log messages to be logged locally. # ifdef(`LOGHOST', , user.err /dev/sysmsg user.err /var/adm/messages user.alert `root, operator' user.emerg * ) I've tried may combinations, in both ssshd_config and syslog.conf. Can someone clue me in on the magic formula? -Chip -------------- next part -------------- An HTML attachment was scrubbed... URL: From svavar at pipar-tbwa.is Thu Mar 31 13:21:32 2016 From: svavar at pipar-tbwa.is (=?UTF-8?q?Svavar_=C3=96rn_Eysteinsson?=) Date: Thu, 31 Mar 2016 13:21:32 +0000 Subject: [OmniOS-discuss] Installation Boot hangs after CPU/USB initialization - Intel Motherboard In-Reply-To: <9D7D3346-F46A-40FB-A7B7-8D00746373AD@miras.org> References: <10A21637-B670-4350-853E-6DA0549E8B20@omniti.com> <7A8A7BDC-23BB-47F6-A6A6-756D1354BE34@pipar-tbwa.is> <9680905E-E048-4E8E-B24C-239131BEE859@pipar-tbwa.is> <9D7D3346-F46A-40FB-A7B7-8D00746373AD@miras.org> Message-ID: <1D511EEF-E560-44CA-B1E4-20568611CAC5@pipar-tbwa.is> I also did that. aka, disabled the usb controller and tried to install omnios through SATA/CDROM. didn't work. :( > On 31. mar. 2016, at 11:16, Michael Rasmussen wrote: > > Try to disable usb in bios before installing. > > On March 31, 2016 11:59:20 AM GMT+02:00, "Svavar ?rn Eysteinsson" wrote: > So, Solaris 11.3 boots up and installs fine but not OmniOS. > I sure would like to use OmniOS on this machine, so I ask are there any other debuging > options for me at the installation level ? > > Could I try to install OmniOS on a HDD/SSD on other machine with ACHI and boot it up on the problematic machine? > Or is that just a no-go from the start? > > Thanks again people. > > Svavar > Reykjavik - Iceland > > > > On 14. mar. 2016, at 21:28, Christian Kivalo wrote: > > > > On 2016-03-09 11:00, Svavar ?rn Eysteinsson wrote: > Hi Dan. > Thanks for the reply. > I disabled all the C-states and it's the same story. > > If I boot from the SATA CD-ROM installation it hangs on boot right after : > cpu7: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu6: Intel(r) Xeon(r) CPU E5520 @ 2.27Ghz > cpu7: initialization complete - online > cpu6: initialization complete - online > See here : > https://dl.dropboxusercontent.com/u/16134/omnios_intel_cstates_disabled_boot.JPG > If I boot from USB stick, with USB 2.0 enabled and or disabled the > last message is always releated to usb stuff > right after cpu6: initialization complete - online. > So my BIOS configuration right now is : > Intel QPI Frequency Select [Auto Max] > Intel Turbo Boots Technology [Enabled] > Enhanced Intel Speedstep Tech [Enabled] > Processor C3 [Disabled] > Processor C6 [Disabled] > Intel Hyper-Threading Tech [Disabled] > Core Multi-Processing [All] > Execute Disable Bit > [Disabled] > Intel Virtualization Technology [Disasbled] > Intel VT for Directed I/O [Disabled] > and some more. > My Bios Configuration can been seen here : > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios1.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios2.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios3.JPG > https://dl.dropboxusercontent.com/u/16134/omnios_intel_bios4.JPG > Maybe something for you there was a thread here some months ago... > It starts here https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03194.html > > and some messages into the the thread X2APIC is mentioned > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03200.html > > https://www.mail-archive.com/omnios-discuss%40lists.omniti.com/msg03201.html > > and a link a blogpost by jeff sipek > http://blahg.josefsipek.net/?p=488 > > > Best regards, > Svavar O > On 8. mar. 2016, at 15:13, Dan McDonald wrote: > On Mar 8, 2016, at 6:17 AM, Svavar ?rn Eysteinsson wrote: > Should I disable other CPU ? Should a enable C2, C3 state in BIOS or is that not relevant? > Disable C-states, that's general illumos advice (regardless of distro). > Dan > > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- > Christian Kivalo > > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > ---- This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lotheac at iki.fi Thu Mar 31 13:34:40 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Thu, 31 Mar 2016 16:34:40 +0300 Subject: [OmniOS-discuss] sshd logging In-Reply-To: References: Message-ID: <20160331133440.GB22952@kekkonen.niksula.hut.fi> On Thu, Mar 31 2016 08:11:15 -0500, Schweiss, Chip wrote: > In sshd_config I have: > > # Syslog facility and level > SyslogFacility AUTH > LogLevel VERBOSE You tell it to log to the auth facility, but you also need to tell syslog to put auth messages where you want them. > *.err;kern.notice;auth.notice /dev/sysmsg You ask that auth messages higher than 'notice' level go to the console. A quick glance at my logs reveals most messages logged by sshd are at the 'info' level (which is lower than notice). > auth.notice ifdef(`LOGHOST', /var/log/authlog, @loghost) According to syslog.conf(4), this means that if the current machine has the same address as 'loghost' (which I think is an alias for localhost in the default /etc/hosts), then put auth.notice and above to /var/log/authlog, otherwise send them to the 'loghost' machine. But again, your log level is 'notice'; try bumping it to 'info' (and make sure that /var/log/authlog exists if that's where you want these logs). -- Lauri Tirkkonen | lotheac @ IRCnet From bfriesen at simple.dallas.tx.us Thu Mar 31 13:40:35 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 31 Mar 2016 08:40:35 -0500 (CDT) Subject: [OmniOS-discuss] Ang: Continuing zone shutdown issues with r151016 In-Reply-To: References: <0F636D59-DA67-4B58-9E13-E632BA0F22F0@omniti.com> Message-ID: On Thu, 31 Mar 2016, Jim Klimov wrote: > > Forgot to add: the cases similar to what i described can be inspected with fuser (-c, -m IIRC) to see if any processes block the zoneroot filesystems. Also blockage can be due to mounts underneath and other kernel-provided blockers, that userspace fuser won't show. > > In particular then look out for nfs+autofs client in a zone (something sol10 documented as a no-no, especiakly to not use nfs from gz of the same server, but which just worked and was cool to e.g. share homedirs or distribs, and so was widely used) - perhaps autofs failed to unmount something from a remote host, then was killed by lz shutdown, and then the remaining sub-fs blocks the zoneroot teardown. This is not my situation. None of the zones are currently doing nfs client mounts or have any dependence on the root zone other than lofi mounts to a zfs filesystem. However, there is a "Device busy" type error. Some may remember that there are messages on the console which suggest a resource contention problem and that I previously captured some information and made it available under this ftp URL: ftp://ftp.simplesystems.org/pub/outgoing/omnios/zoneadmd/ The zoneadmd program issues logs with text like: "failed to open console master: Device busy" " WARNING: could not open master side of zone console for pkgbuild to release slave handle: Device busy" "WARNING: console /devices//pseudo/zconsnex at 1/zcons at 1 found, but it could not be removed.: I/O error" It is not clear if this "console master" is continually busy (someting forgot/failed to close it) or if it is a transient condition which would clear itself and be fine after waiting some tens of milliseconds and retrying. What is clear is that the condition continues after it first occurs although other usages of the zone are fine. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From danmcd at omniti.com Thu Mar 31 14:40:48 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 31 Mar 2016 10:40:48 -0400 Subject: [OmniOS-discuss] OpenSSL futures Message-ID: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> As I'm updating and checking packages for r151018's release, I notice that OpenSSL is rapidly approaching a 1.1.0 release. I visited their Release Strategy page: http://openssl.org/policies/releasestrat.html And noticed that 1.0.2 is LTS until 2019. OTOH, 1.1.x will likely become LTS for some x, and there's also LibreSSL... I'm starting this thread to hear what the community has to say about where OmniOS should go w.r.t. its OpenSSL release. I have internal customers too, of course, but I'll engage them separately. We need to have an OpenSSL because illumos requires one. We *could* do the SmartOS thing and keep our own SUNW/OMNI*...() api set, though. I can't guarantee what I'll ultimately decide, but knowing what people think can't hurt. Thanks, Dan From lotheac at iki.fi Thu Mar 31 14:53:28 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Thu, 31 Mar 2016 17:53:28 +0300 Subject: [OmniOS-discuss] OpenSSL futures In-Reply-To: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> References: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> Message-ID: <20160331145328.GA25765@kekkonen.niksula.hut.fi> On Thu, Mar 31 2016 10:40:48 -0400, Dan McDonald wrote: > And noticed that 1.0.2 is LTS until 2019. OTOH, 1.1.x will likely become LTS for some x, and there's also LibreSSL... > > I'm starting this thread to hear what the community has to say about where OmniOS should go w.r.t. its OpenSSL release. I have internal customers too, of course, but I'll engage them separately. We need to have an OpenSSL because illumos requires one. We *could* do the SmartOS thing and keep our own SUNW/OMNI*...() api set, though. I would personally like to see LibreSSL either in OmniOS itself or the possibility to use it in my own application stack (eg. the SmartOS route of changing the symbol names of the OS-shipped TLS library to avoid conflicts). -- Lauri Tirkkonen | lotheac @ IRCnet From bfriesen at simple.dallas.tx.us Thu Mar 31 14:56:43 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 31 Mar 2016 09:56:43 -0500 (CDT) Subject: [OmniOS-discuss] OpenSSL futures In-Reply-To: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> References: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> Message-ID: On Thu, 31 Mar 2016, Dan McDonald wrote: > > I'm starting this thread to hear what the community has to say about > where OmniOS should go w.r.t. its OpenSSL release. I have internal > customers too, of course, but I'll engage them separately. We need > to have an OpenSSL because illumos requires one. We *could* do the > SmartOS thing and keep our own SUNW/OMNI*...() api set, though. Currently many OmniOS users are building their own applications using the OpenSSL that comes with OmniOS. If OmniOS removes the OpenSSL version that these applications were using, then the applications will fail to run. While it is useful to offer the current OpenSSL release branch (OmniOS provides "fresh" software), there needs to be a way to bridge so that OmniOS users are not struck with severely broken servers (requiring that all packages depending on OpenSSL be rebuilt) due to stepping to the next OmniOS release. With sufficient advance notice we can build our own OpenSSL and rebuild all of our packages to use our own OpenSSL and update application configurations (e.g certificates) so that they still work. By doing so we lose the security-fix support that OmniOS has been providing and need to take this reposibility for ourselves. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From bfriesen at simple.dallas.tx.us Thu Mar 31 15:01:43 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 31 Mar 2016 10:01:43 -0500 (CDT) Subject: [OmniOS-discuss] OpenSSL futures In-Reply-To: <20160331145328.GA25765@kekkonen.niksula.hut.fi> References: <90D036FC-34E8-412A-828A-4ECC6CB1907A@omniti.com> <20160331145328.GA25765@kekkonen.niksula.hut.fi> Message-ID: On Thu, 31 Mar 2016, Lauri Tirkkonen wrote: > > I would personally like to see LibreSSL either in OmniOS itself or the > possibility to use it in my own application stack (eg. the SmartOS route > of changing the symbol names of the OS-shipped TLS library to avoid > conflicts). While the notion of LibreSSL is great, we should take care because LibreSSL is targeted to OpenBSD APIs and requirements. Some behaviors have subtle (yet significant) differences and the differences need to be carefully considered to make sure that behavior is correct under Illumos. For Linux there are known cases where the OS difference resulted in a security weakness (which may still exist). Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From alka at hfg-gmuend.de Thu Mar 31 15:46:27 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Thu, 31 Mar 2016 17:46:27 +0200 Subject: [OmniOS-discuss] OmniOS 151016/ 151017 not listed in network environment Message-ID: <56FD4653.6060806@hfg-gmuend.de> We have an AD environment where the AD server is the master browser to list all servers and workstations on our Windows machines under network This works for a couple of OmniOS 151014 servers but not for my 151016/ 151017 testservers. Is this a known problem or is there something that I should consider? From danmcd at omniti.com Thu Mar 31 17:02:15 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 31 Mar 2016 13:02:15 -0400 Subject: [OmniOS-discuss] OmniOS 151016/ 151017 not listed in network environment In-Reply-To: <56FD4653.6060806@hfg-gmuend.de> References: <56FD4653.6060806@hfg-gmuend.de> Message-ID: <4C8EDCED-A272-43D7-A30E-DAF2771968A9@omniti.com> > On Mar 31, 2016, at 11:46 AM, Guenther Alka wrote: > > We have an AD environment where the AD server is the master browser > to list all servers and workstations on our Windows machines under network > > This works for a couple of OmniOS 151014 servers but not for my 151016/ 151017 testservers. Is this a known problem or is there something that I should consider? Are you using the in-kernel SMB service? That got updated in a big way as part of 016 (non-global-zone SMB serving), and more is coming in 018. I'm not a Windows/AD expert, so please, when reporting AD issues, be a bit more pedantic in your explanation. You should also check the illumos mailing list archives as well. Thanks, Dan From alka at hfg-gmuend.de Thu Mar 31 18:30:25 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Thu, 31 Mar 2016 20:30:25 +0200 Subject: [OmniOS-discuss] OmniOS 151016/ 151017 not listed in network environment In-Reply-To: <4C8EDCED-A272-43D7-A30E-DAF2771968A9@omniti.com> References: <56FD4653.6060806@hfg-gmuend.de> <4C8EDCED-A272-43D7-A30E-DAF2771968A9@omniti.com> Message-ID: <56FD6CC0.50508@hfg-gmuend.de> Hallo Dan This is not related to Windows AD but a problem that the Solarish kernel SMB server does not offer a master browser capability that is needed to provide the list of SMB servers in Windows "network neighborhood". So you need a Windows or SAMBA server in the subnet that provides this feature. This worked together with my 151014 boxes that are listed in network neighborhood see https://docs.oracle.com/cd/E36784_01/html/E36832/winclientcannotcontactbynetbios.html thanks Gea Am 31.03.2016 um 19:02 schrieb Dan McDonald: >> On Mar 31, 2016, at 11:46 AM, Guenther Alka wrote: >> >> We have an AD environment where the AD server is the master browser >> to list all servers and workstations on our Windows machines under network >> >> This works for a couple of OmniOS 151014 servers but not for my 151016/ 151017 testservers. Is this a known problem or is there something that I should consider? > Are you using the in-kernel SMB service? That got updated in a big way as part of 016 (non-global-zone SMB serving), and more is coming in 018. > > I'm not a Windows/AD expert, so please, when reporting AD issues, be a bit more pedantic in your explanation. You should also check the illumos mailing list archives as well. > > Thanks, > Dan >