From danmcd at omniti.com Thu Sep 1 22:18:57 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 1 Sep 2016 18:18:57 -0400 Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS - zonecfg(1M) networking Message-ID: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com> Aka. "A tale of two LX zonecfgs". I've updated the io-lx-public repo https://github.com/danmcd/io-lx-public/ AND the IPS server for bloody+LX: http://pkg.omniti.com/omnios/LX/ to include the Joyent-style zonecfg(1M) method of LX zone network configuration. For example, a worked LX zone before would have: # zonecfg -z lx0 export create -b set zonepath=/zones/lx0 set brand=lx set autoboot=false set ip-type=exclusive add net set physical=lx0 end add attr set name=kernel-version set type=string set value=2.6.32 end # But now, you can add attributes to pre-configure the LX zone's basic networking (changes prefixed with '+'): # zonecfg -z lx0 export create -b set zonepath=/zones/lx0 set brand=lx set autoboot=false set ip-type=exclusive add net set physical=lx0 + add property (name=gateway,value="192.168.0.1") + add property (name=ips,value="192.168.0.69/24") + add property (name=primary,value="true") end + add attr + set name=dns-domain + set type=string + set value=example.com + end + add attr + set name=resolvers + set type=string + set value=192.168.0.1 + end add attr set name=kernel-version set type=string set value=2.6.32 end # And your LX zone, thanks to the lxinit(1M) binary will do the right thing. Update your LX test boxes to this, and see how it works. Thanks, Dan From danmcd at omniti.com Thu Sep 1 22:25:24 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 1 Sep 2016 18:25:24 -0400 Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS - zonecfg(1M) networking In-Reply-To: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com> References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com> Message-ID: <66468812-08E9-40CE-8E98-1853723B5690@omniti.com> > On Sep 1, 2016, at 6:18 PM, Dan McDonald wrote: > > Aka. "A tale of two LX zonecfgs". > > I've updated the io-lx-public repo > > https://github.com/danmcd/io-lx-public/ > > AND the IPS server for bloody+LX: > > http://pkg.omniti.com/omnios/LX/ I almost forgot: The webrevs are updated too: http://kebe.com/~danmcd/webrevs/io-lx-public/ http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/ Dan From mir at miras.org Thu Sep 1 23:19:59 2016 From: mir at miras.org (Michael Rasmussen) Date: Fri, 2 Sep 2016 01:19:59 +0200 Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS - zonecfg(1M) networking In-Reply-To: <66468812-08E9-40CE-8E98-1853723B5690@omniti.com> References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com> <66468812-08E9-40CE-8E98-1853723B5690@omniti.com> Message-ID: <20160902011959.4455e3de@sleipner.datanom.net> On Thu, 1 Sep 2016 18:25:24 -0400 Dan McDonald wrote: > > I almost forgot: The webrevs are updated too: > > http://kebe.com/~danmcd/webrevs/io-lx-public/ > > http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/ > For both links: Forbidden You don't have permission to access /~danmcd/webrevs/io-lx-public/ on this server. ??? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: I can resist anything but temptation. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Fri Sep 2 00:53:16 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 1 Sep 2016 20:53:16 -0400 Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS - zonecfg(1M) networking In-Reply-To: <20160902011959.4455e3de@sleipner.datanom.net> References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com> <66468812-08E9-40CE-8E98-1853723B5690@omniti.com> <20160902011959.4455e3de@sleipner.datanom.net> Message-ID: <46548612-3343-47EE-9FDA-0798FFB53DD9@omniti.com> > On Sep 1, 2016, at 7:19 PM, Michael Rasmussen wrote: > > On Thu, 1 Sep 2016 18:25:24 -0400 > Dan McDonald wrote: > >> >> I almost forgot: The webrevs are updated too: >> >> http://kebe.com/~danmcd/webrevs/io-lx-public/ >> >> http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/ >> > For both links: > Forbidden > > You don't have permission to access /~danmcd/webrevs/io-lx-public/ on this server. Please try now. Sorry about that, Dan From hannohirschberger at googlemail.com Fri Sep 2 08:25:45 2016 From: hannohirschberger at googlemail.com (Hanno Hirschberger) Date: Fri, 2 Sep 2016 10:25:45 +0200 Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon E3-1240 v3 Message-ID: Hey guys! I run OmniOS r151014 on a SuperMicro server with an Intel Xeon E3-1240 v3 CPU. We need kvm for a qemu-kvm machine but the kvm device is missing. The symlink /dev/kvm is there but there's no /devices/pseudo/kvm at 0:kvm where it's pointing. I enabled Intel VT-x on the mainboard. The kernel module files are present but OmniOS is unable to load them ("file not found" - obviously). So I'm stuck now and don't know how to analyze this further. Please help me out! Do you have any hints or encountered a similar problem? Is the processor unsupported by any chance? The mainboard is a Supermicro Server Board X10SLM-F. Let me know if you need anything. Best regards, Hanno From danmcd at omniti.com Fri Sep 2 14:06:02 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 2 Sep 2016 10:06:02 -0400 Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon E3-1240 v3 In-Reply-To: References: Message-ID: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com> > On Sep 2, 2016, at 4:25 AM, Hanno Hirschberger wrote: > > Hey guys! > > I run OmniOS r151014 on a SuperMicro server with an Intel Xeon E3-1240 v3 CPU. We need kvm for a qemu-kvm machine but the kvm device is missing. This has nothing to do with the processor. > The symlink /dev/kvm is there but there's no /devices/pseudo/kvm at 0:kvm where it's pointing. I enabled Intel VT-x on the mainboard. The kernel module files are present but OmniOS is unable to load them ("file not found" - obviously). So I'm stuck now and don't know how to analyze this further. Please help me out! Please do this: pkg list kvm You should see two lines of output: r151014(~)[0]% pkg list kvm NAME (PUBLISHER) VERSION IFO driver/virtualization/kvm 1.0.5.11-0.151014 i-- system/kvm 1.0.5.11-0.151014 i-- r151014(~)[0]% Are those packages missing? If so, "pkg install" them both. It's strange that they would be missing, however, given they're part of the "entire" consolidation. If they are there in "pkg list", you may wish to "pkg verify driver/virtualization/kvm system/kvm" to make sure. Dan From danmcd at omniti.com Sat Sep 3 05:48:18 2016 From: danmcd at omniti.com (Dan McDonald) Date: Sat, 3 Sep 2016 01:48:18 -0400 Subject: [OmniOS-discuss] I broke Centos/Redhat LX zones, plus -s now takes ZFS filesystem/snapshot Message-ID: <527A65A1-E20F-415F-AF3D-514F22F5582B@omniti.com> I've just updated the LX brand package on the http://pkg.omniti.com/omnios/LX repo. Besides fixing Centos/Redhat *and* busybox LX zones, I've included a new method for -s on install, where it takes a filesystem argument, in lieu of a pathname to a ZFS send stream. If you have a "images" dataset with filesystems in it, you can just do: zoneadm -z lx0 install -s rpool/images/centos6 for example. Sorry about the bug, and enjoy the image-clone-install, Dan From johan.kragsterman at capvert.se Sat Sep 3 11:05:02 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Sat, 3 Sep 2016 13:05:02 +0200 Subject: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1 wwn.2100001B321EDFB9 Message-ID: Hi! I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one? The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's. The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle. So, how do YOU do it? And why...? Opinions pls...? Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert From alka at hfg-gmuend.de Sat Sep 3 11:57:30 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Sat, 3 Sep 2016 13:57:30 +0200 Subject: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1 wwn.2100001B321EDFB9 In-Reply-To: References: Message-ID: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de> You can control access and visibility of LUNs either based on Initiators (host groups), targets (target groups), network links (target portal groups) or views that make a logical unit visible in a target. Every solution works with a different use case idea. In napp-it I am currently working on full appliance redundancy with Z-Raid pool mirroring over iSCSI for a storage and service failover cluster. I have played with the solutions above but in the end I decided to use none of them but use manual target discovery with switchable IP aliases for iSCSI targets to restrict and switchover LUNs and for NFS/SMB failover between the active master storage appliance and the slave. The reason was simplicity and that this works on a lower level that does not rely to Comstar timings and delays. Gea Am 03.09.2016 um 13:05 schrieb Johan Kragsterman: > Hi! > > > I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one? > > The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's. > > The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle. > > So, how do YOU do it? And why...? > > Opinions pls...? > > > Best regards from/Med v?nliga h?lsningar fr?n > > Johan Kragsterman > > Capvert > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From johan.kragsterman at capvert.se Sat Sep 3 12:15:56 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Sat, 3 Sep 2016 14:15:56 +0200 Subject: [OmniOS-discuss] Ang: Re: stmfadm add-hg-member -F -g cluster1 wwn.2100001B321EDFB9 In-Reply-To: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de> References: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de>, Message-ID: Hi! Thanks for the input, Gea! -----"OmniOS-discuss" skrev: ----- Till: omnios-discuss at lists.omniti.com Fr?n: Guenther Alka S?nt av: "OmniOS-discuss" Datum: 2016-09-03 12:01 ?rende: Re: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1 wwn.2100001B321EDFB9 You can control access and visibility of LUNs either based on Initiators (host groups), targets (target groups), network links (target portal groups) or views that make a logical unit visible in a target. Every solution works with a different use case idea. In napp-it I am currently working on full appliance redundancy with Z-Raid pool mirroring over iSCSI for a storage and service failover cluster. ?I have played with the solutions above but in the end I decided to use none of them but use manual target discovery with switchable IP aliases for iSCSI targets to restrict and switchover LUNs and for NFS/SMB failover between the active master storage appliance and the slave. The reason was simplicity and that this works on a lower level that does not rely to Comstar timings and delays. Gea I mainly use FC, and never iScsi, so my control mechanisms are host and target groups, and views. My setups are often that I run clusters with multiple host machines, which runs multiple virtual machines, and if I should map every lu with a new view it would be a hell of a job. It is much easier to just put the wwn in the cluster host group, than to maintain all these views. Rgrds Johan Am 03.09.2016 um 13:05 schrieb Johan Kragsterman: > Hi! > > > I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one? > > The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's. > > The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle. > > So, how do YOU do it? And why...? > > Opinions pls...? > > > Best regards from/Med v?nliga h?lsningar fr?n > > Johan Kragsterman > > Capvert > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From bfriesen at simple.dallas.tx.us Sat Sep 3 21:42:53 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Sat, 3 Sep 2016 16:42:53 -0500 (CDT) Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS? Message-ID: Is the LSI 9300-16i SAS support in OmniOS known to be robust? I am planning to use it with the SuperMicro SC836BA-R920B 3U case and Supermicro X10DRi-T4+ motherboard to build a storage server. Is there anything about this HBA and motherboard that might cause an issue? Thanks, Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From peter.tribble at gmail.com Sun Sep 4 09:00:36 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Sun, 4 Sep 2016 10:00:36 +0100 Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS? In-Reply-To: References: Message-ID: On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > Is the LSI 9300-16i SAS support in OmniOS known to be robust? > > I am planning to use it with the SuperMicro SC836BA-R920B 3U case and > Supermicro X10DRi-T4+ motherboard to build a storage server. > > Is there anything about this HBA and motherboard that might cause an issue? > Should be good. We're using that X10DRi-T4+ motherboard with the LSI 9300-8i HBA (in the 2U chassis, where the 16i wouldn't fit), and everything has worked perfectly so far. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Sun Sep 4 14:46:45 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Sun, 4 Sep 2016 09:46:45 -0500 (CDT) Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS? In-Reply-To: References: Message-ID: On Sun, 4 Sep 2016, Peter Tribble wrote: > On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn < > bfriesen at simple.dallas.tx.us> wrote: > >> Is the LSI 9300-16i SAS support in OmniOS known to be robust? >> >> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and >> Supermicro X10DRi-T4+ motherboard to build a storage server. >> >> Is there anything about this HBA and motherboard that might cause an issue? >> > > Should be good. We're using that X10DRi-T4+ motherboard with the LSI > 9300-8i HBA (in the 2U chassis, > where the 16i wouldn't fit), and everything has worked perfectly so far. That is good to hear. I did not see any hardware on the motherboard which is not supported by OmniOS. Something which is not clear to me (or to my computer builder) is if using a 12gb SAS HBA and 12gb SAS disks with a passive backplane described by SuperMicro as being 3g/6g might cause issues. It is not clear if the end devices somehow detect that the chassis is certified for 6g or if they will use 12g and possibly work just fine, or encounter issues. The signalling and cables for 6g and 12g are the same, but with 12g using a higher clocking rate. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From matej at zunaj.si Mon Sep 5 06:58:57 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Mon, 5 Sep 2016 08:58:57 +0200 Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS? In-Reply-To: References: Message-ID: We are using SAS3 drives with LSI 9300-8e and SC837E26-RJBOD1 and it?s working great for the last 4 months. I?m not sure if it?s running SAS2 or SAS3, since I dont see those information nowhere (I checked messages and sas3ircu). It only says SAS Matej > On 04 Sep 2016, at 16:46, Bob Friesenhahn wrote: > > On Sun, 4 Sep 2016, Peter Tribble wrote: > >> On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn < >> bfriesen at simple.dallas.tx.us> wrote: >> >>> Is the LSI 9300-16i SAS support in OmniOS known to be robust? >>> >>> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and >>> Supermicro X10DRi-T4+ motherboard to build a storage server. >>> >>> Is there anything about this HBA and motherboard that might cause an issue? >>> >> >> Should be good. We're using that X10DRi-T4+ motherboard with the LSI >> 9300-8i HBA (in the 2U chassis, >> where the 16i wouldn't fit), and everything has worked perfectly so far. > > That is good to hear. I did not see any hardware on the motherboard which is not supported by OmniOS. > > Something which is not clear to me (or to my computer builder) is if using a 12gb SAS HBA and 12gb SAS disks with a passive backplane described by SuperMicro as being 3g/6g might cause issues. It is not clear if the end devices somehow detect that the chassis is certified for 6g or if they will use 12g and possibly work just fine, or encounter issues. The signalling and cables for 6g and 12g are the same, but with 12g using a higher clocking rate. > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From hannohirschberger at googlemail.com Mon Sep 5 08:13:23 2016 From: hannohirschberger at googlemail.com (Hanno Hirschberger) Date: Mon, 5 Sep 2016 10:13:23 +0200 Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon E3-1240 v3 In-Reply-To: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com> References: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com> Message-ID: <564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com> Sorry Dan, forgot to reply to the mailing list. On 02.09.2016 16:06, Dan McDonald wrote: > Please do this: > pkg list kvm > [...] > > Are those packages missing? If so, "pkg install" them both. Both are installed. Here's the output: NAME (PUBLISHER) VERSION IFO driver/virtualization/kvm 1.0.5.11-0.151014 i-- system/kvm 1.0.5.11-0.151014 i-- > [...] If they are there in "pkg list", you may wish to "pkg verify driver/virtualization/kvm system/kvm" to make sure. Now it's getting interesting. Here's the error the verification is telling me: PACKAGE STATUS pkg://omnios/drive/virtualization/kvm ERROR driver: kvm etc/name_to_major: 'kvm' entry not present I checked the file and indeed there's no such entry. I mounted a few older boot environments. There's no 'kvm' entry in any of the /etc/name_to_major files. Do you have any idea how to tackle this? Hanno From an3s.annema at gmail.com Mon Sep 5 09:51:06 2016 From: an3s.annema at gmail.com (Andries Annema) Date: Mon, 5 Sep 2016 11:51:06 +0200 Subject: [OmniOS-discuss] r151014 repo offline? In-Reply-To: References: <8ecefd54-9758-98c8-d83e-0935d0aec4fd@gmail.com> <52CBF697-0C20-4665-A7FF-119B3381F0DB@omniti.com> Message-ID: <000301d2075b$0b0f5be0$212e13a0$@gmail.com> Hi Dan, I hate to bring this matter up again, but it seems there's still something loose here. This page on itself works flawlessly: http://pkg.omniti.com/omnios/r151014/en/index.shtml But upon hitting the "browse" button next to the tiny little "Release and Branch" pull-down menu, set to the default value of "11-0.151014", the catalog.shtml-page that should come up fails with a read timeout. The address bar then shows this URL: http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=entire%4011%2C 5.11-0.151014&action=Browse However - and this surprised me and it might give you a hint on where the loose screw is located - when selecting "All" in that pull-down menu, after about 30 seconds (maybe a bit slow?), a catalog.shtml-page with a package list is indeed displayed. The address bar then shows this URL: http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=&action=Browse Furthermore, when accessing from OmniOS by running: $ pkgrepo info -s http://pkg.omniti.com/omnios/r151014 it reports: PUBLISHER PACKAGES STATUS UPDATED omnios 1022 processing 2016-08-10T18:35:55.166990Z Not sure if the "processing" status here is of any concern. While investigating this and writing this report, I also noticed that for some time the whole thing seemed to collapse: suddenly the "pkgrepo info" command reported "500 Internal Server Error" and later "Framework error: code: 28 reason: Operation too slow.", and all the webpages mentioned above timed out. I waited half an hour or so and then it seemed to have gotten on its feet again, as all responded as reported above. This repo seems a bit shaky. Hope this report is of any help in getting it blazing again. Regards, Andries -----Original Message----- From: Dan McDonald [mailto:danmcd at omniti.com] Sent: zaterdag 30 juli 2016 22:01 To: omnios-discuss Cc: Andreas Luka; Andries Annema; Dan McDonald Subject: Re: [OmniOS-discuss] r151014 repo offline? I checked the internal-side of things, and also had problems with catalog.shtml. I've since done "pkgrepo rebuild" and "pkgrepo refresh", and I got a catalog.shtml list, but it took a long time. I will investigate further on Monday. Thanks for the report, Dan From davide.poletto at gmail.com Mon Sep 5 12:03:18 2016 From: davide.poletto at gmail.com (Davide Poletto) Date: Mon, 5 Sep 2016 14:03:18 +0200 Subject: [OmniOS-discuss] r151014 repo offline? In-Reply-To: <000301d2075b$0b0f5be0$212e13a0$@gmail.com> References: <8ecefd54-9758-98c8-d83e-0935d0aec4fd@gmail.com> <52CBF697-0C20-4665-A7FF-119B3381F0DB@omniti.com> <000301d2075b$0b0f5be0$212e13a0$@gmail.com> Message-ID: Me too, here it's the same: pkgrepo info -s http://pkg.omniti.com/omnios/r151014 PUBLISHER PACKAGES STATUS UPDATED omnios 1022 processing 2016-08-10T18:35:55.166990Z And, as Andries pointed out yet, just browsing the 11-0.151014 Release returns a solid "Inactivity Timeout" (an HTTP/1.1 504 Connection Timed Out 181660 ms for the GET) showing a web page with this URL: http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=entire%4011%2C5.11-0.151014&action=Browse AFAIK browsing the http://pkg.omniti.com/omnios/r151014/en/index.shtml doesn't look stable at all (few minutes ago I was able to reach the page very slowly, now it ends with a timeout) Tested using two different connections through two different ISPs (Italy). On Mon, Sep 5, 2016 at 11:51 AM, Andries Annema wrote: > Hi Dan, > > I hate to bring this matter up again, but it seems there's still something > loose here. > > This page on itself works flawlessly: > http://pkg.omniti.com/omnios/r151014/en/index.shtml > But upon hitting the "browse" button next to the tiny little "Release and > Branch" pull-down menu, set to the default value of "11-0.151014", the > catalog.shtml-page that should come up fails with a read timeout. The > address bar then shows this URL: > http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?versio > n=entire%4011%2C > 5.11-0.151014&action=Browse > > > However - and this surprised me and it might give you a hint on where the > loose screw is located - when selecting "All" in that pull-down menu, after > about 30 seconds (maybe a bit slow?), a catalog.shtml-page with a package > list is indeed displayed. The address bar then shows this URL: > http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?versio > n=&action=Browse > > > Furthermore, when accessing from OmniOS by running: > $ pkgrepo info -s http://pkg.omniti.com/omnios/r151014 > it reports: > PUBLISHER PACKAGES STATUS UPDATED > omnios 1022 processing 2016-08-10T18:35:55.166990Z > Not sure if the "processing" status here is of any concern. > > While investigating this and writing this report, I also noticed that for > some time the whole thing seemed to collapse: suddenly the "pkgrepo info" > command reported "500 Internal Server Error" and later "Framework error: > code: 28 reason: Operation too slow.", and all the webpages mentioned above > timed out. > I waited half an hour or so and then it seemed to have gotten on its feet > again, as all responded as reported above. > > This repo seems a bit shaky. Hope this report is of any help in getting it > blazing again. > > Regards, > Andries > > > -----Original Message----- > From: Dan McDonald [mailto:danmcd at omniti.com] > Sent: zaterdag 30 juli 2016 22:01 > To: omnios-discuss > Cc: Andreas Luka; Andries Annema; Dan McDonald > Subject: Re: [OmniOS-discuss] r151014 repo offline? > > I checked the internal-side of things, and also had problems with > catalog.shtml. > > I've since done "pkgrepo rebuild" and "pkgrepo refresh", and I got a > catalog.shtml list, but it took a long time. > > I will investigate further on Monday. > > Thanks for the report, > Dan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info.fmo at gmx.de Tue Sep 6 13:18:06 2016 From: info.fmo at gmx.de (Frank M.) Date: Tue, 6 Sep 2016 15:18:06 +0200 Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS? In-Reply-To: References: Message-ID: <976878138.20160906151806@gmx.de> Hi, I have an new system with a LSI 9300-16i in slot3 of a X10DRi-T4+ motherboard on a 6G-Backplane. In my short tests with barmetal OmniOS there were no problems. But I would use VMware-virtualized OmniOS with passthrough LSI-HBA. And here I have massive problems... The disks/pools crash after a short time with low load. I testet the HBA in another PCIe-slot - I havn?t seen any problems, but I havn?t looked into the logfile. Supermicro recommended to set jumper JI2C1 and JI2C2 to 2-3 - but they were set to this already. So I set the jumper to 1-2. Now disks/pools crash still earlier. The HBA gets very hot, but I testet it also with "special" airflow - the same.... I attached the logfile... What does it mean "FastPath": ---- Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w5000cca24200d4fd FastPath Capable and Enabled Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w4433221106000000 FastPath Capable but Disabled Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w5000cca24200d4c9 FastPath Capable and Enabled Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w5000cca2426fff85 FastPath Capable and Enabled Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w5000cca24200d33d FastPath Capable and Enabled Sep 6 15:07:05 test-san1 scsi: [ID 243001 kern.info] w5000cca24200d4c9 FastPath Capable and Enabled ---- and later: ---- Sep 6 15:10:19 test-san1 scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:19 test-san1 MPT Firmware Fault, code: 5854 Sep 6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:21 test-san1 MPT Firmware version v12.0.0.0 (SAS3008) Sep 6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:21 test-san1 mpt_sas19 SAS 3 Supported Sep 6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:21 test-san1 mpt19: IOC Operational. Sep 6 15:10:28 test-san1 scsi: [ID 243001 kern.info] w4433221100000000 FastPath Capable but Disabled Sep 6 15:10:28 test-san1 last message repeated 1 time Sep 6 15:10:28 test-san1 scsi: [ID 243001 kern.info] w4433221101000000 FastPath Capable but Disabled Sep 6 15:10:28 test-san1 last message repeated 1 time Sep 6 15:10:29 test-san1 scsi: [ID 243001 kern.info] w4433221102000000 FastPath Capable but Disabled Sep 6 15:10:29 test-san1 last message repeated 1 time Sep 6 15:10:29 test-san1 scsi: [ID 243001 kern.info] w4433221103000000 FastPath Capable but Disabled Sep 6 15:10:29 test-san1 last message repeated 1 time Sep 6 15:10:29 test-san1 scsi: [ID 243001 kern.info] w4433221106000000 FastPath Capable but Disabled Sep 6 15:10:29 test-san1 last message repeated 1 time Sep 6 15:10:29 test-san1 scsi: [ID 243001 kern.info] w4433221107000000 FastPath Capable but Disabled Sep 6 15:10:29 test-san1 last message repeated 1 time Sep 6 15:10:31 test-san1 scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:31 test-san1 MPT Firmware Fault, code: 5854 Sep 6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:32 test-san1 MPT Firmware version v12.0.0.0 (SAS3008) Sep 6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:32 test-san1 mpt_sas19 SAS 3 Supported Sep 6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19): Sep 6 15:10:32 test-san1 mpt19: IOC Operational. Sep 6 15:10:40 test-san1 scsi: [ID 243001 kern.info] w4433221100000000 FastPath Capable but Disabled Sep 6 15:10:40 test-san1 scsi: [ID 243001 kern.info] w4433221101000000 FastPath Capable but Disabled Sep 6 15:10:40 test-san1 scsi: [ID 243001 kern.info] w4433221102000000 FastPath Capable but Disabled Sep 6 15:10:40 test-san1 scsi: [ID 243001 kern.info] w4433221103000000 FastPath Capable but Disabled Sep 6 15:10:41 test-san1 scsi: [ID 243001 kern.info] w4433221106000000 FastPath Capable but Disabled Sep 6 15:10:41 test-san1 scsi: [ID 243001 kern.info] w4433221107000000 FastPath Capable but Disabled ---- For information: pool1 6x SATA-SSD Raidz2 pool2 4x SAS-HDD 2 mirrored vdev with SATA-SSD l2arc I hope anybody has any useful hints... PS: Tonight I want to test with only one HDD... Frank Sunday, September 4, 2016, 4:46:45 PM, you wrote: BF> On Sun, 4 Sep 2016, Peter Tribble wrote: >> On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn < >> bfriesen at simple.dallas.tx.us> wrote: >> >>> Is the LSI 9300-16i SAS support in OmniOS known to be robust? >>> >>> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and >>> Supermicro X10DRi-T4+ motherboard to build a storage server. >>> >>> Is there anything about this HBA and motherboard that might cause an issue? >>> >> >> Should be good. We're using that X10DRi-T4+ motherboard with the LSI >> 9300-8i HBA (in the 2U chassis, >> where the 16i wouldn't fit), and everything has worked perfectly so far. BF> That is good to hear. I did not see any hardware on the motherboard BF> which is not supported by OmniOS. BF> Something which is not clear to me (or to my computer builder) is if BF> using a 12gb SAS HBA and 12gb SAS disks with a passive backplane BF> described by SuperMicro as being 3g/6g might cause issues. It is not BF> clear if the end devices somehow detect that the chassis is certified BF> for 6g or if they will use 12g and possibly work just fine, or BF> encounter issues. The signalling and cables for 6g and 12g are the BF> same, but with 12g using a higher clocking rate. BF> Bob -- Best regards, Frank mailto:info.fmo at gmx.de --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft. https://www.avast.com/antivirus -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: messages-test-san.txt URL: From danmcd at omniti.com Tue Sep 6 14:53:27 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 6 Sep 2016 10:53:27 -0400 Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon E3-1240 v3 In-Reply-To: <564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com> References: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com> <564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com> Message-ID: <4DD6D14B-495C-401A-9525-8AF41230DF98@omniti.com> > On Sep 5, 2016, at 4:13 AM, Hanno Hirschberger wrote: > > > I checked the file and indeed there's no such entry. I mounted a few older boot environments. There's no 'kvm' entry in any of the /etc/name_to_major files. Do you have any idea how to tackle this? Try re-running devfsadm: # devfsadm # grep kvm /etc/name_to_major Also, perhaps "pkg verify kvm" may be useful. Dan From vizoster at gmail.com Tue Sep 6 16:56:37 2016 From: vizoster at gmail.com (Vizo Allman) Date: Tue, 6 Sep 2016 12:56:37 -0400 Subject: [OmniOS-discuss] new to omnios and zfs... Message-ID: question: very new to zfs but looking at it to hold my VMs for kvm server over NFS I have some samsung 1tb pro ssd drives can I use those ? I am having a little trouble parsing the info I am seeing online. For the server I was planning on using a Supermicro with a 12 core cpu and 32 gigs of RAM it has a LSI-9211 in IT mode. I eventually want to use this setup type in prodution for some small networks. Any advise is appreciated. -- "Nor aught availed him now To have build in Heaven high towers Nor did he scape By all his engines But was headlong sent with his industrious crew To build in Hell "Milton, "Paradise Lost" -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Tue Sep 6 17:31:45 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Tue, 6 Sep 2016 12:31:45 -0500 (CDT) Subject: [OmniOS-discuss] new to omnios and zfs... In-Reply-To: References: Message-ID: On Tue, 6 Sep 2016, Vizo Allman wrote: > question: very new to zfs but looking at it to hold my VMs for kvm server > over NFS > I have some samsung 1tb pro ssd drives > can I use those ? I am having a little trouble parsing the info I am > seeing online. What would be the purpose of these Samsung drives (e.g. synchronous write intent log, L2 ARC, main store) in your server? For use as a synchronous intent log you should look for SSDs which support a super-cap to assure that all confirmed data is commited to underlying FLASH if the power fails. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From vizoster at gmail.com Tue Sep 6 17:44:47 2016 From: vizoster at gmail.com (Vizo Allman) Date: Tue, 6 Sep 2016 13:44:47 -0400 Subject: [OmniOS-discuss] new to omnios and zfs... In-Reply-To: References: Message-ID: It would be the main store. I have 5 of these drives I will use them to store my VM using linux kvm over NFS share to hold my mysql and ms sql VM with DB. Small for now but I would like a similar setup for a small production network that has the same services. Hardware: supermicro mbd-h8sgl-f with a 12 core AMD opteron 32 Gigs ECC memory (can add more) LSI-9211 in IT mode. Two Intel? 82574L controllers Dual-Port Gigabit Ethernet On Tue, Sep 6, 2016 at 1:31 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Tue, 6 Sep 2016, Vizo Allman wrote: > > question: very new to zfs but looking at it to hold my VMs for kvm server >> over NFS >> I have some samsung 1tb pro ssd drives >> can I use those ? I am having a little trouble parsing the info I am >> seeing online. >> > > What would be the purpose of these Samsung drives (e.g. synchronous write > intent log, L2 ARC, main store) in your server? > > For use as a synchronous intent log you should look for SSDs which support > a super-cap to assure that all confirmed data is commited to underlying > FLASH if the power fails. > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > -- "Nor aught availed him now To have build in Heaven high towers Nor did he scape By all his engines But was headlong sent with his industrious crew To build in Hell "Milton, "Paradise Lost" -------------- next part -------------- An HTML attachment was scrubbed... URL: From nagele at wildbit.com Tue Sep 6 17:50:14 2016 From: nagele at wildbit.com (Chris Nagele) Date: Tue, 6 Sep 2016 13:50:14 -0400 Subject: [OmniOS-discuss] new to omnios and zfs... In-Reply-To: References: Message-ID: What kind of chassis are you looking at? I've tried the all SSD route a few times, but you really want to use SAS drives and the cost is not worth it. We ended up going with HGST (0B31069) for the log and cache, and then larger 4TB or 6TB drives using 3 way mirrors for data. Chris From vizoster at gmail.com Tue Sep 6 17:56:08 2016 From: vizoster at gmail.com (Vizo Allman) Date: Tue, 6 Sep 2016 13:56:08 -0400 Subject: [OmniOS-discuss] new to omnios and zfs... In-Reply-To: References: Message-ID: https://www.supermicro.com/Aplus/system/1U/1012/AS-1012G-MTF.cfm I added the lsi-9211. I actually have a few like this on hand On Tue, Sep 6, 2016 at 1:50 PM, Chris Nagele wrote: > What kind of chassis are you looking at? I've tried the all SSD route > a few times, but you really want to use SAS drives and the cost is not > worth it. We ended up going with HGST (0B31069) for the log and cache, > and then larger 4TB or 6TB drives using 3 way mirrors for data. > > Chris > -- "Nor aught availed him now To have build in Heaven high towers Nor did he scape By all his engines But was headlong sent with his industrious crew To build in Hell "Milton, "Paradise Lost" -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertfantini at gmail.com Tue Sep 6 21:08:56 2016 From: robertfantini at gmail.com (Robert Fantini) Date: Tue, 6 Sep 2016 17:08:56 -0400 Subject: [OmniOS-discuss] how to find which disk is ' target 1 lun0' Message-ID: Hello Our system froze , console had time out errors on target 1 lun 0 . I've check around on the system and on napp-it menus but could not find for certain which disk is target 1 lun 0 . Can someone tell me a way to do so? best regards, Robert Fantini -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Wed Sep 7 11:15:28 2016 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 7 Sep 2016 21:15:28 +1000 Subject: [OmniOS-discuss] PHP on LTS Message-ID: <20160907211528.35bd852c@kendall> OK, my secret shame: I want to install PHP on r151014. How ? Latest version available. ______________ Michael Mounteney From jimklimov at cos.ru Wed Sep 7 12:15:02 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Wed, 07 Sep 2016 14:15:02 +0200 Subject: [OmniOS-discuss] how to find which disk is ' target 1 lun0' In-Reply-To: References: Message-ID: <03BEDB1C-A762-496C-90D8-72E04A6C54D6@cos.ru> 6 ???????? 2016??. 23:08:56 CEST, Robert Fantini ?????: >Hello > >Our system froze , console had time out errors on target 1 lun 0 . > >I've check around on the system and on napp-it menus but could not find >for >certain which disk is target 1 lun 0 . > >Can someone tell me a way to do so? > > >best regards, > >Robert Fantini > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Possibly dmesg van glean some details (it sort of lists various na es the devices go by as it enumerates and attaches storage device nodes). Or 'zpool status' can say what's faulty. Good luck, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jdg117 at elvis.arl.psu.edu Thu Sep 8 15:17:13 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Thu, 08 Sep 2016 11:17:13 -0400 Subject: [OmniOS-discuss] PHP on LTS In-Reply-To: Your message of "Wed, 07 Sep 2016 21:15:28 +1000." <20160907211528.35bd852c@kendall> References: <20160907211528.35bd852c@kendall> Message-ID: <201609081517.u88FHDxt013817@elvis.arl.psu.edu> In message <20160907211528.35bd852c at kendall>, Michael Mounteney writes: >OK, my secret shame: I want to install PHP on r151014. How ? Latest >version available. On r151018, PHP-7 builds from source: # pkg install developer/gcc51 system/header gnu-make $ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/opt/gcc-5.1.0/bin:/opt/apache2/bin:/opt/postgres/bin \ CC=gcc CFLAGS="-m64 -std=gnu99" LDFLAGS=-m64 \ ./configure --prefix=/opt/apache2/php-7 \ --with-apxs2=/opt/apache2/bin/apxs \ --with-pgsql=/opt/postgres John groenveld at acm.org From pkam at bloom.pl Mon Sep 12 09:41:46 2016 From: pkam at bloom.pl ([BLOOM] Piotr Kaminski) Date: Mon, 12 Sep 2016 11:41:46 +0200 Subject: [OmniOS-discuss] otherwise stable system sudden crashing In-Reply-To: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local> References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local> Message-ID: <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl> Hi everybody I've just subscribed to the list, although I have been using OmniOS for a a few years. I would like go back to the issue raised by Floris van Essen. Some time ago I had the same issue. I upgraded Omnios from 151014 to 151016 and had the same issue - the host rebooted when accessed by Linux client. I reverted to 151014 because that is production file server. Rock solid and stable since then. Now, on completely different system - it is new install of OmniOS 151018 VM under ESXi 6, with passthrough LSI card, the system was solid while serving NFS shares. And then I turned on some CIFS shares, one of them is mounted on Ubuntu system. OmniOS has crashed twice in 2 hours! I am desperate now. It is a new important server and it cannot crash. In the whole internet I have found only one post from Floris with similar looking issue. Anyone able to help? With regards -- Piotr Kami?ski PGP KeyID 0x804df875a899f9ac From danmcd at omniti.com Mon Sep 12 14:42:32 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 12 Sep 2016 10:42:32 -0400 Subject: [OmniOS-discuss] otherwise stable system sudden crashing In-Reply-To: <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl> References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local> <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl> Message-ID: <325F9F2D-CE1A-4A4F-9B04-32846B05988B@omniti.com> > On Sep 12, 2016, at 5:41 AM, [BLOOM] Piotr Kaminski wrote: > > Now, on completely different system - it is new install of OmniOS 151018 > VM under ESXi 6, with passthrough LSI card, the system was solid while > serving NFS shares. And then I turned on some CIFS shares, one of them > is mounted on Ubuntu system. OmniOS has crashed twice in 2 hours! You need to share core files. "OmniOS crashes" is bad news, but without the corefiles, it's hard to know what is happening. Dan From skeltonr at btconnect.com Mon Sep 12 17:23:07 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Mon, 12 Sep 2016 18:23:07 +0100 Subject: [OmniOS-discuss] Zone network fails to work. Message-ID: <57D6E47B.4020504@btconnect.com> Hi, I have a Sun X4170 with igb network. I am using omnios-r151018-ae3141d Here are my steps to create a zone called fred :- dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE usbecm0 Ethernet down 10 full usbecm0 igb2 Ethernet unknown 0 half igb2 igb0 Ethernet up 1000 full igb0 igb3 Ethernet unknown 0 half igb3 igb1 Ethernet unknown 0 half igb1 dladm create-vnic -l igb0 fred zfs create -o mountpoint=/zones rpool/zones /usr/sbin/zonecfg -z fred create set zonepath=/zones/fred set autoboot=true set limitpriv=default,dtrace_proc,dtrace_user set ip-type=exclusive add net set physical=fred end verify commit exitz oneadm -z fred install zoneadm -z fred boot zlogin -e+ -C fred dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE VID fred ? 1000 2:8:20:f4:52:49 random 0 ipadm create-if fred ipadm create-addr -T static -a 192.168.240.214/24 fred/v4static route -p add default 192.168.240.254 ping 192.168.240.214 192.168.240.214 is alive ping 192.168.240.254 no answer from 192.168.240.254 ifconfig lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 fred: flags=1000843 mtu 1500 index 2 inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255 ether 2:8:20:f4:52:49 lo0: flags=2002000849 mtu 8252 index 1 inet6 ::1/128 fred: flags=20002000840 mtu 1500 index 2 inet6 ::/0 ether 2:8:20:f4:52:49 What am I missing :-( Cheers Richard From danmcd at omniti.com Mon Sep 12 17:36:30 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 12 Sep 2016 13:36:30 -0400 Subject: [OmniOS-discuss] Zone network fails to work. In-Reply-To: <57D6E47B.4020504@btconnect.com> References: <57D6E47B.4020504@btconnect.com> Message-ID: <0AFBE5AF-7824-45CF-B78F-442E6E21C57B@omniti.com> vNICS MUST end in a number. That's a requirement. "fred0" or "fred23" is fine, but "fred" breaks things. Dan From danmcd at omniti.com Mon Sep 12 19:05:14 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 12 Sep 2016 15:05:14 -0400 Subject: [OmniOS-discuss] otherwise stable system sudden crashing In-Reply-To: References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local> <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl> <325F9F2D-CE1A-4A4F-9B04-32846B05988B@omniti.com> Message-ID: <63B18CF9-C802-4A30-8B26-B1DE7A155B31@omniti.com> I took a look at your dumps. Both vmdump.0 and vmdump1 have the same stack: vpanic() 0xfffffffffba90de8() zfs_acl_ids_create+0x4d2(ffffff0799931ab0, 2, ffffff00194bdcb0, ffffff055d97ddb0 , 0, ffffff00194bdbf0) zfs_make_xattrdir+0x96(ffffff0799931ab0, ffffff00194bdcb0, ffffff00194be018, ffffff055d97ddb0) zfs_get_xattrdir+0x103(ffffff0799931ab0, ffffff00194be018, ffffff055d97ddb0, e) zfs_lookup+0x1b6(ffffff056969d740, fffffffffbb78cd0, ffffff00194be018, ffffff00194bdf30, e, ffffff055db38740) fop_lookup+0xa2(ffffff056969d740, fffffffffbb78cd0, ffffff00194be018, ffffff00194bdf30, e, ffffff055db38740) xattr_dir_realdir+0xb3(ffffff079edb1880, ffffff00194be018, 6, ffffff055d97ddb0, 0) xattr_lookup_cb+0x65(ffffff079edb1880, ffffff05da046a00, ffffff00194be1b0, ffffff00194be0b8, ffffff055d97ddb0, 80000) gfs_dir_lookup_dynamic+0x7c(fffffffffbb5d3d0, ffffff0563cea1f8, ffffff05da046a00 , ffffff079edb1880, ffffff00194be1b0, ffffff055d97ddb0) gfs_dir_lookup+0x18c(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, ffffff055d97ddb0, 80000, ffffff00194be55c) gfs_vop_lookup+0x35(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, 0, 80000, 0) fop_lookup+0xa2(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, 0, 80000, 0) Do you know exactly what you did to reproduce this panic? After r151018 was released, there were at least FOUR changes near the code that's panicking on you: 6762 POSIX write should imply DELETE_CHILD on directories - and some additional considerations 6763 aclinherit=restricted masks inherited permissions by group perms (groupmask) 6764 zfs issues with inheritance flags during chmod(2) with aclmode=passthrough 6765 zfs_zaccess_delete() comments do not accurately reflect delete permissions for ACLs Unfortunately, none of those bugs mention a kernel panic. Are you willing to try OmniOS bloody (with DEBUG enabled) on a be on your system? Or at least are you willing to run a DEBUG kernel on your r151018 environment? Either one of these would be most helpful. Thanks, Dan From skeltonr at btconnect.com Mon Sep 12 20:35:33 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Mon, 12 Sep 2016 21:35:33 +0100 Subject: [OmniOS-discuss] Zone network fails to work. In-Reply-To: <57D6E47B.4020504@btconnect.com> References: <57D6E47B.4020504@btconnect.com> Message-ID: <57D71195.7000601@btconnect.com> Sorry in my haste I posted fred not fred5 See corrections below :-) Richard Skelton wrote: > Hi, > I have a Sun X4170 with igb network. > I am using omnios-r151018-ae3141d > Here are my steps to create a zone called fred5 :- > dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > usbecm0 Ethernet down 10 full usbecm0 > igb2 Ethernet unknown 0 half igb2 > igb0 Ethernet up 1000 full igb0 > igb3 Ethernet unknown 0 half igb3 > igb1 Ethernet unknown 0 half igb1 > > > dladm create-vnic -l igb0 fred5 > zfs create -o mountpoint=/zones rpool/zones > > /usr/sbin/zonecfg -z fred5 > > create > set zonepath=/zones/fred5 > set autoboot=true > set limitpriv=default,dtrace_proc,dtrace_user > set ip-type=exclusive > add net > set physical=fred5 > end > verify > commit > exitz > oneadm -z fred5 install > zoneadm -z fred5 boot > zlogin -e+ -C fred5 > > dladm show-vnic > LINK OVER SPEED MACADDRESS MACADDRTYPE VID > fred5 ? 1000 2:8:20:f4:52:49 random 0 > > ipadm create-if fred5 > > ipadm create-addr -T static -a 192.168.240.214/24 fred5/v4static > route -p add default 192.168.240.254 > > ping 192.168.240.214 > 192.168.240.214 is alive > ping 192.168.240.254 > no answer from 192.168.240.254 > > > ifconfig > lo0: flags=2001000849 mtu > 8232 index 1 > inet 127.0.0.1 netmask ff000000 > fred5: flags=1000843 mtu 1500 index 2 > inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255 > ether 2:8:20:f4:52:49 > lo0: flags=2002000849 mtu > 8252 index 1 > inet6 ::1/128 > fred5: flags=20002000840 mtu 1500 index 2 > inet6 ::/0 > ether 2:8:20:f4:52:49 > > > What am I missing :-( > > Cheers > Richard > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From danmcd at omniti.com Wed Sep 14 15:52:50 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 14 Sep 2016 11:52:50 -0400 Subject: [OmniOS-discuss] CVE-2016-7167 in curl now fixed Message-ID: <881257C1-9D69-42BD-99EC-4D65DDBEBB5F@omniti.com> All supported repos (r15101[468], bloody) now have updated cURL packages with a patch for the aforementioned CVE. Please update at your convenience. A reboot is not required, but some services may need to be restarted that don't have proper SMF restart dependencies in them. Dan From an3s.annema at gmail.com Thu Sep 15 18:15:05 2016 From: an3s.annema at gmail.com (Andries Annema) Date: Thu, 15 Sep 2016 20:15:05 +0200 Subject: [OmniOS-discuss] Zone network fails to work. In-Reply-To: <57D71195.7000601@btconnect.com> References: <57D6E47B.4020504@btconnect.com> <57D71195.7000601@btconnect.com> Message-ID: <74892e16-b91d-83f3-a000-cf422ab815b7@gmail.com> Hi Richard, About two years ago I bumped into the same, or at least a similar, problem on my brand new r151014 rig: the system was able to ping hosts on the same network segment, but a ping to my router/gateway resulted in "no answer from / Unknown host". In my notes from back then I find that I ended up in verifying the arp cache by entering: # arp -a The resulting list did not show the router, so I added it manually: # arp -s A page that might shed some more light on the matter: https://www.petri.com/csc_arp_cache I have no clue as to why a single host on the local network would not show up in that arp table while others would, but back then the above solved my problem. Haven't had the issue since then, so it might be worth giving it a try. If it doesn't help you, then I hope someone who really knows the ins and outs of networking is listening in. :-) Regards, Andries On 2016-09-12 22:35, Richard Skelton wrote: > Sorry in my haste I posted fred not fred5 > See corrections below :-) > > Richard Skelton wrote: >> Hi, >> I have a Sun X4170 with igb network. >> I am using omnios-r151018-ae3141d >> Here are my steps to create a zone called fred5 :- >> dladm show-phys >> LINK MEDIA STATE SPEED DUPLEX DEVICE >> usbecm0 Ethernet down 10 full usbecm0 >> igb2 Ethernet unknown 0 half igb2 >> igb0 Ethernet up 1000 full igb0 >> igb3 Ethernet unknown 0 half igb3 >> igb1 Ethernet unknown 0 half igb1 >> >> >> dladm create-vnic -l igb0 fred5 >> zfs create -o mountpoint=/zones rpool/zones >> >> /usr/sbin/zonecfg -z fred5 >> >> create >> set zonepath=/zones/fred5 >> set autoboot=true >> set limitpriv=default,dtrace_proc,dtrace_user >> set ip-type=exclusive >> add net >> set physical=fred5 >> end >> verify >> commit >> exitz >> oneadm -z fred5 install >> zoneadm -z fred5 boot >> zlogin -e+ -C fred5 >> >> dladm show-vnic >> LINK OVER SPEED MACADDRESS MACADDRTYPE VID >> fred5 ? 1000 2:8:20:f4:52:49 random 0 >> >> ipadm create-if fred5 >> >> ipadm create-addr -T static -a 192.168.240.214/24 fred5/v4static >> route -p add default 192.168.240.254 >> >> ping 192.168.240.214 >> 192.168.240.214 is alive >> ping 192.168.240.254 >> no answer from 192.168.240.254 >> >> >> ifconfig >> lo0: flags=2001000849 mtu >> 8232 index 1 >> inet 127.0.0.1 netmask ff000000 >> fred5: flags=1000843 mtu 1500 index 2 >> inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255 >> ether 2:8:20:f4:52:49 >> lo0: flags=2002000849 mtu >> 8252 index 1 >> inet6 ::1/128 >> fred5: flags=20002000840 mtu 1500 index 2 >> inet6 ::/0 >> ether 2:8:20:f4:52:49 >> >> >> What am I missing :-( >> >> Cheers >> Richard >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From ergi.thanasko at avsquad.com Fri Sep 16 17:43:30 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Fri, 16 Sep 2016 17:43:30 +0000 Subject: [OmniOS-discuss] Network throughout 1GB/sec Message-ID: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Hi all, We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. Thank you for your time From bfriesen at simple.dallas.tx.us Fri Sep 16 19:33:58 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 16 Sep 2016 14:33:58 -0500 (CDT) Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: On Fri, 16 Sep 2016, Ergi Thanasko wrote: > Given the hardware that we have and the zpool performance, we > expected to see some serious data transfer rates however we only see > around 200-300MB/sec average using rsync or copy paste over NFS. > Standard MTU 1500 and nfs block size. I want to ask the community Are these read rates or write rates? Read rates should be able to come close to pool or wire limits. Write rates over NFS are primarily dominated by latency on a per-writer basis. Increasing the MTU to 9k has been shown to improve throughput quite a lot for large transfers. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From ergi.thanasko at avsquad.com Fri Sep 16 19:38:32 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Fri, 16 Sep 2016 19:38:32 +0000 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com> Hi Bob, We were testing it between two similar servers rsynic and copy paste both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide higher throughput around 600MB/sec [cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home] On Sep 16, 2016, at 12:33 PM, Bob Friesenhahn > wrote: On Fri, 16 Sep 2016, Ergi Thanasko wrote: Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community Are these read rates or write rates? Read rates should be able to come close to pool or wire limits. Write rates over NFS are primarily dominated by latency on a per-writer basis. Increasing the MTU to 9k has been shown to improve throughput quite a lot for large transfers. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ErgiThanasko.png Type: image/png Size: 7910 bytes Desc: ErgiThanasko.png URL: From mtalbott at lji.org Fri Sep 16 19:53:42 2016 From: mtalbott at lji.org (Michael Talbott) Date: Fri, 16 Sep 2016 12:53:42 -0700 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com> Message-ID: Jumbo frames are a major help. Also, try using multiple streams (break single rsync job into multiple jobs). Also be sure to use rsync native protocol and don't tunnel it over ssh. Then there's bbcp which can push a single copy operation into multiple streams to fully saturate your disks/network ;) https://www.slac.stanford.edu/~abh/bbcp/ Michael > On Sep 16, 2016, at 12:38 PM, Ergi Thanasko wrote: > > Hi Bob, > We were testing it between two similar servers rsynic and copy paste both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide higher throughput around 600MB/sec > > >> On Sep 16, 2016, at 12:33 PM, Bob Friesenhahn > wrote: >> >> On Fri, 16 Sep 2016, Ergi Thanasko wrote: >> >>> Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community >> >> Are these read rates or write rates? Read rates should be able to come close to pool or wire limits. Write rates over NFS are primarily dominated by latency on a per-writer basis. >> >> Increasing the MTU to 9k has been shown to improve throughput quite a lot for large transfers. >> >> Bob >> -- >> Bob Friesenhahn >> bfriesen at simple.dallas.tx.us , http://www.simplesystems.org/users/bfriesen/ >> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Fri Sep 16 19:57:53 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Fri, 16 Sep 2016 21:57:53 +0200 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex max_buf=4097152 tcp send_buf=2048576 tcp recv_buf=2048576 tcp, NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576) http://napp-it.org/doc/downloads/performance_smb2.pdf Gea Am 16.09.2016 um 19:43 schrieb Ergi Thanasko: > Hi all, > We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. > Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. > > If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? > Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. > > Thank you for your time > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From bfriesen at simple.dallas.tx.us Fri Sep 16 20:23:46 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 16 Sep 2016 15:23:46 -0500 (CDT) Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com> Message-ID: On Fri, 16 Sep 2016, Ergi Thanasko wrote: > We were testing it between two similar servers rsynic and copy paste > both ways ( read/write) was the same around 300MB/sec average. Of > course the speed test on the pools provide higher throughput around > 600MB/sec I am not sure what 'copy paste' means, but rsync can be used in a couple of different ways. One way is that rsync is writing over NFS to the server. Another way is that rsync communicates via some means (ssh, netcat) to another rsync running on the server. Rsync must always query/check to see if/what data is existing while it does the copy. It is useful to see that if you do two copies at the same time if the combined rate is much more than 300MB/sec average. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From davide.poletto at gmail.com Fri Sep 16 22:40:04 2016 From: davide.poletto at gmail.com (Davide Poletto) Date: Sat, 17 Sep 2016 00:40:04 +0200 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: Hope not be wrong here but Port Trunking usage deserves its part in the whole picture too: be aware that using Port Trunking (with LACP as per IEEE 802.3ad) between your Servers' NIC and your 10Gb Switching infrastructure - and this happens by aggregating "n" identical ports together on both link's ends, as you wrote - doesn't consequently mean that your "one-Host-to-one-Host" generated traffic will be able to use and will be able to saturate all those "n" 10Gb based links concurrently... On Sep 16, 2016 7:45 PM, "Ergi Thanasko" wrote: > Hi all, > We have a a few servers conected via 10g nic LACP, some of them have > 4nic and some have 6nic in a link aggregation mode. We been moving a lot of > data around and we are trying to get the maximum performance. I have seen > zpool can deliver 2-3GB accumulated throughput. Iperf does about > 600-800MB/sec between those two servers. > Given the hardware that we have and the zpool performance, we expected > to see some serious data transfer rates however we only see around > 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU > 1500 and nfs block size. I want to ask the community what to do get some > higher throughout and the application level. I hear ZFS send/receive or > ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is > constantly evolving and we prefer something in the nature of rsync but > to utilize the network hardware. > > If Anyone has a hardware setup that can see 1GB/sec throughput and does > not mind sharing? > Any software that use multithreads sessions to move data around zfs > friendly? We do not mind getting going with a commercial solution like > camvault or veeam if they work. > > Thank you for your time > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daleg at omniti.com Sat Sep 17 00:24:39 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 16 Sep 2016 20:24:39 -0400 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com> Are you doing the rsync over ssh? You might want to look into using HPN-SSH: https://www.psc.edu/index.php/hpn-ssh /dale > On Sep 16, 2016, at 1:43 PM, Ergi Thanasko wrote: > > Hi all, > We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. > Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. > > If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? > Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. > > Thank you for your time > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From ergi.thanasko at avsquad.com Sat Sep 17 00:37:12 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Sat, 17 Sep 2016 00:37:12 +0000 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com> Message-ID: <580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com> no that is even slower, just rsync over mounted nfs, a multithreaded rsync does work have better performance. [cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home] On Sep 16, 2016, at 5:24 PM, Dale Ghent > wrote: Are you doing the rsync over ssh? You might want to look into using HPN-SSH: https://www.psc.edu/index.php/hpn-ssh /dale On Sep 16, 2016, at 1:43 PM, Ergi Thanasko wrote: Hi all, We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. Thank you for your time _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ErgiThanasko.png Type: image/png Size: 7910 bytes Desc: ErgiThanasko.png URL: From ergi.thanasko at avsquad.com Sat Sep 17 00:57:44 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Sat, 17 Sep 2016 00:57:44 +0000 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> Message-ID: <2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com> HI Gea, Great info, are you seeing 1000MB/s doing iperf or actually transfer rates rsync, cp bbcp?. > On Sep 16, 2016, at 12:57 PM, Guenther Alka wrote: > > I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex > max_buf=4097152 tcp > send_buf=2048576 tcp > recv_buf=2048576 tcp, > > NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576) > > http://napp-it.org/doc/downloads/performance_smb2.pdf > > > Gea > > Am 16.09.2016 um 19:43 schrieb Ergi Thanasko: >> Hi all, >> We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. >> Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. >> >> If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? >> Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. >> >> Thank you for your time >> >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From stephan.budach at jvm.de Sat Sep 17 05:47:18 2016 From: stephan.budach at jvm.de (Stephan Budach) Date: Sat, 17 Sep 2016 07:47:18 +0200 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> Message-ID: Am 17.09.16 um 00:40 schrieb Davide Poletto: > > Hope not be wrong here but Port Trunking usage deserves its part in > the whole picture too: be aware that using Port Trunking (with LACP as > per IEEE 802.3ad) between your Servers' NIC and your 10Gb Switching > infrastructure - and this happens by aggregating "n" identical ports > together on both link's ends, as you wrote - doesn't consequently mean > that your "one-Host-to-one-Host" generated traffic will be able to use > and will be able to saturate all those "n" 10Gb based links > concurrently... > > +1 It's a common misunderstanding that bundling a couple of ports together with LACP results in bigger throughput, as one connection will always only deliver it's maximum physical bandwidth. There are bonding modes, which support that, but you'd have to be able to configure this end to end, with some round-robin settings, as of on wich NIC the next packet will be stuffed into the "pipe". Regular 802.3ad will not give you that, as it has been designed for redundancy, not performance. So, LACP gives you mutiple read streams up to the number of NICs involved in the bundle, but writes from the host will always be on one NIC for any given connection. The classic algorithm is based on the source and target MAC addresses. So you will not be able to get 2 to 3 GB/s over a 10GbE LACP bundle. Cheers, Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Sat Sep 17 08:50:54 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Sat, 17 Sep 2016 10:50:54 +0200 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> <2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com> Message-ID: <7e30147c-be84-5b6c-272e-56431b892f48@hfg-gmuend.de> hi Intention of this test and tuning cycle was to check 4k video editing capability from OSX and Windows to a Solaris or OmniOS storage over 10G/40G to SSD or NVMe storage. With tunings and large 4k videofiles I was able to get about 900MB/s on write with peaks up to 1000 MB/s over SMB 2.1 tested with the video editing tool AJA on Windows and speed test on OSX. These are more or less sequential tests with a lot of large files. Values on Solaris were slightly better than on OmniOS so I asume OS or driver defaults are more optimized there at least regarding 10G/40G. Reads were always slower and more critical to settings and cablings. NFS values on OSX were not nearly as good and quite disappointing (at least on OSX). With smaller NTSC/PAL video settings (test uses many small files then) performance went down to 500-600 MB/s with writes and a little lower on reads. I currently do some tests with i40e on Intel XL710 where the difference is heavy with 2200MB write on Solaris and 1500 MB/s on OmniOS with same settings while reads are currently a disaster at least on Windows with up to 300 MB/s on Solaris and 150MB/s on OmniOS. In best cases these values were nearly as good as iperf values so near wire speed. I would not asume that you can reach high performance values with rsync. Zfs send should be faster as it creates a filestream. I woukd not expect that they can come close to the above even with zfs send over mbuffer or netcat. A pure copy should be as fast from/to OSX orWindows or with cp over netcat. Gea Am 17.09.2016 um 02:57 schrieb Ergi Thanasko: > HI Gea, > Great info, are you seeing 1000MB/s doing iperf or actually transfer rates rsync, cp bbcp?. > > >> On Sep 16, 2016, at 12:57 PM, Guenther Alka wrote: >> >> I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex >> max_buf=4097152 tcp >> send_buf=2048576 tcp >> recv_buf=2048576 tcp, >> >> NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576) >> >> http://napp-it.org/doc/downloads/performance_smb2.pdf >> >> >> Gea >> >> Am 16.09.2016 um 19:43 schrieb Ergi Thanasko: >>> Hi all, >>> We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. >>> Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. >>> >>> If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? >>> Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. >>> >>> Thank you for your time >>> >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Sat Sep 17 21:03:44 2016 From: mir at miras.org (Michael Rasmussen) Date: Sat, 17 Sep 2016 23:03:44 +0200 Subject: [OmniOS-discuss] kernel panic Message-ID: <20160917230344.493adaee@sleipner.datanom.net> Hi all, On my Omnios r151014 (omnios-b13298f) I had a sudden kernel panic this evening with the following display in /var/adm/messages: Sep 17 21:35:29 nas unix: [ID 836849 kern.notice] Sep 17 21:35:29 nas ^Mpanic[cpu2]/thread=ffffff071b9a8400: Sep 17 21:35:29 nas genunix: [ID 683410 kern.notice] BAD TRAP: type=d (#gp General protection) rp=ffffff00206eb700 addr=ffffff06e6fd8d00 Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] Sep 17 21:35:29 nas unix: [ID 839527 kern.notice] perl: Sep 17 21:35:29 nas unix: [ID 753105 kern.notice] #gp General protection Sep 17 21:35:29 nas unix: [ID 358286 kern.notice] addr=0xffffff06e6fd8d00 Sep 17 21:35:29 nas unix: [ID 243837 kern.notice] pid=18522, pc=0xfffffffffbb4e61f, sp=0xffffff00206eb7f0, eflags=0x10282 Sep 17 21:35:29 nas unix: [ID 211416 kern.notice] cr0: 80050033 cr4: 6f8 Sep 17 21:35:29 nas unix: [ID 624947 kern.notice] cr2: 817824c Sep 17 21:35:29 nas unix: [ID 625075 kern.notice] cr3: 3d63d5000 Sep 17 21:35:29 nas unix: [ID 625715 kern.notice] cr8: c Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] rdi: ffffff06e6fd8d00 rsi: 1fffffe0ed7a1000 rdx: 1000 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] rcx: ffffff00206eba90 r8: ffffff00206eb9f0 r9: 1000 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] rax: ffffff071db23ad8 rbx: ffffff06e6fd8d00 rbp: ffffff00206eb8a0 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] r10: baddcafebaddcafe r11: 0 r12: ffffff00206eb9f0 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] r13: ffffff00206eba90 r14: 1fffffe0ed7a1000 r15: ffffff06e6fd8d00 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] fsb: 0 gsb: ffffff06e6106500 ds: 4b Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] es: 4b fs: 0 gs: 1c3 Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] trp: d err: 0 rip: fffffffffbb4e61f Sep 17 21:35:29 nas unix: [ID 592667 kern.notice] cs: 30 rfl: 10282 rsp: ffffff00206eb7f0 Sep 17 21:35:29 nas unix: [ID 266532 kern.notice] ss: 0 Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb5e0 unix:real_mode_stop_cpu_stage2_end+9e13 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb6f0 unix:trap+a30 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb700 unix:cmntrap+e6 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb8a0 genunix:fop_getpage+3f () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb960 genunix:anon_getpage+18d () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eba80 genunix:segvn_faultpage+168 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebc50 genunix:segvn_fault+8e6 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebd60 genunix:as_fault+31a () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebdf0 unix:pagefault+96 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebf00 unix:trap+2c7 () Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebf10 unix:cmntrap+e6 () Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] Sep 17 21:35:29 nas genunix: [ID 672855 kern.notice] syncing file systems... Sep 17 21:35:30 nas genunix: [ID 904073 kern.notice] done Sep 17 21:35:31 nas genunix: [ID 111219 kern.notice] dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel Sep 17 21:35:31 nas genunix: [ID 405573 kern.info] NOTICE: ahci1: ahci_tran_reset_dport port 0 reset port Sep 17 21:35:31 nas genunix: [ID 405573 kern.info] NOTICE: ahci1: ahci_tran_reset_dport port 1 reset port Anybody knows what is going on here? NB. I have the kernel dump file in case anybody should be interested. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: "Life and death are seldom logical." "But attaining a desired goal always is." -- McCoy and Spock, "The Galileo Seven", stardate 2821.7 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Sat Sep 17 22:08:10 2016 From: mir at miras.org (Michael Rasmussen) Date: Sun, 18 Sep 2016 00:08:10 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <20160917230344.493adaee@sleipner.datanom.net> References: <20160917230344.493adaee@sleipner.datanom.net> Message-ID: <20160918000810.49ca7650@sleipner.datanom.net> On Sat, 17 Sep 2016 23:03:44 +0200 Michael Rasmussen wrote: > Hi all, > > On my Omnios r151014 (omnios-b13298f) I had a sudden kernel panic this > evening with the following display in /var/adm/messages: > And after reboot: Sep 17 21:51:07 nas savecore: [ID 833026 auth.error] Decompress the crash dump w ith Sep 17 21:51:07 nas 'savecore -vf /var/crash/unknown/vmdump.2' Sep 17 21:51:08 nas fmd: [ID 377184 daemon.error] SUNW-MSG-ID: SUNOS-8000-KL, TY PE: Defect, VER: 1, SEVERITY: Major Sep 17 21:51:08 nas EVENT-TIME: Sat Sep 17 21:51:07 CEST 2016 Sep 17 21:51:08 nas PLATFORM: To-Be-Filled-By-O.E.M., CSN: -, HOSTNAME: nas Sep 17 21:51:08 nas SOURCE: software-diagnosis, REV: 0.1 Sep 17 21:51:08 nas EVENT-ID: 914a5680-21c2-6d8b-e476-e7be57cf40e9 Sep 17 21:51:08 nas DESC: The system has rebooted after a kernel panic. Refer t o http://illumos.org/msg/SUNOS-8000-KL for more information. Sep 17 21:51:08 nas AUTO-RESPONSE: The failed system image was dumped to the dum p device. If savecore is enabled (see dumpadm(1M)) a copy of the dump will be w ritten to the savecore directory /var/crash/unknown. Sep 17 21:51:08 nas IMPACT: There may be some performance impact while the panic is copied to the savecore directory. Disk space usage by panics can be substan tial. Sep 17 21:51:08 nas REC-ACTION: If savecore is not enabled then please take step s to preserve the crash image. Sep 17 21:51:08 nas Use 'fmdump -Vp -u 914a5680-21c2-6d8b-e476-e7be57cf40e9' to view more panic detail. Please refer to the knowledge article for additional in formation. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: She was good at playing abstract confusion in the same way a midget is good at being short. -- Clive James, on Marilyn Monroe -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From jimklimov at cos.ru Sun Sep 18 06:38:59 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Sun, 18 Sep 2016 08:38:59 +0200 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com> <580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com> Message-ID: 17 ???????? 2016??. 2:37:12 CEST, Ergi Thanasko ?????: >no that is even slower, just rsync over mounted nfs, a multithreaded >rsync does work have better performance. > >[cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home] > >On Sep 16, 2016, at 5:24 PM, Dale Ghent >> wrote: > >Are you doing the rsync over ssh? You might want to look into using >HPN-SSH: > >https://www.psc.edu/index.php/hpn-ssh > >/dale > >On Sep 16, 2016, at 1:43 PM, Ergi Thanasko >wrote: > >Hi all, >We have a a few servers conected via 10g nic LACP, some of them have >4nic and some have 6nic in a link aggregation mode. We been moving a >lot of data around and we are trying to get the maximum performance. I >have seen zpool can deliver 2-3GB accumulated throughput. Iperf does >about 600-800MB/sec between those two servers. >Given the hardware that we have and the zpool performance, we >expected to see some serious data transfer rates however we only see >around 200-300MB/sec average using rsync or copy paste over NFS. >Standard MTU 1500 and nfs block size. I want to ask the community what >to do get some higher throughout and the application level. I hear ZFS >send/receive or ZFS shadow does work faster but it does snapshots. >Out data (Terabytes) is constantly evolving and we prefer something >in the nature of rsync but to utilize the network hardware. > >If Anyone has a hardware setup that can see 1GB/sec throughput and >does not mind sharing? >Any software that use multithreads sessions to move data around zfs >friendly? We do not mind getting going with a commercial solution like >camvault or veeam if they work. > >Thank you for your time > > > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Note that with rsync over nfs (or any seemingly local FS) your sender spawns a recipient process which must also read the existing data to see what differences rsync should transfer. So over your links you read the entirety of data from remote host, and send something back. Unless your other side has a pathologically weak cpu (think hobby ARM board), you gotta be faster off with real networked rsync (native or rsh/ssh etc.), where sender and recipient run on respective hosts and process the bulk of data locally. Also NFS I/O by design defaults to synchronous writes, so forfeits write-caching (with ZFS you can have ZIL on SSD to mitigate that performance impact and keep data security guarantees); some other OSes pretend they are fast by ignoring the sync requirement, and then you have fun times after a crash, failover etc. of the NFS server side. Jim -- Typos courtesy of K-9 Mail on my Samsung Android From skeltonr at btconnect.com Sun Sep 18 08:55:24 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Sun, 18 Sep 2016 09:55:24 +0100 Subject: [OmniOS-discuss] I would like to create a zfs file system in my zone Message-ID: <57DE567C.6050100@btconnect.com> Hi, I would like to create a zfs file system in my zone and create snapshots like I would in a global zone. In the zone I see:- root at fred5:/root# zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 38.6G 93.2G 23.5K /rpool rpool/zones 2.94G 93.2G 19K /zones rpool/zones/fred5 1.85G 93.2G 19K /zones/fred5 rpool/zones/fred5/ROOT 1.85G 93.2G 19K legacy rpool/zones/fred5/ROOT/zbe 1.85G 93.2G 1.85G legacy root at fred5:/root# ls /zones/fred5 ls: cannot access '/zones/fred5': No such file or directory root at fred5:/root# zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT rpool 136G 30.3G 106G - 19% 22% 1.00x ONLINE - root at fred5:/root# Before I install any optional software I would like to create a zfs file system and mount it on /opt in the zone. How can I do this? Cheers Richard From peter.tribble at gmail.com Sun Sep 18 09:32:24 2016 From: peter.tribble at gmail.com (Peter Tribble) Date: Sun, 18 Sep 2016 10:32:24 +0100 Subject: [OmniOS-discuss] I would like to create a zfs file system in my zone In-Reply-To: <57DE567C.6050100@btconnect.com> References: <57DE567C.6050100@btconnect.com> Message-ID: On Sun, Sep 18, 2016 at 9:55 AM, Richard Skelton wrote: > Hi, > I would like to create a zfs file system in my zone and create snapshots > like I would in a global zone. > In the zone I see:- > root at fred5:/root# zfs list > NAME USED AVAIL REFER MOUNTPOINT > rpool 38.6G 93.2G 23.5K /rpool > rpool/zones 2.94G 93.2G 19K /zones > rpool/zones/fred5 1.85G 93.2G 19K /zones/fred5 > rpool/zones/fred5/ROOT 1.85G 93.2G 19K legacy > rpool/zones/fred5/ROOT/zbe 1.85G 93.2G 1.85G legacy > root at fred5:/root# ls /zones/fred5 > ls: cannot access '/zones/fred5': No such file or directory > So the way it works is this: the dataset rpool/zones/fred5/ROOT is delegated to the zone. Anything above that is associated with the global zone; that dataset or below is associated with the local zone. You get to see all the parents in the hierarchy, with their mountpoints in their own context, which is a bit confusing. > root at fred5:/root# zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > rpool 136G 30.3G 106G - 19% 22% 1.00x ONLINE - > root at fred5:/root# > > Before I install any optional software I would like to create a zfs file > system and mount it on /opt in the zone. > How can I do this? > In this case, it could be: zfs create -o mountpoint=/opt rpool/zones/fred5/ROOT/opt However, I wouldn't normally do this. If I was creating datasets for use by a zone, I would create a separate dataset such as rpool/fred5 in the global zone (with mountpoint set to none), use 'add dataset' in the zone configuration to delegate that dataset to the zone, and use that to create my own filesystems in the zone. The aim being to separate my own data from that managed by the operating system. (Although the same issue of conflicts arises with the use of /opt, which the OS might think it has control of. Creating /opt/local or /opt/company-name one level down might be a better choice.) -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Sun Sep 18 14:44:22 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Sun, 18 Sep 2016 09:44:22 -0500 (CDT) Subject: [OmniOS-discuss] I would like to create a zfs file system in my zone In-Reply-To: <57DE567C.6050100@btconnect.com> References: <57DE567C.6050100@btconnect.com> Message-ID: On Sun, 18 Sep 2016, Richard Skelton wrote: > > Before I install any optional software I would like to create a zfs file > system and mount it on /opt in the zone. > How can I do this? Don't attempt to do this because OmniOS delivers some essential package content to /opt (e.g. gcc run-time libraries) and you will encounter booting problems after an upgrade (I was slightly singed by that). Default OmniOS has /opt in the boot environment so if you move it out of the boot environment then the installation will be inconsistent. You should be free to mount on mountpoints within /opt. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From vab at bb-c.de Sun Sep 18 14:47:24 2016 From: vab at bb-c.de (Volker A. Brandt) Date: Sun, 18 Sep 2016 16:47:24 +0200 Subject: [OmniOS-discuss] I would like to create a zfs file system in my zone In-Reply-To: References: <57DE567C.6050100@btconnect.com> Message-ID: <22494.43260.50648.709397@shelob.bb-c.de> Peter Tribble writes: > On Sun, Sep 18, 2016 at 9:55 AM, Richard Skelton > wrote: > Before I install any optional software I would like to create a zfs file > system and mount it on /opt in the zone. > How can I do this? > > > In this case, it could be: > > zfs create -o mountpoint=/opt rpool/zones/fred5/ROOT/opt On Solaris 11, a "zfs create -o mountpoint=/opt rpool/opt" will also work. (Can't test on OmniOS ATM as my OmniOS system is down.) This has the advantage that the ZFS dataset will appear as "rpool/opt" in the zone, and as "//opt" in the global zone. > However, I wouldn't normally do this. If I was creating datasets for use by a > zone, I would create a separate dataset such as rpool/fred5 in the global zone > (with mountpoint set to none), use 'add dataset' in the zone configuration to > delegate that dataset to the zone, and use that to create my own filesystems > in the zone. The aim being to separate my own data from that managed by the > operating system. > > (Although the same issue of conflicts arises with the use of /opt, which the > OS might think it has control of. Creating /opt/local or /opt/company-name > one level down might be a better choice.) One thing to remember is that the directory /opt normally is part of the zone root, and thus part of the managed boot environments (BEs). If you create a dataset and mount it under /opt (or /opt/local), it will not be part of any BE. If you then create IPS packages that install things in /opt, you will run into inconsistencies in the package database when you switch back and forth between BEs. Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Oracle Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim, GERMANY Email: vab at bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgr??e: 46 Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt "When logic and proportion have fallen sloppy dead" From jdg117 at elvis.arl.psu.edu Sun Sep 18 22:40:53 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Sun, 18 Sep 2016 18:40:53 -0400 Subject: [OmniOS-discuss] CMake fail due to ncurses vs curses Message-ID: <201609182240.u8IMerGf015467@elvis.arl.psu.edu> Has anyone got past this conflict? $ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/usr/gnu/i386-pc-solaris2.11/bin CC=gcc CXX=g++ CFLAGS=-m64 CXXFLAGS=-m64 LDFLAGS=-m64 ./bootstrap --system-libs --prefix=/opt/cmake --no-system-libarchive --no-system-jsoncpp --parallel=8 [ 87%] Linking CXX executable ../bin/ccmake Undefined first referenced symbol in file initscr32 CMakeFiles/ccmake.dir/CursesDialog/ccmake.cxx.o (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1) w32attron CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1) w32attroff CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1) ld: fatal: symbol referencing errors. No output written to ../bin/ccmake collect2: error: ld returned 1 exit status Source/CMakeFiles/ccmake.dir/build.make:419: recipe for target 'bin/ccmake' failed gmake[2]: *** [bin/ccmake] Error 1 CMakeFiles/Makefile2:1696: recipe for target 'Source/CMakeFiles/ccmake.dir/all' failed gmake[1]: *** [Source/CMakeFiles/ccmake.dir/all] Error 2 Makefile:160: recipe for target 'all' failed gmake: *** [all] Error 2 I can add -lcurses to Source/CMakeFiles/ccmake.dir/link.txt, below, but that seems like a Bad Idea[tm]. nm(1) suggests that initscr32 and friends are defined in libncurses.so How do I chase down the reference to libcurses functions? /usr/bin/g++ -m64 -m64 CMakeFiles/ccmake.dir/CursesDialog/cmCursesOptionsWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesBoolWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesCacheEntryComposite.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesDummyWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesFilePathWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesLabelWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesMainForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesPathWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesStringWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/ccmake.cxx.o -o ../bin/ccmake -L/usr/lib/64 libCMakeLib.a /usr/lib/64/libform.so /usr/lib/64/libncurses.so kwsys/libcmsys.a -ldl /usr/lib/64/libexpat.so ../Utilities/cmlibarchive/libarchive/! libcmlibarchive.a -lz /usr/lib/64/liblzma.so -lmd -lbz2 ../Utilities/cmcompress/libcmcompress.a /usr/lib/64/libcurl.so ../Utilities/cmjsoncpp/libcmjsoncpp.a -Wl,-R/usr/lib/64 TIA, hope to post a build recipe for GitLab CE this week. John groenveld at acm.org From lotheac at iki.fi Mon Sep 19 07:52:33 2016 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Mon, 19 Sep 2016 10:52:33 +0300 Subject: [OmniOS-discuss] CMake fail due to ncurses vs curses In-Reply-To: <201609182240.u8IMerGf015467@elvis.arl.psu.edu> References: <201609182240.u8IMerGf015467@elvis.arl.psu.edu> Message-ID: <20160919075232.GA11664@kekkonen.niksula.hut.fi> On Sun, Sep 18 2016 18:40:53 -0400, John D Groenveld wrote: > Has anyone got past this conflict? > > $ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/usr/gnu/i386-pc-solaris2.11/bin CC=gcc CXX=g++ CFLAGS=-m64 CXXFLAGS=-m64 LDFLAGS=-m64 ./bootstrap --system-libs --prefix=/opt/cmake --no-system-libarchive --no-system-jsoncpp --parallel=8 > > [ 87%] Linking CXX executable ../bin/ccmake I just built it without the curses interface. See https://github.com/niksula/omnios-build-scripts/blob/master/cmake/build.sh#L43 -- Lauri Tirkkonen | lotheac @ IRCnet From danmcd at omniti.com Mon Sep 19 15:35:33 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 19 Sep 2016 11:35:33 -0400 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <20160918000810.49ca7650@sleipner.datanom.net> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> Message-ID: > On Sep 17, 2016, at 6:08 PM, Michael Rasmussen wrote: > > And after reboot: > Sep 17 21:51:07 nas savecore: [ID 833026 auth.error] Decompress the crash dump w > ith > Sep 17 21:51:07 nas 'savecore -vf /var/crash/unknown/vmdump.2' > Sep 17 21:51:08 nas fmd: [ID 377184 daemon.error] SUNW-MSG-ID: SUNOS-8000-KL, TY > PE: Defect, VER: 1, SEVERITY: Major FMA always gets a fault report after a kernel panic. I see you have a vmdump.2 present. Can you make that downloadable somewhere? I'd like to take a closer look. The stack trace looks... almost random. It could be a double-fault (though I thought those were explicitly called out), but it looks like it panicked while handling a userland fault. This is going to sound like I'm pointing fingers elsewhere, but how old is your HW? Dan From mir at miras.org Mon Sep 19 15:51:28 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 19 Sep 2016 17:51:28 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> Message-ID: <20160919175128.11d667e3@sleipner.datanom.net> On Mon, 19 Sep 2016 11:35:33 -0400 Dan McDonald wrote: > > I see you have a vmdump.2 present. Can you make that downloadable somewhere? I'd like to take a closer look. > ftp://ftp.datanom.net/pub/vmdump.2 > The stack trace looks... almost random. It could be a double-fault (though I thought those were explicitly called out), but it looks like it panicked while handling a userland fault. This is going to sound like I'm pointing fingers elsewhere, but how old is your HW? > From smbios: Motherboard Manufacturer: ASRock Product: A75 Extreme6 CPU Manufacturer: AMD Version: AMD A8-3800 APU with Radeon(tm) HD Graphics Location Tag: CPUSocket Family: 72 (AMD A-series) Family Ext: 72 (AMD A-series) CPUID: 0x178bfbff00300f10 Type: 3 (central processor) Socket Upgrade: 41 (socket FM1) Socket Status: Populated Processor Status: 1 (enabled) Supported Voltages: 1.2V Core Count: 4 Cores Enabled: 4 Thread Count: 4 Processor Characteristics: 0x4 SMB_PRC_64BIT (64-bit capable) External Clock Speed: Unknown Maximum Speed: 2400MHz Current Speed: 2400MHz L1 Cache: 23 L2 Cache: 24 L3 Cache: None -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: "There was a boy called Eustace Clarence Scrubb, and he almost deserved it." -- C. S. Lewis, "The Chronicles of Narnia" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Mon Sep 19 15:55:54 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 19 Sep 2016 17:55:54 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <20160919175128.11d667e3@sleipner.datanom.net> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> Message-ID: <20160919175554.43bfc2dc@sleipner.datanom.net> On Mon, 19 Sep 2016 17:51:28 +0200 Michael Rasmussen wrote: > ftp://ftp.datanom.net/pub/vmdump.2 > Also via http: http://ftp.datanom.net/vmdump.2 -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Neil Armstrong tripped. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Mon Sep 19 17:33:02 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 19 Sep 2016 13:33:02 -0400 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <20160919175128.11d667e3@sleipner.datanom.net> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> Message-ID: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> > On Sep 19, 2016, at 11:51 AM, Michael Rasmussen wrote: > > On Mon, 19 Sep 2016 11:35:33 -0400 > Dan McDonald wrote: > >> >> I see you have a vmdump.2 present. Can you make that downloadable somewhere? I'd like to take a closer look. >> > ftp://ftp.datanom.net/pub/vmdump.2 This process: R 15776 2671 10 10 0 0x4a004000 ffffff075dadd0a0 perl T 0xffffff072006ec60 Hit a page fault: > 0xffffff071b9a8400::findstack -v stack pointer for thread ffffff071b9a8400: ffffff00206eb1f0 ffffff00206eb290 page_ctr_sub_internal+0x65(0, 3, ffffff0012581270, 1) ffffff00206eb2f0 page_ctr_sub+0x7e(0, 3, ffffff0012581270, 1) ffffff00206eb410 page_get_mnode_freelist+0x3ff(0, 4, 3, 0, 202000b) ffffff00206eb4e0 page_get_freelist+0x16d(286, 0, baddcafebaddcafe, fffffffffb934d9f, ffffff00206eb510, ffffff06e6fd8d00, ffffffff00000008) ffffff00206eb550 param_preset() ffffff00206eb5e0 0xfffffffffb8480fc() ffffff00206eb6f0 trap+0xa30(ffffff00206eb700, ffffff06e6fd8d00, 2) ffffff00206eb700 0xfffffffffb8001d6() ffffff00206eb8a0 fop_getpage+0x3f(ffffff06e6fd8d00, 1fffffe0ed7a1000, 1000, ffffff00206eba90, ffffff00206eb9f0, 1000, ffffff0760c25938, 8178000, ffffff0000000002, ffffff071db23ad8, 0) ffffff00206eb960 anon_getpage+0x18d(ffffff00206eba08, ffffff00206eba90, ffffff00206eb9f0, 1000, ffffff0760c25938, 8178000, 2, ffffff071db23ad8) ffffff00206eba80 segvn_faultpage+0x168(ffffff07213e2020, ffffff0760c25938, 8178000, 116000, 0, ffffff00206ebb50, 0, 1, 2, 1) ffffff00206ebc50 segvn_fault+0x8e6(ffffff07213e2020, ffffff0760c25938, 8178000 , 1000, 1, 2) ffffff00206ebd60 as_fault+0x31a(ffffff07213e2020, ffffff072245adc8, 817824c, 1 , 1, 2) ffffff00206ebdf0 pagefault+0x96(817824c, 1, 2, 0) ffffff00206ebf00 trap+0x2c7(ffffff00206ebf10, 817824c, 2) ffffff00206ebf10 0xfffffffffb8001d6() > And then trapped further while doing fop_getpage(). What's your swap device? I also couldn't help but notice this: > ffffff06e6fd8d00::print vnode_t v_vfsp v_vfsp = 0xbaddcafebaddcafe > WHOA! > ffffff06e6fd8d00::whatis ffffff06e6fd8d00 is ffffff06e6fd85c0+740, allocated from rootnex_dmahdl > So fop_getpage() was sent a vnode_t that at least had one field uninitialized (0xbaddcafe). Turns out the whole wad is: > ffffff06e6fd8d00/10P 0xffffff06e6fd8d00: 0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe > The anon cache entry fed into anon_getpage seems legit, but corrupt somehow: > ffffff00206eba08/P 0xffffff00206eba08: 0xffffff076bd0a5c0 > 0xffffff076bd0a5c0::print "struct anon" { an_vp = 0xffffff06e6fd8d00 an_pvp = 0 an_off = 0x1fffffe0ed7a1000 an_poff = 0 an_hash = 0 an_refcnt = 0x1 } > 0xffffff06e6fd8d00::whatis ffffff06e6fd8d00 is ffffff06e6fd85c0+740, allocated from rootnex_dmahdl > 0xffffff06e6fd8d00/20P 0xffffff06e6fd8d00: 0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe 0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe > 0xffffff076bd0a5c0::whatis ffffff076bd0a5c0 is allocated from anon_cache > Generally speaking, this is memory corruption. What caused it I'm not sure, however. Dan From mir at miras.org Mon Sep 19 17:43:13 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 19 Sep 2016 19:43:13 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> Message-ID: <20160919194313.2f302ae8@sleipner.datanom.net> On Mon, 19 Sep 2016 13:33:02 -0400 Dan McDonald wrote: > This process: > > R 15776 2671 10 10 0 0x4a004000 ffffff075dadd0a0 perl > T 0xffffff072006ec60 > > Only perl running here: # ps -ef |grep perl root 2684 2674 3 Sep 17 ? 267:07 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h root 2656 1 0 Sep 17 ? 0:31 /usr/perl5/bin/perl /usr/lib/intrd root 2695 2684 0 Sep 17 ? 53:48 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h > And then trapped further while doing fop_getpage(). What's your swap device? > # swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 91,2 8 4194296 4194296 -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Beggar to well-dressed businessman: "Could you spare $20.95 for a fifth of Chivas?" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Mon Sep 19 17:49:16 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 19 Sep 2016 13:49:16 -0400 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <20160919194313.2f302ae8@sleipner.datanom.net> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> <20160919194313.2f302ae8@sleipner.datanom.net> Message-ID: <1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com> > On Sep 19, 2016, at 1:43 PM, Michael Rasmussen wrote: > > On Mon, 19 Sep 2016 13:33:02 -0400 > Dan McDonald wrote: > >> This process: >> >> R 15776 2671 10 10 0 0x4a004000 ffffff075dadd0a0 perl >> T 0xffffff072006ec60 >> >> > Only perl running here: > # ps -ef |grep perl > root 2684 2674 3 Sep 17 ? 267:07 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h > root 2656 1 0 Sep 17 ? 0:31 /usr/perl5/bin/perl /usr/lib/intrd > root 2695 2684 0 Sep 17 ? 53:48 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h Remember, a page fault isn't a bad thing. It just means the process has a paged swapped out: > ::ps -f !grep 15776 R 15776 2671 10 10 0 0x4a004000 ffffff075dadd0a0 perl /var/web-gui/data/scripts/agent-nappit.pl '1474140713_hmp' '' > And THAT was the particular process running. >> And then trapped further while doing fop_getpage(). What's your swap device? >> > # swap -l > swapfile dev swaplo blocks free > /dev/zvol/dsk/rpool/swap 91,2 8 4194296 4194296 SO you've 4G of swap on rpool. Typical out of the box deployment. Something corrupted the anon cache entry, making it point to something in unused/unallocated/un-something hyperspace. The big question is what. This is why I asked about your hardware. It wasn't obvious, but since you're running NAPP-IT, I assume you're using ECC memory? Dan From mir at miras.org Mon Sep 19 17:52:54 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 19 Sep 2016 19:52:54 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> Message-ID: <20160919195254.55709606@sleipner.datanom.net> On Mon, 19 Sep 2016 13:33:02 -0400 Dan McDonald wrote: > > Generally speaking, this is memory corruption. What caused it I'm not sure, however. > A bit flip caused by radiation from Cosmos? ;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: My doctorate's in Literature, but it seems like a pretty good pulse to me. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Mon Sep 19 17:56:31 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 19 Sep 2016 19:56:31 +0200 Subject: [OmniOS-discuss] kernel panic In-Reply-To: <1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com> References: <20160917230344.493adaee@sleipner.datanom.net> <20160918000810.49ca7650@sleipner.datanom.net> <20160919175128.11d667e3@sleipner.datanom.net> <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com> <20160919194313.2f302ae8@sleipner.datanom.net> <1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com> Message-ID: <20160919195631.76ecba37@sleipner.datanom.net> On Mon, 19 Sep 2016 13:49:16 -0400 Dan McDonald wrote: > > Something corrupted the anon cache entry, making it point to something in unused/unallocated/un-something hyperspace. The big question is what. This is why I asked about your hardware. It wasn't obvious, but since you're running NAPP-IT, I assume you're using ECC memory? > I am afraid not. No support on the MB. Box is to be Upgraded sometime in the near future but serves a purpose for the moment. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: My doctorate's in Literature, but it seems like a pretty good pulse to me. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From robertfantini at gmail.com Wed Sep 21 14:33:35 2016 From: robertfantini at gmail.com (Robert Fantini) Date: Wed, 21 Sep 2016 10:33:35 -0400 Subject: [OmniOS-discuss] fan control In-Reply-To: References: Message-ID: Just to close this off, Our issue was hardware related. Motherboard / chassis / power supply mismatch on a testing system. New system fans work as expected. On Tue, May 24, 2016 at 7:28 PM, Robert Fantini wrote: > we've installed napp-it to a couple of systems . they run a lot louder - > fans do not seem to be controlled from software. > > is there a software package that that can be installed to handle sensors > and fans? > > Or is this something to deal with in Supermicro bios? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoffn at gnaa.net Wed Sep 21 18:40:51 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Wed, 21 Sep 2016 11:40:51 -0700 Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions Message-ID: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net> Hi. When I try to upgrade to r151018 from r151016 it can't complete. #pkg update Creating Plan (Checking for conflicting actions): / pkg update: The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4: link: pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z file: pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z These packages may not be installed together. Any non-conflicting set may be, or the packages must be corrected before they can be installed. The following packages all deliver file actions to etc/ssh/ssh_config: pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z These packages may not be installed together. Any non-conflicting set may be, or the packages must be corrected before they can be installed. these are the ssh packages currently installed: #pkg list | grep ssh network/openssh 7.2.2-0.151016 i-- service/network/ssh 0.5.11-0.151016 i-- Any suggestions on how to get it upgraded? thanks, Geoff From danmcd at omniti.com Wed Sep 21 18:50:11 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 21 Sep 2016 14:50:11 -0400 Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions In-Reply-To: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net> References: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net> Message-ID: > On Sep 21, 2016, at 2:40 PM, Geoff Nordli wrote: > > Hi. > > When I try to upgrade to r151018 from r151016 it can't complete. > > #pkg update > Creating Plan (Checking for conflicting actions): / > pkg update: The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4: > > link: > pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z > file: > pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. > > The following packages all deliver file actions to etc/ssh/ssh_config: > > pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z > pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z > > These packages may not be installed together. Any non-conflicting set may > be, or the packages must be corrected before they can be installed. Add these flags to the 'pkg upgrade' command: --reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key which will convert you from SunSSH to OpenSSH (which is good, because SunSSH goes away after r151018). Dan From geoffn at gnaa.net Wed Sep 21 20:36:37 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Wed, 21 Sep 2016 13:36:37 -0700 Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions In-Reply-To: References: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net> Message-ID: On 2016-09-21 11:50 AM, Dan McDonald wrote: >> On Sep 21, 2016, at 2:40 PM, Geoff Nordli wrote: >> >> Hi. >> >> When I try to upgrade to r151018 from r151016 it can't complete. >> >> #pkg update >> Creating Plan (Checking for conflicting actions): / >> pkg update: The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4: >> >> link: >> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z >> file: >> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z >> >> These packages may not be installed together. Any non-conflicting set may >> be, or the packages must be corrected before they can be installed. >> >> The following packages all deliver file actions to etc/ssh/ssh_config: >> >> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z >> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z >> >> These packages may not be installed together. Any non-conflicting set may >> be, or the packages must be corrected before they can be installed. > Add these flags to the 'pkg upgrade' command: > > --reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key > > which will convert you from SunSSH to OpenSSH (which is good, because SunSSH goes away after r151018). > > Dan > thanks Dan. This command did the trick: pkg update --reject network/ssh-common --reject service/network/ssh --reject ssh/ssh-key I couldn't do the --reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key pkg update: The following pattern(s) did not match any allowable packages. Try using a different matching pattern, or refreshing publisher information: omnios/network/ssh -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Sep 21 20:50:24 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 21 Sep 2016 16:50:24 -0400 Subject: [OmniOS-discuss] Bloody Update for September 21st & LX update/question References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com> Message-ID: <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com> Hello! I'm starting to wind down LX for now, and wind up the r151020 release. I'm leaning toward folding in LX support (see below) into stock OmniOS for r151020 as "LX Beta", with fixes and supportability showing up in r151022 (next LTS). uname -a says omnios-master-a10017b, which means illumos-omnios was at commit a10017b, and omnios-build was at commit a0dbf1f. New with this bloody update: - Bash is now 4.4 - OpenSSH support/SunSSH EOL is now easier to upgrade to. Instead of --reject lines, if you have SunSSH installed, you'll still *SEE* those packages, but they are now empty and "renamed" to the OpenSSH packages. Post-upgrade "pkg list | grep ssh" can show you what I mean if you're still on SunSSH. - Curl is now 7.50.3 (and was updated earlier as part of the security fix) - SEVERAL NVMe fixes. - nlockmgr now more reliable at boot time (7231 nlockmgr failing to start up during bootup) - Various ZFS bugfixes ALSO UPDATED is the bloody+LX repo: http://pkg.omniti.com/omnios/LX/. It is updated to illumos-joyent changes since their commit 6a41e72, and now quiesced. One bug was also fixed... SMAP support on Broadwell and newer Intel processors broke LX zones on OmniOS, because of a mismerge. I'M VERY CLOSE to folding in LX support into illumos-omnios's master branch, prior to construction of the r151020 branch. If people have opinions on this topic, I'd very much like to hear it. Also, is anyone besides OmniTI developers using LX for OmniOS? Thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dain.bentley at gmail.com Wed Sep 21 21:08:00 2016 From: dain.bentley at gmail.com (Dain Bentley) Date: Wed, 21 Sep 2016 21:08:00 +0000 (UTC) Subject: [OmniOS-discuss] Bloody Update for September 21st & LX update/question In-Reply-To: <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com> References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com> <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com> Message-ID: I use it Get Outlook for iOS On Wed, Sep 21, 2016 at 4:51 PM -0400, "Dan McDonald" wrote: Hello! I'm starting to wind down LX for now, and wind up the r151020 release. ?I'm leaning toward folding in LX support (see below) into stock OmniOS for r151020 as "LX Beta", with fixes and supportability showing up in r151022 (next LTS). uname -a says omnios-master-a10017b, which means illumos-omnios was at commit a10017b, and omnios-build was at commit a0dbf1f. New with this bloody update: - Bash is now 4.4 - OpenSSH support/SunSSH EOL is now easier to upgrade to. ?Instead of --reject lines, if you have SunSSH installed, you'll still *SEE* those packages, but they are now empty and "renamed" to the OpenSSH packages. ?Post-upgrade "pkg list | grep ssh" can show you what I mean if you're still on SunSSH. - Curl is now 7.50.3 (and was updated earlier as part of the security fix) - SEVERAL NVMe fixes. - nlockmgr now more reliable at boot time (7231 nlockmgr failing to start up during bootup) - Various ZFS bugfixes ALSO UPDATED is the bloody+LX repo: ?http://pkg.omniti.com/omnios/LX/. ?It is updated to illumos-joyent changes since their commit 6a41e72, and now quiesced. ?One bug was also fixed... SMAP support on Broadwell and newer Intel processors broke LX zones on OmniOS, because of a mismerge. I'M VERY CLOSE to folding in LX support into illumos-omnios's master branch, prior to construction of the r151020 branch. ?If people have opinions on this topic, I'd very much like to hear it. ?Also, is anyone besides OmniTI developers using LX for OmniOS? Thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Sep 22 18:22:39 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 22 Sep 2016 14:22:39 -0400 Subject: [OmniOS-discuss] OpenSSL 1.0.2i is out Message-ID: And available for "pkg update" on all supported OmniOS releases, plus bloody. Dan From skeltonr at btconnect.com Thu Sep 22 19:42:53 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Thu, 22 Sep 2016 20:42:53 +0100 Subject: [OmniOS-discuss] How can I increase the number of file descriptors for the apache service Message-ID: <57E4343D.1080809@btconnect.com> Hi, I have installed the apache22 package from ms.omniti.com and enabled the service included. I now fined the httpd has only 256 file descriptors. plimit 1188 1188: /opt/apache22/bin/amd64/httpd -f /opt/apache22/conf/httpd.conf -k star resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 10240 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 256 65536 vmemory(kbytes) unlimited unlimited The service starts httpd :- /opt/apache22/bin/httpd -f /opt/apache22/conf/httpd.conf -k start Found that by running :- svccfg -s svc:/network/http:apache22 listprop start/exec start/exec astring "/opt/apache22/bin/httpd -f /opt/apache22/conf/httpd.conf -k start" How do I set the file descriptors to 1024? Cheers Richard From mtalbott at lji.org Fri Sep 23 01:07:56 2016 From: mtalbott at lji.org (Michael Talbott) Date: Thu, 22 Sep 2016 18:07:56 -0700 Subject: [OmniOS-discuss] LX zone empty error messages Message-ID: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> So I'm finally getting around to trying out LX zones and can't seem to get the zone installed for some reason. Here's what I've done and where I'm stuck: # Fresh install of latest bloody # Setup basic networking # pkg update -v # reboot # pkg set-publisher -G http://pkg.omniti.com/omnios/bloody/ -g http://pkg.omniti.com/omnios/LX/ omnios # pkg install brand/lx # curl -o /root/centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file # zonecfg -z lx0 # create -b # set zonepath=/zones/lx0 # set brand=lx # set autoboot=false # set ip-type=exclusive # add net # set physical=lx0 # add property (name=gateway,value="192.168.0.1") # add property (name=ips,value="192.168.0.69/24") # add property (name=primary,value="true") # end # add attr # set name=dns-domain # set type=string # set value=example.com # end # add attr # set name=resolvers # set type=string # set value=192.168.0.1 # end # add attr # set name=kernel-version # set type=string # set value=2.6.32 # end # commit -- This is where it breaks # zoneadm -z lx0 install -s /root/centos6.zss.gz I get this.. root at bloodybox:/root# zoneadm -z lx0 install -s /root/centos6.zss.gz WARNING: skipping network interface 'lx0': object not found ERROR: Ok, I think I can fix the iface issue like so: # dladm create-vnic lx0 -l e1000g0 But, the empty error message persists and the zone never gets installed :( root at bloodybox:/root# zoneadm -z lx0 uninstall root at bloodybox:/root# zoneadm -z lx0 install -s /root/centos6.zss.gz ERROR: And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying? Thanks Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Fri Sep 23 01:42:18 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 22 Sep 2016 21:42:18 -0400 Subject: [OmniOS-discuss] LX zone empty error messages In-Reply-To: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> Message-ID: > On Sep 22, 2016, at 9:07 PM, Michael Talbott wrote: > And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying? Weird. One thing you can try is not only to uninstall the zone, but also delete its configuration (after exporting it) and then re-create it. I'm trying to reproduce your bug right now. I got this: zonecfg:lx2> export create -b set zonepath=/zones/lx2 set brand=lx set autoboot=false set ip-type=exclusive add net set physical=lx2 add property (name=gateway,value="x.y.z.1") add property (name=primary,value="true") add property (name=ips,value="x.y.z.246/24") end add attr set name=dns-domain set type=string set value=x.y.z.net end add attr set name=resolvers set type=string set value=x.y.z.1 end add attr set name=kernel-version set type=string set value=2.6.32 end zonecfg:lx2> exit # zoneadm list -cv ID NAME STATUS PATH BRAND IP 0 global running / ipkg shared 4 lx1 running /zones/lx1 lx excl - tz2 installed /zones/tz2 lipkg excl - lx0 installed /zones/lx0 lx excl - lx2 configured /zones/lx2 lx excl # zoneadm -z lx2 install -s /rpool/lx-stuff/joyent-images/centos6.gz WARNING: skipping network interface 'lx2': object not found A ZFS file system has been created for this zone. # zoneadm list -cv ID NAME STATUS PATH BRAND IP 0 global running / ipkg shared 4 lx1 running /zones/lx1 lx excl - tz2 installed /zones/tz2 lipkg excl - lx0 installed /zones/lx0 lx excl - lx2 installed /zones/lx2 lx excl # Okay... dumb question: is "/zones" on your box its own ZFS dataset? # zfs list | grep /zones rpool/zones 8.57G 34.6G 20K /zones rpool/zones/lx0 1.33G 34.6G 1.87G /zones/lx0 rpool/zones/lx1 1.86G 34.6G 1.81G /zones/lx1 rpool/zones/lx2 787M 34.6G 787M /zones/lx2 rpool/zones/tz2 4.62G 34.6G 20.5K /zones/tz2 rpool/zones/tz2/ROOT 4.62G 34.6G 19K legacy rpool/zones/tz2/ROOT/zbe-14 2.32M 34.6G 1.31G legacy rpool/zones/tz2/ROOT/zbe-15 2.50M 34.6G 1.24G legacy rpool/zones/tz2/ROOT/zbe-16 100K 34.6G 1.11G legacy rpool/zones/tz2/ROOT/zbe-17 1.89M 34.6G 1.25G legacy rpool/zones/tz2/ROOT/zbe-21 1.89M 34.6G 1.26G legacy rpool/zones/tz2/ROOT/zbe-22 2.55M 34.6G 1.26G legacy rpool/zones/tz2/ROOT/zbe-25 10.4M 34.6G 1.12G legacy rpool/zones/tz2/ROOT/zbe-29 73K 34.6G 1.12G legacy rpool/zones/tz2/ROOT/zbe-30 28.8M 34.6G 1.12G legacy rpool/zones/tz2/ROOT/zbe-31 18.5M 34.6G 1.11G legacy rpool/zones/tz2/ROOT/zbe-32 4.55G 34.6G 1.13G legacy rpool/zones/tz2/ROOT/zbe-33 68.5K 34.6G 1.13G legacy # It should be, and MAYBE that's causing your problem? Dan From mtalbott at lji.org Fri Sep 23 02:08:45 2016 From: mtalbott at lji.org (Michael Talbott) Date: Thu, 22 Sep 2016 19:08:45 -0700 Subject: [OmniOS-discuss] LX zone empty error messages In-Reply-To: References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> Message-ID: Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :) Thanks! > On Sep 22, 2016, at 6:42 PM, Dan McDonald wrote: > > >> On Sep 22, 2016, at 9:07 PM, Michael Talbott wrote: > > > >> And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying? > > Weird. > > One thing you can try is not only to uninstall the zone, but also delete its configuration (after exporting it) and then re-create it. > > I'm trying to reproduce your bug right now. I got this: > > zonecfg:lx2> export > create -b > set zonepath=/zones/lx2 > set brand=lx > set autoboot=false > set ip-type=exclusive > add net > set physical=lx2 > add property (name=gateway,value="x.y.z.1") > add property (name=primary,value="true") > add property (name=ips,value="x.y.z.246/24") > end > add attr > set name=dns-domain > set type=string > set value=x.y.z.net > end > add attr > set name=resolvers > set type=string > set value=x.y.z.1 > end > add attr > set name=kernel-version > set type=string > set value=2.6.32 > end > zonecfg:lx2> exit > # zoneadm list -cv > ID NAME STATUS PATH BRAND IP > 0 global running / ipkg shared > 4 lx1 running /zones/lx1 lx excl > - tz2 installed /zones/tz2 lipkg excl > - lx0 installed /zones/lx0 lx excl > - lx2 configured /zones/lx2 lx excl > # zoneadm -z lx2 install -s /rpool/lx-stuff/joyent-images/centos6.gz > WARNING: skipping network interface 'lx2': object not found > A ZFS file system has been created for this zone. > # zoneadm list -cv > ID NAME STATUS PATH BRAND IP > 0 global running / ipkg shared > 4 lx1 running /zones/lx1 lx excl > - tz2 installed /zones/tz2 lipkg excl > - lx0 installed /zones/lx0 lx excl > - lx2 installed /zones/lx2 lx excl > # > > > Okay... dumb question: is "/zones" on your box its own ZFS dataset? > > # zfs list | grep /zones > rpool/zones 8.57G 34.6G 20K /zones > rpool/zones/lx0 1.33G 34.6G 1.87G /zones/lx0 > rpool/zones/lx1 1.86G 34.6G 1.81G /zones/lx1 > rpool/zones/lx2 787M 34.6G 787M /zones/lx2 > rpool/zones/tz2 4.62G 34.6G 20.5K /zones/tz2 > rpool/zones/tz2/ROOT 4.62G 34.6G 19K legacy > rpool/zones/tz2/ROOT/zbe-14 2.32M 34.6G 1.31G legacy > rpool/zones/tz2/ROOT/zbe-15 2.50M 34.6G 1.24G legacy > rpool/zones/tz2/ROOT/zbe-16 100K 34.6G 1.11G legacy > rpool/zones/tz2/ROOT/zbe-17 1.89M 34.6G 1.25G legacy > rpool/zones/tz2/ROOT/zbe-21 1.89M 34.6G 1.26G legacy > rpool/zones/tz2/ROOT/zbe-22 2.55M 34.6G 1.26G legacy > rpool/zones/tz2/ROOT/zbe-25 10.4M 34.6G 1.12G legacy > rpool/zones/tz2/ROOT/zbe-29 73K 34.6G 1.12G legacy > rpool/zones/tz2/ROOT/zbe-30 28.8M 34.6G 1.12G legacy > rpool/zones/tz2/ROOT/zbe-31 18.5M 34.6G 1.11G legacy > rpool/zones/tz2/ROOT/zbe-32 4.55G 34.6G 1.13G legacy > rpool/zones/tz2/ROOT/zbe-33 68.5K 34.6G 1.13G legacy > # > > > It should be, and MAYBE that's causing your problem? > > Dan > From danmcd at omniti.com Fri Sep 23 02:31:43 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 22 Sep 2016 22:31:43 -0400 Subject: [OmniOS-discuss] LX zone empty error messages In-Reply-To: References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> Message-ID: <2F65EF55-2B24-4D80-AF26-062BB729593F@omniti.com> > On Sep 22, 2016, at 10:08 PM, Michael Talbott wrote: > > Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :) Sounds like, however, there needs to be better diagnostics somewhere. Thanks for adding that to the tests. (I suspect the Joyent-originated brand code assumes /zones always exists as a ZFS dataset, as it does in all SmartOS deployments.) Thanks! Dan From danmcd at omniti.com Fri Sep 23 21:29:31 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 23 Sep 2016 17:29:31 -0400 Subject: [OmniOS-discuss] LX zone empty error messages In-Reply-To: References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org> Message-ID: > On Sep 22, 2016, at 10:08 PM, Michael Talbott wrote: > > Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :) Followup, as much for me as for you, The failure isn't more verbose because the zones-common function get_zonepath_ds() requires a variable be set by the brand script, which our LX zone code does not, currently. We need to fix that, so instead of: ERROR: we can instead print something like what the ipkg/lipkg brand does: ERROR: the zonepath must be a ZFS dataset. The parent directory of the zonepath must be a ZFS dataset so that the zonepath ZFS dataset can be created properly. Now I'm beginning to understand why the brand scripts of ipkg were brought into Oracle's version of illumos-gate for Oracle Solaris. Some of the ipkg/lipkg items can/should be more common across other brands as well. Until that day, however, I will make sure the lx_install script has a proper $f_no_ds variable set. Thanks again, Michael! Dan From garland at bustle.com Fri Sep 23 23:25:03 2016 From: garland at bustle.com (Garland McAlexander) Date: Fri, 23 Sep 2016 19:25:03 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) Message-ID: Hi All, Trying to install the latest stable Omnios via USB. After booting from USB, I am greeted by the Omnios logo. One that proceeds into another screen, it loads a few things, gets to "syncing system files" and immediately after that something flashes (unable to see the text, even when spamming pause/break) and my server reboots. I have tried to recreate the USB multiple times, and I've done MD5 checking on my download. Any assistance would be absolutely fantastic. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Fri Sep 23 23:53:32 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 23 Sep 2016 18:53:32 -0500 (CDT) Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: Message-ID: On Fri, 23 Sep 2016, Garland McAlexander wrote: > Hi All, > > Trying to install the latest stable Omnios via USB. > > After booting from USB, I am greeted by the Omnios logo. One that proceeds > into another screen, it loads a few things, gets to "syncing system files" > and immediately after that something flashes (unable to see the text, even > when spamming pause/break) and my server reboots. > > I have tried to recreate the USB multiple times, and I've done MD5 checking > on my download. > > Any assistance would be absolutely fantastic. The most likely cause of the problem would be due to using USB3 hardware with an OS which only supports USB1 and USB2. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From mtalbott at lji.org Sat Sep 24 19:00:04 2016 From: mtalbott at lji.org (Michael Talbott) Date: Sat, 24 Sep 2016 12:00:04 -0700 Subject: [OmniOS-discuss] Reversed All-In-One Message-ID: So I just had this crazy idea while thinking about lx branded zones.. I'm loving lx zones. What a great piece of computing voodoo. I'd love to say there's no more need for things like ESX in an all in one setup, but, there's still windows and some random others that has their niche needs on a box. Anyway, I know there's plenty of folks out there that use ESXi as the root system then virtualize OmniOS and pass through storage controller cards to it to become a storage VM for their other VMs. I started thinking "what-if" we reversed that process to get the best of ALL worlds. That is, run OmniOS as root system and control storage natively and run all linux needs in zones. Then if, and a big if, we could run ESXi in an lx zone.. And IF that is even possible, another big if.. IF we can present a zone with a ZFS volume as a raw block device, ESXi could then use it's native VMFS format on said block device. If that were so, that would eliminate all traditional latencies and bottlenecks caused by the network stack be it iSCSI or NFS and provide all the benefits of ZFS such as checksums and compression inline to ESX.. Obviously it couldn't vmotion in a cluster since the storage would be local to that box, but, the thought is currently for an all in one setup and not a cluster ;) So I guess the first question is, does anyone think that ESXi's kernel is anywhere close enough to a linux kernel for this to even be feasible to run in an LX zone or am I totally just daydreaming here? I have my suspicions, but, that's all they are at this point. Second, and regardless of the above, does anyone have any good info on how to create a custom lx image? Surely it's not as simple as installing it on some box and then use tar and/or dd? Michael From paul.jochum at nokia.com Sun Sep 25 14:36:26 2016 From: paul.jochum at nokia.com (Paul Jochum) Date: Sun, 25 Sep 2016 09:36:26 -0500 Subject: [OmniOS-discuss] Reversed All-In-One In-Reply-To: References: Message-ID: Hi Michael: As to your 2nd qustion, about having good info on how to create a custom lx image, I have used the following for smartOS: https://github.com/joyent/centos-lx-brand-image-builder (I am not sure how closely OmniOS tracks the smartOS lx branded zones, but wanted to pass it along in case it helps). There are more links (in case you don't want to use centos as the base at: https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset Paul On 09/24/2016 02:00 PM, Michael Talbott wrote: > So I just had this crazy idea while thinking about lx branded zones.. I'm loving lx zones. What a great piece of computing voodoo. I'd love to say there's no more need for things like ESX in an all in one setup, but, there's still windows and some random others that has their niche needs on a box. > > Anyway, I know there's plenty of folks out there that use ESXi as the root system then virtualize OmniOS and pass through storage controller cards to it to become a storage VM for their other VMs. I started thinking "what-if" we reversed that process to get the best of ALL worlds. That is, run OmniOS as root system and control storage natively and run all linux needs in zones. Then if, and a big if, we could run ESXi in an lx zone.. And IF that is even possible, another big if.. IF we can present a zone with a ZFS volume as a raw block device, ESXi could then use it's native VMFS format on said block device. If that were so, that would eliminate all traditional latencies and bottlenecks caused by the network stack be it iSCSI or NFS and provide all the benefits of ZFS such as checksums and compression inline to ESX.. Obviously it couldn't vmotion in a cluster since the storage would be local to that box, but, the thought is currently for an all in one setup and not a cluster! > ;) > > So I guess the first question is, does anyone think that ESXi's kernel is anywhere close enough to a linux kernel for this to even be feasible to run in an LX zone or am I totally just daydreaming here? I have my suspicions, but, that's all they are at this point. > > Second, and regardless of the above, does anyone have any good info on how to create a custom lx image? Surely it's not as simple as installing it on some box and then use tar and/or dd? > > > Michael > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From jimklimov at cos.ru Sun Sep 25 16:48:13 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Sun, 25 Sep 2016 18:48:13 +0200 Subject: [OmniOS-discuss] Reversed All-In-One In-Reply-To: References: Message-ID: <7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru> 25 ???????? 2016??. 16:36:26 CEST, Paul Jochum ?????: >Hi Michael: > > As to your 2nd qustion, about having good info on how to create a >custom lx image, I have used the following for smartOS: >https://github.com/joyent/centos-lx-brand-image-builder > (I am not sure how closely OmniOS tracks the smartOS lx branded >zones, but wanted to pass it along in case it helps). There are more >links (in case you don't want to use centos as the base at: >https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset > >Paul > > >On 09/24/2016 02:00 PM, Michael Talbott wrote: >> So I just had this crazy idea while thinking about lx branded zones.. >I'm loving lx zones. What a great piece of computing voodoo. I'd love >to say there's no more need for things like ESX in an all in one setup, >but, there's still windows and some random others that has their niche >needs on a box. >> >> Anyway, I know there's plenty of folks out there that use ESXi as the >root system then virtualize OmniOS and pass through storage controller >cards to it to become a storage VM for their other VMs. I started >thinking "what-if" we reversed that process to get the best of ALL >worlds. That is, run OmniOS as root system and control storage natively >and run all linux needs in zones. Then if, and a big if, we could run >ESXi in an lx zone.. And IF that is even possible, another big if.. IF >we can present a zone with a ZFS volume as a raw block device, ESXi >could then use it's native VMFS format on said block device. If that >were so, that would eliminate all traditional latencies and bottlenecks >caused by the network stack be it iSCSI or NFS and provide all the >benefits of ZFS such as checksums and compression inline to ESX.. >Obviously it couldn't vmotion in a cluster since the storage would be >local to that box, but, the thought is currently for an all in one >setup and not a clust! > er! >> ;) >> >> So I guess the first question is, does anyone think that ESXi's >kernel is anywhere close enough to a linux kernel for this to even be >feasible to run in an LX zone or am I totally just daydreaming here? I >have my suspicions, but, that's all they are at this point. >> >> Second, and regardless of the above, does anyone have any good info >on how to create a custom lx image? Surely it's not as simple as >installing it on some box and then use tar and/or dd? >> >> >> Michael >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss As far as I know, ESX used a custom kernel as hypervosor (closer to BSD than Linux, perhaps because the latter would have to get opensourced). The linuxish environment seen in older (3.x) versions and hidden but uncoverable in newer ones is a controlling VM (IIRC, may be wrong here) allowed to grab the console and control the HW resources to present them back to hypervisor, and indeed based on Linux (quite compatible to redhat back then). To the point where I could compile kernel modules like md-raid and drivers to use setups other than certified officially. I guess the entanglement there is too intimate to port elsewhere, and why would you - given that you can run KVMs or VirtualBoxes natively in a global or local zone? ;) Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Sun Sep 25 16:55:51 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Sun, 25 Sep 2016 18:55:51 +0200 Subject: [OmniOS-discuss] Bloody Update for September 21st & LX update/question In-Reply-To: References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com> <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com> Message-ID: 21 ???????? 2016??. 23:08:00 CEST, Dain Bentley ?????: >I use it > >Get Outlook for iOS > > > > >On Wed, Sep 21, 2016 at 4:51 PM -0400, "Dan McDonald" > wrote: > > > > > > > > > > >Hello! > >I'm starting to wind down LX for now, and wind up the r151020 release. >?I'm leaning toward folding in LX support (see below) into stock OmniOS >for r151020 as "LX Beta", with fixes and supportability showing up in >r151022 (next LTS). > >uname -a says omnios-master-a10017b, which means illumos-omnios was at >commit a10017b, and omnios-build was at commit a0dbf1f. > >New with this bloody update: > >- Bash is now 4.4 > >- OpenSSH support/SunSSH EOL is now easier to upgrade to. ?Instead of >--reject lines, if you have SunSSH installed, you'll still *SEE* those >packages, but they are now empty and "renamed" to the OpenSSH packages. >?Post-upgrade "pkg list | grep ssh" can show you what I mean if you're >still on SunSSH. > >- Curl is now 7.50.3 (and was updated earlier as part of the security >fix) > >- SEVERAL NVMe fixes. > >- nlockmgr now more reliable at boot time (7231 nlockmgr failing to >start up during bootup) > >- Various ZFS bugfixes > > > >ALSO UPDATED is the bloody+LX repo: ?http://pkg.omniti.com/omnios/LX/. >?It is updated to illumos-joyent changes since their commit 6a41e72, >and now quiesced. ?One bug was also fixed... SMAP support on Broadwell >and newer Intel processors broke LX zones on OmniOS, because of a >mismerge. > >I'M VERY CLOSE to folding in LX support into illumos-omnios's master >branch, prior to construction of the r151020 branch. ?If people have >opinions on this topic, I'd very much like to hear it. ?Also, is anyone >besides OmniTI developers using LX for OmniOS? > >Thanks, >Dan > > > > > > > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss I play with it. :) Given desktop (laptop) usage of OI Hipster, I also honestly look forward to when LX stabilizes in OmniOS so I can snatch it and use also while commuting, experimenting or otherwise not near an OmniOS server ;) Jim -- Typos courtesy of K-9 Mail on my Samsung Android From garland at bustle.com Sun Sep 25 17:02:49 2016 From: garland at bustle.com (Garland McAlexander) Date: Sun, 25 Sep 2016 13:02:49 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: Message-ID: So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot? It seems odd that it would get to the point of loading, only to fail if it was USB related. On Fri, Sep 23, 2016 at 7:53 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote: > On Fri, 23 Sep 2016, Garland McAlexander wrote: > > Hi All, >> >> Trying to install the latest stable Omnios via USB. >> >> After booting from USB, I am greeted by the Omnios logo. One that proceeds >> into another screen, it loads a few things, gets to "syncing system files" >> and immediately after that something flashes (unable to see the text, even >> when spamming pause/break) and my server reboots. >> >> I have tried to recreate the USB multiple times, and I've done MD5 >> checking >> on my download. >> >> Any assistance would be absolutely fantastic. >> > > The most likely cause of the problem would be due to using USB3 hardware > with an OS which only supports USB1 and USB2. > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Sun Sep 25 19:34:51 2016 From: danmcd at omniti.com (Dan McDonald) Date: Sun, 25 Sep 2016 15:34:51 -0400 Subject: [OmniOS-discuss] Reversed All-In-One In-Reply-To: References: Message-ID: > On Sep 25, 2016, at 10:36 AM, Paul Jochum wrote: > > > As to your 2nd qustion, about having good info on how to create a custom lx image, I have used the following for smartOS: https://github.com/joyent/centos-lx-brand-image-builder > (I am not sure how closely OmniOS tracks the smartOS lx branded zones, but wanted to pass it along in case it helps). There are more links (in case you don't want to use centos as the base at: > https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset The LX Beta for OmniOS (r151020 will have LX Beta in it) has its install script capable of ZFS send streams (-s with an absolute pathname) or ZFS snapshots-or-datasets (-s using a zfs dataset name), or tarballs (-t). The -s data was intended to be used with either SmartOS compressed ZFS send streams or datasets managed by SmartOS's imgadm(1M) command. Dan From danmcd at omniti.com Sun Sep 25 19:36:01 2016 From: danmcd at omniti.com (Dan McDonald) Date: Sun, 25 Sep 2016 15:36:01 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: Message-ID: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> > On Sep 25, 2016, at 1:02 PM, Garland McAlexander wrote: > > So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot? The USB3.0 slot presents as currently-unsupported XHCI. You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS. Dan From matej at zunaj.si Mon Sep 26 07:04:25 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Mon, 26 Sep 2016 09:04:25 +0200 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> Message-ID: Usually one can disable USB3 support in BIOS and then it should work. Matej > On 25 Sep 2016, at 21:36, Dan McDonald wrote: > > >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander wrote: >> >> So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot? > > The USB3.0 slot presents as currently-unsupported XHCI. You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS. > > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Mon Sep 26 14:45:49 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 26 Sep 2016 10:45:49 -0400 Subject: [OmniOS-discuss] HEADS UP --> ANOTHER OpenSSL update References: <8CF9A138-CFCC-47B9-82AE-03E7217D2FE2@omniti.com> Message-ID: <157A1C64-B0CD-43B5-815B-AD5B9985EBEE@omniti.com> Say hello to 1.0.2j. Thank goodness I follow security types on twitter or I'd have missed this pre-OpenZFS. (Anyone on this list who's AT OpenZFS today, say hi! I'm wearing a "There's no place like usr/src/uts" shirt.) All of the repos have updated (and signed for 01[468]) OpenSSL 1.0.2j packages now. Happy re-updating! Dan From garland at bustle.com Mon Sep 26 15:04:23 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 11:04:23 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> Message-ID: I've disabled all USB 3.0 support in BIOS, but it's still failing to boot from USB. I still get the splash screen, and then a few lines of text until it reboots itself. On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik wrote: > Usually one can disable USB3 support in BIOS and then it should work. > > Matej > > > On 25 Sep 2016, at 21:36, Dan McDonald wrote: > > > > > >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander > wrote: > >> > >> So would a workaround be installing it via a USB2.0 drive, or will USB > not work at all due to the USB3.0 slot? > > > > The USB3.0 slot presents as currently-unsupported XHCI. You either need > to convinced your bios to do EHCI (USB2) or wait for XHCI support to land > in illumos & OmniOS. > > > > Dan > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daleg at omniti.com Mon Sep 26 15:32:26 2016 From: daleg at omniti.com (Dale Ghent) Date: Mon, 26 Sep 2016 11:32:26 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> Message-ID: <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> 2 options until the USB3 XHCI driver is ready (which it is close to being so) 1) Put in a USB 2 PCIe add-on card at least temporarily. 2) If your box has IPMI and virtual CDROM/remote disk KVM services, use those instead. > On Sep 26, 2016, at 11:04 AM, Garland McAlexander wrote: > > I've disabled all USB 3.0 support in BIOS, but it's still failing to boot from USB. I still get the splash screen, and then a few lines of text until it reboots itself. > > On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik wrote: > Usually one can disable USB3 support in BIOS and then it should work. > > Matej > > > On 25 Sep 2016, at 21:36, Dan McDonald wrote: > > > > > >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander wrote: > >> > >> So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot? > > > > The USB3.0 slot presents as currently-unsupported XHCI. You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS. > > > > Dan > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From lkateley at kateley.com Mon Sep 26 15:54:07 2016 From: lkateley at kateley.com (Linda Kateley) Date: Mon, 26 Sep 2016 10:54:07 -0500 Subject: [OmniOS-discuss] Reversed All-In-One In-Reply-To: <7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru> References: <7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru> Message-ID: + I run with virtual box on top of omni. Also really digging Proxmox which has native zfs. Linda On 9/25/16 11:48 AM, Jim Klimov wrote: > 25 ???????? 2016 ?. 16:36:26 CEST, Paul Jochum ?????: >> Hi Michael: >> >> As to your 2nd qustion, about having good info on how to create a >> custom lx image, I have used the following for smartOS: >> https://github.com/joyent/centos-lx-brand-image-builder >> (I am not sure how closely OmniOS tracks the smartOS lx branded >> zones, but wanted to pass it along in case it helps). There are more >> links (in case you don't want to use centos as the base at: >> https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset >> >> Paul >> >> >> On 09/24/2016 02:00 PM, Michael Talbott wrote: >>> So I just had this crazy idea while thinking about lx branded zones.. >> I'm loving lx zones. What a great piece of computing voodoo. I'd love >> to say there's no more need for things like ESX in an all in one setup, >> but, there's still windows and some random others that has their niche >> needs on a box. >>> Anyway, I know there's plenty of folks out there that use ESXi as the >> root system then virtualize OmniOS and pass through storage controller >> cards to it to become a storage VM for their other VMs. I started >> thinking "what-if" we reversed that process to get the best of ALL >> worlds. That is, run OmniOS as root system and control storage natively >> and run all linux needs in zones. Then if, and a big if, we could run >> ESXi in an lx zone.. And IF that is even possible, another big if.. IF >> we can present a zone with a ZFS volume as a raw block device, ESXi >> could then use it's native VMFS format on said block device. If that >> were so, that would eliminate all traditional latencies and bottlenecks >> caused by the network stack be it iSCSI or NFS and provide all the >> benefits of ZFS such as checksums and compression inline to ESX.. >> Obviously it couldn't vmotion in a cluster since the storage would be >> local to that box, but, the thought is currently for an all in one >> setup and not a clust! >> er! >>> ;) >>> >>> So I guess the first question is, does anyone think that ESXi's >> kernel is anywhere close enough to a linux kernel for this to even be >> feasible to run in an LX zone or am I totally just daydreaming here? I >> have my suspicions, but, that's all they are at this point. >>> Second, and regardless of the above, does anyone have any good info >> on how to create a custom lx image? Surely it's not as simple as >> installing it on some box and then use tar and/or dd? >>> >>> Michael >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > As far as I know, ESX used a custom kernel as hypervosor (closer to BSD than Linux, perhaps because the latter would have to get opensourced). The linuxish environment seen in older (3.x) versions and hidden but uncoverable in newer ones is a controlling VM (IIRC, may be wrong here) allowed to grab the console and control the HW resources to present them back to hypervisor, and indeed based on Linux (quite compatible to redhat back then). To the point where I could compile kernel modules like md-raid and drivers to use setups other than certified officially. > > I guess the entanglement there is too intimate to port elsewhere, and why would you - given that you can run KVMs or VirtualBoxes natively in a global or local zone? ;) > > Jim > -- > Typos courtesy of K-9 Mail on my Samsung Android > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From garland at bustle.com Mon Sep 26 15:56:03 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 11:56:03 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> Message-ID: Just attempted via IPMI CDROM and the same issue persists, gets to splash screen, loads a few lines of text and forces a reboot. On Mon, Sep 26, 2016 at 11:32 AM, Dale Ghent wrote: > > 2 options until the USB3 XHCI driver is ready (which it is close to being > so) > > 1) Put in a USB 2 PCIe add-on card at least temporarily. > 2) If your box has IPMI and virtual CDROM/remote disk KVM services, use > those instead. > > > On Sep 26, 2016, at 11:04 AM, Garland McAlexander > wrote: > > > > I've disabled all USB 3.0 support in BIOS, but it's still failing to > boot from USB. I still get the splash screen, and then a few lines of text > until it reboots itself. > > > > On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik wrote: > > Usually one can disable USB3 support in BIOS and then it should work. > > > > Matej > > > > > On 25 Sep 2016, at 21:36, Dan McDonald wrote: > > > > > > > > >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander > wrote: > > >> > > >> So would a workaround be installing it via a USB2.0 drive, or will > USB not work at all due to the USB3.0 slot? > > > > > > The USB3.0 slot presents as currently-unsupported XHCI. You either > need to convinced your bios to do EHCI (USB2) or wait for XHCI support to > land in illumos & OmniOS. > > > > > > Dan > > > > > > _______________________________________________ > > > OmniOS-discuss mailing list > > > OmniOS-discuss at lists.omniti.com > > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Mon Sep 26 16:01:47 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 26 Sep 2016 18:01:47 +0200 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> Message-ID: <20160926180147.58c7e5d3@sleipner.datanom.net> On Mon, 26 Sep 2016 11:56:03 -0400 Garland McAlexander wrote: > Just attempted via IPMI CDROM and the same issue persists, gets to splash > screen, loads a few lines of text and forces a reboot. > If that's the case then it can hardly be a USB-only issue. Maybe the chipset on the motherboard or the CPU is to new? What chipset and CPU is on the board? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Make sure your code "does nothing" gracefully. - The Elements of Programming Style (Kernighan & Plaugher) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From garland at bustle.com Mon Sep 26 16:03:26 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 12:03:26 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: <20160926180147.58c7e5d3@sleipner.datanom.net> References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> Message-ID: The board is a Supermicro X10DRW-iT which is a C612 chipset. On Mon, Sep 26, 2016 at 12:01 PM, Michael Rasmussen wrote: > On Mon, 26 Sep 2016 11:56:03 -0400 > Garland McAlexander wrote: > > > Just attempted via IPMI CDROM and the same issue persists, gets to splash > > screen, loads a few lines of text and forces a reboot. > > > If that's the case then it can hardly be a USB-only issue. Maybe the > chipset on the motherboard or the CPU is to new? > > What chipset and CPU is on the board? > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > Make sure your code "does nothing" gracefully. > - The Elements of Programming Style (Kernighan & Plaugher) > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Mon Sep 26 16:07:33 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 26 Sep 2016 18:07:33 +0200 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> Message-ID: <20160926180733.5cee0873@sleipner.datanom.net> On Mon, 26 Sep 2016 12:03:26 -0400 Garland McAlexander wrote: > The board is a Supermicro X10DRW-iT which is a C612 chipset. > Are you on latest BIOS firmware? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Absinthe makes the tart grow fonder. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From garland at bustle.com Mon Sep 26 16:10:41 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 12:10:41 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: <20160926180733.5cee0873@sleipner.datanom.net> References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> <20160926180733.5cee0873@sleipner.datanom.net> Message-ID: I'm on 2.0a, which looks to be the latest version. On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen wrote: > On Mon, 26 Sep 2016 12:03:26 -0400 > Garland McAlexander wrote: > > > The board is a Supermicro X10DRW-iT which is a C612 chipset. > > > Are you on latest BIOS firmware? > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > Absinthe makes the tart grow fonder. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johan.kragsterman at capvert.se Mon Sep 26 16:29:47 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Mon, 26 Sep 2016 18:29:47 +0200 Subject: [OmniOS-discuss] Ang: Re: Installation Failing (Reboots) In-Reply-To: References: , <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> <20160926180733.5cee0873@sleipner.datanom.net> Message-ID: Hi! I've seen this sometime, if I remember right, when I forgot to disable C-states. So, have you checked that C-states is disabled? Rgrds /J -----"OmniOS-discuss" skrev: ----- Till: Michael Rasmussen Fr?n: Garland McAlexander S?nt av: "OmniOS-discuss" Datum: 2016-09-26 16:11 Kopia: omnios-discuss ?rende: Re: [OmniOS-discuss] Installation Failing (Reboots) I'm on 2.0a, which looks to be the latest version.? On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen wrote: On Mon, 26 Sep 2016 12:03:26 -0400 Garland McAlexander wrote: > The board is a Supermicro X10DRW-iT which is a C612 chipset. > Are you on latest BIOS firmware? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Absinthe makes the tart grow fonder. _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From garland at bustle.com Mon Sep 26 16:34:31 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 12:34:31 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> <20160926180733.5cee0873@sleipner.datanom.net> Message-ID: Hi, I currently have all power technology (P, C, T) states disabled. On Mon, Sep 26, 2016 at 12:29 PM, Johan Kragsterman < johan.kragsterman at capvert.se> wrote: > > Hi! > > I've seen this sometime, if I remember right, when I forgot to disable > C-states. > > So, have you checked that C-states is disabled? > > Rgrds /J > > > -----"OmniOS-discuss" skrev: > ----- > Till: Michael Rasmussen > Fr?n: Garland McAlexander > S?nt av: "OmniOS-discuss" > Datum: 2016-09-26 16:11 > Kopia: omnios-discuss > ?rende: Re: [OmniOS-discuss] Installation Failing (Reboots) > > I'm on 2.0a, which looks to be the latest version. > > On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen wrote: > On Mon, 26 Sep 2016 12:03:26 -0400 > Garland McAlexander wrote: > > > The board is a Supermicro X10DRW-iT which is a C612 chipset. > > > Are you on latest BIOS firmware? > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > Absinthe makes the tart grow fonder. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Mon Sep 26 16:41:45 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 26 Sep 2016 18:41:45 +0200 Subject: [OmniOS-discuss] Ang: Re: Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160926180147.58c7e5d3@sleipner.datanom.net> <20160926180733.5cee0873@sleipner.datanom.net> Message-ID: <20160926184145.012ef299@sleipner.datanom.net> On Mon, 26 Sep 2016 18:29:47 +0200 Johan Kragsterman wrote: > Hi! > > I've seen this sometime, if I remember right, when I forgot to disable C-states. > > So, have you checked that C-states is disabled? > I have never heard of enabled C-states should prevent booting? I have heard of C-states influence stability in the long run. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: God is dead and I don't feel all too well either.... -- Ralph Moonen -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From gate03 at landcroft.co.uk Mon Sep 26 19:58:23 2016 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 27 Sep 2016 05:58:23 +1000 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> Message-ID: <20160927055823.778d3b9e@kendall> On Mon, 26 Sep 2016 11:56:03 -0400 Garland McAlexander wrote: > Just attempted via IPMI CDROM and the same issue persists, gets to > splash screen, loads a few lines of text and forces a reboot. I had this once and it turned out to be defective media. Try another stick? ______________ Michael Mounteney From garland at bustle.com Mon Sep 26 19:59:06 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 15:59:06 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: <20160927055823.778d3b9e@kendall> References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: I've tried multiple images via IPMI now, and I've md5 verified them all before attempting to install. On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney wrote: > On Mon, 26 Sep 2016 11:56:03 -0400 > Garland McAlexander wrote: > > > Just attempted via IPMI CDROM and the same issue persists, gets to > > splash screen, loads a few lines of text and forces a reboot. > > I had this once and it turned out to be defective media. Try another > stick? > > ______________ > Michael Mounteney > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff at 1beyond.com Mon Sep 26 20:19:42 2016 From: jeff at 1beyond.com (Jeff Berkembrock) Date: Mon, 26 Sep 2016 16:19:42 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: Now that you're using IPMI, try to press Print Screen right as the text flashes on the screen. You might be able to snap a screenshot of it and read what it says. Best Regards, Jeff Berkembrock www.1beyond.com 617-591-2200 x2212 On Mon, Sep 26, 2016 at 3:59 PM, Garland McAlexander wrote: > I've tried multiple images via IPMI now, and I've md5 verified them all > before attempting to install. > > On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney > wrote: > >> On Mon, 26 Sep 2016 11:56:03 -0400 >> Garland McAlexander wrote: >> >> > Just attempted via IPMI CDROM and the same issue persists, gets to >> > splash screen, loads a few lines of text and forces a reboot. >> >> I had this once and it turned out to be defective media. Try another >> stick? >> >> ______________ >> Michael Mounteney >> > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garland at bustle.com Mon Sep 26 20:31:24 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 16:31:24 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: That lead me to recording the IPMI session and I now get this: ? On Mon, Sep 26, 2016 at 4:19 PM, Jeff Berkembrock wrote: > Now that you're using IPMI, try to press Print Screen right as the text > flashes on the screen. You might be able to snap a screenshot of it and > read what it says. > > Best Regards, > > Jeff Berkembrock > www.1beyond.com > 617-591-2200 x2212 > > On Mon, Sep 26, 2016 at 3:59 PM, Garland McAlexander > wrote: > >> I've tried multiple images via IPMI now, and I've md5 verified them all >> before attempting to install. >> >> On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney < >> gate03 at landcroft.co.uk> wrote: >> >>> On Mon, 26 Sep 2016 11:56:03 -0400 >>> Garland McAlexander wrote: >>> >>> > Just attempted via IPMI CDROM and the same issue persists, gets to >>> > splash screen, loads a few lines of text and forces a reboot. >>> >>> I had this once and it turned out to be defective media. Try another >>> stick? >>> >>> ______________ >>> Michael Mounteney >>> >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-09-26 at 4.30.42 PM.png Type: image/png Size: 122333 bytes Desc: not available URL: From josh at sysmgr.org Mon Sep 26 20:33:48 2016 From: josh at sysmgr.org (Joshua M. Clulow) Date: Mon, 26 Sep 2016 13:33:48 -0700 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: On 26 September 2016 at 12:59, Garland McAlexander wrote: > I've tried multiple images via IPMI now, and I've md5 verified them all > before attempting to install. Does your system have serial-over-LAN functionality available through IPMI? I would recommend trying to get the system to boot using the serial port as the console -- that way, when the text flashes up you'll be able to capture it before the system reboots. I've used the OmniOS installer via the serial port (in a virtual machine) before. I think that all I had to do was edit the grub menu entry and add the appropriate console settings, e.g. -B console=ttyb,ttyb-mode="115200,8,n,1,-" (NB: that's for "ttyb", aka COM2. If your IPMI serial-over-LAN port is COM1 you'll want ttya instead.) -- Joshua M. Clulow UNIX Admin/Developer http://blog.sysmgr.org From garland at bustle.com Mon Sep 26 20:50:05 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 16:50:05 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: https://youtu.be/JlbxQg01AdA Here it is in action. On Mon, Sep 26, 2016 at 4:33 PM, Joshua M. Clulow wrote: > On 26 September 2016 at 12:59, Garland McAlexander > wrote: > > I've tried multiple images via IPMI now, and I've md5 verified them all > > before attempting to install. > > Does your system have serial-over-LAN functionality available through > IPMI? I would recommend trying to get the system to boot using the > serial port as the console -- that way, when the text flashes up > you'll be able to capture it before the system reboots. > > I've used the OmniOS installer via the serial port (in a virtual > machine) before. I think that all I had to do was edit the grub menu > entry and add the appropriate console settings, e.g. > > -B console=ttyb,ttyb-mode="115200,8,n,1,-" > > (NB: that's for "ttyb", aka COM2. If your IPMI serial-over-LAN port > is COM1 you'll want ttya instead.) > > -- > Joshua M. Clulow > UNIX Admin/Developer > http://blog.sysmgr.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh at sysmgr.org Mon Sep 26 20:52:49 2016 From: josh at sysmgr.org (Joshua M. Clulow) Date: Mon, 26 Sep 2016 13:52:49 -0700 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: On 26 September 2016 at 13:31, Garland McAlexander wrote: > That lead me to recording the IPMI session and I now get this: I've seen that message ("unsupported mixed cpu monitor/mwait support detected") in another thread recently. In that case, I believe the solution was to reset the BIOS to an "optimised performance" factory defaults set. Can you try various default BIOS settings that to see if that allows the system to boot? Cheers. -- Joshua M. Clulow UNIX Admin/Developer http://blog.sysmgr.org From garland at bustle.com Mon Sep 26 21:07:04 2016 From: garland at bustle.com (Garland McAlexander) Date: Mon, 26 Sep 2016 17:07:04 -0400 Subject: [OmniOS-discuss] Installation Failing (Reboots) In-Reply-To: References: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com> <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com> <20160927055823.778d3b9e@kendall> Message-ID: Spot on! Resetting to optimized defaults did the trick. Such strange behaviour. Thanks a TON for the advice! On Mon, Sep 26, 2016 at 4:52 PM, Joshua M. Clulow wrote: > On 26 September 2016 at 13:31, Garland McAlexander > wrote: > > That lead me to recording the IPMI session and I now get this: > > I've seen that message ("unsupported mixed cpu monitor/mwait support > detected") in another thread recently. In that case, I believe the > solution was to reset the BIOS to an "optimised performance" factory > defaults set. > > Can you try various default BIOS settings that to see if that allows > the system to boot? > > > Cheers. > > -- > Joshua M. Clulow > UNIX Admin/Developer > http://blog.sysmgr.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info.fmo at gmx.de Mon Sep 26 22:14:28 2016 From: info.fmo at gmx.de (Frank M.) Date: Tue, 27 Sep 2016 00:14:28 +0200 Subject: [OmniOS-discuss] bloody update error pkg Message-ID: <53794389.20160927001428@gmx.de> Hi, I wanted to update my 151019-test-installation: root at test-san1:/root# pkg update Checking that pkg(5) is up to date - Errors were encountered while attempting to retrieve package or file data for the requested operation. Details follow: http protocol error: code: 404 reason: Not Found URL: 'http://pkg.omniti.com/omnios/bloody/omnios/manifest/0/package%2Fpkg at 0.5.11%2C5.11-0.151019%3A20160824T050133Z' (happened 4 times) Frank --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft. https://www.avast.com/antivirus From danmcd at omniti.com Mon Sep 26 22:28:43 2016 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 26 Sep 2016 18:28:43 -0400 Subject: [OmniOS-discuss] bloody update error pkg In-Reply-To: <53794389.20160927001428@gmx.de> References: <53794389.20160927001428@gmx.de> Message-ID: Pardon the top-post. I'm rebuilding the package index, which should fix this. I believe it's papering over a possible problem in our front-end. Try in another 20 minutes or so. Dan > On Sep 26, 2016, at 6:14 PM, Frank M. wrote: > > Hi, > > I wanted to update my 151019-test-installation: > > root at test-san1:/root# pkg update > Checking that pkg(5) is up to date - > > Errors were encountered while attempting to retrieve package or file data for > the requested operation. > Details follow: > > http protocol error: code: 404 reason: Not Found > URL: 'http://pkg.omniti.com/omnios/bloody/omnios/manifest/0/package%2Fpkg at 0.5.11%2C5.11-0.151019%3A20160824T050133Z' (happened 4 times) > > > Frank > > > --- > Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft. > https://www.avast.com/antivirus > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From bouktaib.mohssin at gmail.com Tue Sep 27 11:23:42 2016 From: bouktaib.mohssin at gmail.com (Bouktaib Mohssin) Date: Tue, 27 Sep 2016 12:23:42 +0100 Subject: [OmniOS-discuss] Booting omnios from grub2 Message-ID: ?Hello Community I'm trying to boot omnios from grub2 using my ubuntu server ,i've triedit with smartos & it work ( https://wiki.smartos.org/display/DOC/Booting+SmartOS+from+GRUB2) my question is how to boot omnios from grub2 thanks :)? -------------- next part -------------- An HTML attachment was scrubbed... URL: From nshalman at omniti.com Tue Sep 27 13:21:02 2016 From: nshalman at omniti.com (Nahum Shalman) Date: Tue, 27 Sep 2016 09:21:02 -0400 Subject: [OmniOS-discuss] Booting omnios from grub2 In-Reply-To: References: Message-ID: Hi, I see I will now have to pay for my sins for having written that wiki page... The short answer is that SmartOS is far more amenable to booting from arbitrary boot loaders than OmniOS. With SmartOS all you care about is booting the correct kernel and boot archive. OmniOS has boot environments (BEs). Booting into the correct BE requires more information on the kernel command line than SmartOS does. The `beadm` tool on OmniOS integrates with GRUB but has no integration for GRUB2. In the future, upstream illumos has recently landed a port of the FreeBSD loader which has many of the same desirable features as GRUB2, but also includes BE/beadm integration. Also, you'd have to be certain that the version of GRUB2 in question is compatible with the pool features of the pool that OmniOS is on. There are possibly some very clunky ways to get GRUB2 to look at the GRUB configuration from OmniOS, but without explanation of *why* you want to do this and how you expect it to behave, I'm not prepared to opine about precisely how difficult it would be. Could you clarify what you're trying to accomplish? -Nahum On Tue, Sep 27, 2016 at 7:23 AM, Bouktaib Mohssin < bouktaib.mohssin at gmail.com> wrote: > ?Hello Community > > I'm trying to boot omnios from grub2 using my ubuntu server ,i've triedit > with smartos & it work (https://wiki.smartos.org/ > display/DOC/Booting+SmartOS+from+GRUB2) > my question is how to boot omnios from grub2 > > thanks :)? > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Wed Sep 28 07:42:07 2016 From: mir at miras.org (Michael Rasmussen) Date: Wed, 28 Sep 2016 09:42:07 +0200 Subject: [OmniOS-discuss] Changing motherboard Message-ID: <20160928094207.2d8a65b2@sleipner.datanom.net> Hi all, In the coming days I am going to change motherboard in an Omnios server. then change also involves change CPU from AMD to Intel. The system has two pools - rpool and a storage pool. The server has a bond on two add-in 82574L PCIe which will be replaced by two on-board one i217LM and one i210AT. And add-in card hosts a dual Infiniband which will be transferred. The storage pool is hosted on a SAS HBA which likewise will be transferred. the rpool is connected through on-board SATA which is also the way it will be connected on the new board. My hope is that the server just comes up on the new board but are there anything special that should be done before shutting down the old system? Maybe export the storage pool? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Somewhere, just out of sight, the unicorns are gathering. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From alka at hfg-gmuend.de Wed Sep 28 14:29:12 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Wed, 28 Sep 2016 16:29:12 +0200 Subject: [OmniOS-discuss] Changing motherboard In-Reply-To: <20160928094207.2d8a65b2@sleipner.datanom.net> References: <20160928094207.2d8a65b2@sleipner.datanom.net> Message-ID: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de> I would probably - export the datapool - delete all nic interfaces then try to reboot from the disk on the new system - create new nic interfaces and ip adresses - import the datapool Gea Am 28.09.2016 um 09:42 schrieb Michael Rasmussen: > Hi all, > > In the coming days I am going to change motherboard in an Omnios > server. then change also involves change CPU from AMD to Intel. The > system has two pools - rpool and a storage pool. The server has a bond > on two add-in 82574L PCIe which will be replaced by two on-board > one i217LM and one i210AT. And add-in card hosts a dual Infiniband > which will be transferred. The storage pool is hosted on a SAS HBA > which likewise will be transferred. the rpool is connected through > on-board SATA which is also the way it will be connected on the new > board. > > My hope is that the server just comes up on the new board but are there > anything special that should be done before shutting down the old > system? Maybe export the storage pool? > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From johan.kragsterman at capvert.se Wed Sep 28 14:46:39 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Wed, 28 Sep 2016 16:46:39 +0200 Subject: [OmniOS-discuss] Ang: Re: Changing motherboard In-Reply-To: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de> References: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>, <20160928094207.2d8a65b2@sleipner.datanom.net> Message-ID: Hi! -----"OmniOS-discuss" skrev: ----- Till: omnios-discuss at lists.omniti.com Fr?n: Guenther Alka S?nt av: "OmniOS-discuss" Datum: 2016-09-28 16:34 ?rende: Re: [OmniOS-discuss] Changing motherboard I would probably - export the datapool - delete all nic interfaces then try to reboot from the disk on the new system - create new nic interfaces and ip adresses - import the datapool Gea Yeah, it can be a mess with old nic interfaces. I've also been thinking about how to do that. You should delete the eth if's, but perhaps you should also delete the phy's? dladm delete-phys? I remember when I swapped mobo some years ago, and used the old system disk, I got this problem with the old phy's. I deleted them in the new system, but ran into some problems due to that, don't remember now what it was. /J Am 28.09.2016 um 09:42 schrieb Michael Rasmussen: Hi all, In the coming days I am going to change motherboard in an Omnios server. then change also involves change CPU from AMD to Intel. The system has two pools - rpool and a storage pool. The server has a bond on two add-in 82574L PCIe which will be replaced by two on-board one i217LM and one i210AT. And add-in card hosts a dual Infiniband which will be transferred. The storage pool is hosted on a SAS HBA which likewise will be transferred. the rpool is connected through on-board SATA which is also the way it will be connected on the new board. My hope is that the server just comes up on the new board but are there anything special that should be done before shutting down the old system? Maybe export the storage pool? _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Wed Sep 28 17:15:28 2016 From: mir at miras.org (Michael Rasmussen) Date: Wed, 28 Sep 2016 19:15:28 +0200 Subject: [OmniOS-discuss] Changing motherboard In-Reply-To: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de> References: <20160928094207.2d8a65b2@sleipner.datanom.net> <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de> Message-ID: <20160928191528.5b89a93b@sleipner.datanom.net> On Wed, 28 Sep 2016 16:29:12 +0200 Guenther Alka wrote: > > - export the datapool > - delete all nic interfaces > Do you think deleting nics also includes the Infiniband ports/partitions given they reside on a dedicated HCA? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: He hated being thought of as one of those people that wore stupid ornamental armour. It was gilt by association. -- Terry Pratchett, "Night Watch" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Thu Sep 29 00:12:27 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 29 Sep 2016 02:12:27 +0200 Subject: [OmniOS-discuss] vlan and vnic Message-ID: <20160929021227.2fe5f04f@sleipner.datanom.net> Hi all, I have been wondering what is the best way to create vlans in omnios. use dladm create-vlan and assign IP to the vlan or use dladm create-vnic with a vlan tag and assign IP to the vnic What are the pros and cons of each? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Kirk to Enterprise -- beam down yeoman Rand and a six-pack. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From daleg at omniti.com Thu Sep 29 06:43:54 2016 From: daleg at omniti.com (Dale Ghent) Date: Thu, 29 Sep 2016 02:43:54 -0400 Subject: [OmniOS-discuss] vlan and vnic In-Reply-To: <20160929021227.2fe5f04f@sleipner.datanom.net> References: <20160929021227.2fe5f04f@sleipner.datanom.net> Message-ID: > On Sep 28, 2016, at 8:12 PM, Michael Rasmussen wrote: > > Hi all, > > I have been wondering what is the best way to create vlans in omnios. > use dladm create-vlan and assign IP to the vlan > or > use dladm create-vnic with a vlan tag and assign IP to the vnic > > What are the pros and cons of each? Functionally as far as VLANs go, they are equivalent. However, VNICs make things more flexible. VNICs can consume a aggregated set of physical interfaces, or a single physical interface. VNICs in turn can be assigned to Zones. Here's an operational example that we use: 1) 2 physical interfaces: igb0 and igb1 2) igb0 and igb1 are made into an aggregated NIC we will name "aggr0" (dladm create-aggr ... aggr0) Now we have a redundant Layer 2. On the switch, we assign the ports these are connected to all the tagged VLANs that should participate. 3) For the global zone, we make a new VNIC from that aggr. Let's say the global zone's interface is on vlan 5. Thus, we do: gz# dladm create-vnic -d aggr0 -v 5 global0 Okay, good. Now the global zone has its interface (named "global0"), now you just assign it a IP and default route and all that using the standard commands (ipadm, route -p add ...) If you don't have more than 1 physical interface, then you can ignore those first 3 steps. 4) Now we have some zones we want to add, but they need to talk to other VLANs, because VLAN 5 that the global zone is on is a private network that they should not see. Well, we create a VNIC for each zone and assign it to the zone with zonecfg: gz# dladm create-vnic -d aggr0 -v 20 foozone0 gz# zonecfg -z foozone zonecfg:foozone> add net zonecfg:foozone> set physical=foozone0 ... Now the zone "foozone" will have its own "NIC" that can participate only on VLAN 20. /dale From alka at hfg-gmuend.de Thu Sep 29 09:58:42 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Thu, 29 Sep 2016 11:58:42 +0200 Subject: [OmniOS-discuss] Changing motherboard In-Reply-To: <20160928191528.5b89a93b@sleipner.datanom.net> References: <20160928094207.2d8a65b2@sleipner.datanom.net> <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de> <20160928191528.5b89a93b@sleipner.datanom.net> Message-ID: I have not tried IB - If they change link name/numbering then yes Am 28.09.2016 um 19:15 schrieb Michael Rasmussen: > On Wed, 28 Sep 2016 16:29:12 +0200 > Guenther Alka wrote: > >> - export the datapool >> - delete all nic interfaces >> > Do you think deleting nics also includes the Infiniband > ports/partitions given they reside on a dedicated HCA? > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.jochum at nokia.com Thu Sep 29 14:54:39 2016 From: paul.jochum at nokia.com (Paul Jochum) Date: Thu, 29 Sep 2016 09:54:39 -0500 Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? Message-ID: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> Hi All: Looking for the latest recommendations on a low powered and cheap OmniOS server. This would be for home use, looking to use it as way of backing up Linux and Windows boxes. Anyone have a recommendation? thanks in advance, Paul From davide.poletto at gmail.com Thu Sep 29 15:08:59 2016 From: davide.poletto at gmail.com (Davide Poletto) Date: Thu, 29 Sep 2016 17:08:59 +0200 Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> Message-ID: My take: HPE ProLiant MicroServer Gen8 (the one G1610T CPU based) or a Gen7 too: sufficiently silent (not completely silent since has 2 fans), it doesn't consume so much power (with Toshiba DT01ACAxxx drives is really silent - no vibrations), keep in mind that it needs expensive "transposers" if you're planning to use SSD 2,5" disks on its 3,5" bays (it has 4)...overall IMHO, it's compact enough and it works flawlessly with OmniOS (tested on N36L, N40L and N54L which was Gen 7). I have a few at home mostly equipped with 8GB or 16GB ECC Ram (a must with ZFS, clearly) and Toshiba 1TB/2TB HDDs. OS is installed on a dedicated 40GB Intel SSD 320 series connected into the optical drive bay through its 5th SATA cable (internal). If you're brave enough you can test OmniOS on a USB Flash...but I hard endurance troubles (so I switched to SSD for the peace of my mind)...NAS4Free (FreeBSD based) runs flawlessly on USB Stick on a pair of MicroServer I deployed as long term ZFS backup boxes (really rock solid...on a Gen 7...about 37k working hours without a single glitch). Hope it helps. On Thu, Sep 29, 2016 at 4:54 PM, Paul Jochum wrote: > Hi All: > > Looking for the latest recommendations on a low powered and cheap > OmniOS server. This would be for home use, looking to use it as way of > backing up Linux and Windows boxes. Anyone have a recommendation? > > thanks in advance, > Paul > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Thu Sep 29 15:40:20 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 29 Sep 2016 17:40:20 +0200 Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> Message-ID: <20160929174020.63d2944c@sleipner.datanom.net> On Thu, 29 Sep 2016 09:54:39 -0500 Paul Jochum wrote: > Hi All: > > Looking for the latest recommendations on a low powered and cheap OmniOS server. This would be for home use, looking to use it as way of backing up Linux and Windows boxes. Anyone have a recommendation? > You could build one yourself based on Intel I3-4(1|3)xxT (35W) using a supermicro x10sll-f and you will have a solid build for $300 (MB+CPU+RAM). -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: A mathematician is a device for turning coffee into theorems. -- P. Erdos -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From davide.poletto at gmail.com Thu Sep 29 15:47:49 2016 From: davide.poletto at gmail.com (Davide Poletto) Date: Thu, 29 Sep 2016 17:47:49 +0200 Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? In-Reply-To: <20160929174020.63d2944c@sleipner.datanom.net> References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> <20160929174020.63d2944c@sleipner.datanom.net> Message-ID: Uh...forgot...it's also cheap: the G1610T equipped with 4GB ECC - diskless - should cost 170 Euro excluding VAT...and it has HPE iLO4 for management on shared LAN (2x1Gbps). My 2 cents. On Thu, Sep 29, 2016 at 5:40 PM, Michael Rasmussen wrote: > On Thu, 29 Sep 2016 09:54:39 -0500 > Paul Jochum wrote: > > > Hi All: > > > > Looking for the latest recommendations on a low powered and cheap > OmniOS server. This would be for home use, looking to use it as way of > backing up Linux and Windows boxes. Anyone have a recommendation? > > > You could build one yourself based on Intel I3-4(1|3)xxT (35W) using a > supermicro x10sll-f and you will have a solid build for $300 > (MB+CPU+RAM). > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > A mathematician is a device for turning coffee into theorems. > -- P. Erdos > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Thu Sep 29 15:52:17 2016 From: mir at miras.org (Michael Rasmussen) Date: Thu, 29 Sep 2016 17:52:17 +0200 Subject: [OmniOS-discuss] vlan and vnic In-Reply-To: References: <20160929021227.2fe5f04f@sleipner.datanom.net> Message-ID: <20160929175217.0e55356b@sleipner.datanom.net> On Thu, 29 Sep 2016 02:43:54 -0400 Dale Ghent wrote: > > Functionally as far as VLANs go, they are equivalent. However, VNICs make things more flexible. > > VNICs can consume a aggregated set of physical interfaces, or a single physical interface. VNICs in turn can be assigned to Zones. Another feature in vnic not available in vlan is that a vnic can be assigned a mac which can come in handy. A vlan cannot be assigned a mac so I assume a vlan inherits the mac from the link. Given all this I have a hard time to find any pros for vlans compared to vnics. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: On the Internet, no one knows you're using Windows NT -- Submitted by Ramiro Estrugo, restrugo at fateware.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From bfriesen at simple.dallas.tx.us Thu Sep 29 15:52:34 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 29 Sep 2016 10:52:34 -0500 (CDT) Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> Message-ID: On Thu, 29 Sep 2016, Paul Jochum wrote: > Hi All: > > Looking for the latest recommendations on a low powered and cheap OmniOS > server. This would be for home use, looking to use it as way of backing up > Linux and Windows boxes. Anyone have a recommendation? I am very happy with my 1U SuperMicro server based on the Xeon-D SOC. It takes hardly any power at all, makes very little noise, and performance is pretty good. The system boots from two Samsung AHCI M.2 drives (one M.2 slot is via an add-in card). The main drawback is that even though the chassis provides 8 hot-swap SFF drive bays, the SOC only supports 6 SATA interfaces so only 6 of those bays can be used without consuming the single card slot for a HBA. If one adds an HBA then 8 hot-swap SFF slots are available, but only one M.2 slot is then available for use as a boot device. I am not sure how much total storage can be attained with 6 SFF drives for backup server purposes. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From natxo.asenjo at gmail.com Thu Sep 29 18:33:32 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Thu, 29 Sep 2016 20:33:32 +0200 Subject: [OmniOS-discuss] Low powered and cheap OmniOS server? In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com> Message-ID: hi, On Thu, Sep 29, 2016 at 4:54 PM, Paul Jochum wrote: > Hi All: > > Looking for the latest recommendations on a low powered and cheap > OmniOS server. This would be for home use, looking to use it as way of > backing up Linux and Windows boxes. Anyone have a recommendation? > HP microserver 40NL, with a IBM serveraid m1015 sas/sata raid controller in jbod mode. I use the internal 4 sata 2 connectors with 4x3Tib hard drives, and I bought one of these (not available any more, unfortunately): https://azerty.nl/0-1651-403300/sharkoon-sata-quickport-intern-6-bay-.html I replaced the cd/dvd drive with that bay expansion, I have 2 laptop drives as rpool and one ssd as cache. If I run out of space I can add 3 2,5'' hard drives to the pool, or create a new pool. It even has a e-sata connection, so I could add an external enclosure as well. I got the kvm module as well, so it's quite a convenient litlle thing. Not the fastest cpu, but hey, for that price and low consumption .. ;-) -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Fri Sep 30 02:14:39 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 29 Sep 2016 21:14:39 -0500 (CDT) Subject: [OmniOS-discuss] vlan and vnic In-Reply-To: <20160929175217.0e55356b@sleipner.datanom.net> References: <20160929021227.2fe5f04f@sleipner.datanom.net> <20160929175217.0e55356b@sleipner.datanom.net> Message-ID: On Thu, 29 Sep 2016, Michael Rasmussen wrote: > > Given all this I have a hard time to find any pros for vlans compared > to vnics. Obviously this is an Apples vs Orangutans comparison. A vnic has a MAC which uniquely identifies an interface and a vlan is used to separate traffic (like a virtual switch) on the Ethernet. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From mtalbott at lji.org Fri Sep 30 02:21:40 2016 From: mtalbott at lji.org (Michael Talbott) Date: Thu, 29 Sep 2016 19:21:40 -0700 Subject: [OmniOS-discuss] zpool Write Bottlenecks Message-ID: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first. For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity. I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it. Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :( When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :( Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read. I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it. Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool. START=$(date +%s) /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k sync echo $(($(date +%s)-START)) My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware.. Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic? Thanks, Michael From daleg at omniti.com Fri Sep 30 02:46:22 2016 From: daleg at omniti.com (Dale Ghent) Date: Thu, 29 Sep 2016 22:46:22 -0400 Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> Message-ID: Awesome that you're using LX Zones in a way with BeeGFS. A note on your testing methodology, however: http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY > On Sep 29, 2016, at 10:21 PM, Michael Talbott wrote: > > Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first. > > For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity. > > I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it. > > Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :( > > When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :( > > Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read. > > I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it. > > Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool. > > START=$(date +%s) > /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k > sync > echo $(($(date +%s)-START)) > > My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware.. > > Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic? > > > Thanks, > > Michael > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mtalbott at lji.org Fri Sep 30 05:19:38 2016 From: mtalbott at lji.org (Michael Talbott) Date: Thu, 29 Sep 2016 22:19:38 -0700 Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> Message-ID: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other tool for illustrative purposes, I can do that too. It's just so odd that 1 card with a given set of disks attached in a pool provide roughly the same net throughput as 2 cards with 2 sets of disks in said pool. But, for the nay-sayers of dd testing, I'll provide you with this.. Here's an example of using 2 x 11 disk raidz3 vdevs where all 22 drives live on one backplane attached to 1 card with an 8x phy sas connection. And then adding 2 more 11 disk raidz3 vdevs that are connected to that system on a separate card (also using an 8x phy sas link). No compression. Running bonnie++ to saturate the disks while I pull the numbers from iotop. The following is an output of iotop (which uses dtrace for measuring disk io) http://www.brendangregg.com/DTrace/iotop Here's the 22 drive pool (all attached to 1 card): ------------------------------------------------------------------------------------------------------ 2016 Sep 30 00:02:42, load: 3.17, disk_r: 0 KB, disk_w: 3394084 KB UID PID PPID CMD DEVICE MAJ MIN D BYTES 0 7630 0 zpool-datastore sd124 194 7936 W 161812480 0 7630 0 zpool-datastore sd118 194 7552 W 161820672 0 7630 0 zpool-datastore sd127 194 8128 W 161845248 0 7630 0 zpool-datastore sd128 194 8192 W 161845248 0 7630 0 zpool-datastore sd122 194 7808 W 161849344 0 7630 0 zpool-datastore sd119 194 7616 W 161857536 0 7630 0 zpool-datastore sd121 194 7744 W 161857536 0 7630 0 zpool-datastore sd125 194 8000 W 161865728 0 7630 0 zpool-datastore sd123 194 7872 W 161869824 0 7630 0 zpool-datastore sd126 194 8064 W 161873920 0 7630 0 zpool-datastore sd120 194 7680 W 161906688 0 7630 0 zpool-datastore sd136 194 8704 W 165916672 0 7630 0 zpool-datastore sd137 194 8768 W 165916672 0 7630 0 zpool-datastore sd138 194 8832 W 165933056 0 7630 0 zpool-datastore sd135 194 8640 W 165937152 0 7630 0 zpool-datastore sd139 194 8896 W 165941248 0 7630 0 zpool-datastore sd134 194 8576 W 165945344 0 7630 0 zpool-datastore sd130 194 8320 W 165974016 0 7630 0 zpool-datastore sd129 194 8256 W 165978112 0 7630 0 zpool-datastore sd132 194 8448 W 165994496 0 7630 0 zpool-datastore sd133 194 8512 W 165994496 0 7630 0 zpool-datastore sd131 194 8384 W 166006784 ------------------------------------------------------------------------------------------------------ And here's the pool extended with 2 more raidz3s with 2 cards notice it's almost LITERALLY cut in HALF per drive! ------------------------------------------------------------------------------------------------------ 2016 Sep 30 00:01:07, load: 4.59, disk_r: 8 KB, disk_w: 3609852 KB UID PID PPID CMD DEVICE MAJ MIN D BYTES 0 4550 0 zpool-datastore sd133 194 8512 W 76558336 0 4550 0 zpool-datastore sd132 194 8448 W 76566528 0 4550 0 zpool-datastore sd135 194 8640 W 76570624 0 4550 0 zpool-datastore sd134 194 8576 W 76574720 0 4550 0 zpool-datastore sd136 194 8704 W 76582912 0 4550 0 zpool-datastore sd131 194 8384 W 76611584 0 4550 0 zpool-datastore sd130 194 8320 W 76644352 0 4550 0 zpool-datastore sd137 194 8768 W 76648448 0 4550 0 zpool-datastore sd129 194 8256 W 76660736 0 4550 0 zpool-datastore sd138 194 8832 W 76713984 0 4550 0 zpool-datastore sd139 194 8896 W 77369344 0 4550 0 zpool-datastore sd113 194 7232 W 77770752 0 4550 0 zpool-datastore sd115 194 7360 W 77832192 0 4550 0 zpool-datastore sd114 194 7296 W 77836288 0 4550 0 zpool-datastore sd111 194 7104 W 77840384 0 4550 0 zpool-datastore sd112 194 7168 W 77840384 0 4550 0 zpool-datastore sd108 194 6912 W 77844480 0 4550 0 zpool-datastore sd110 194 7040 W 77864960 0 4550 0 zpool-datastore sd116 194 7424 W 77873152 0 4550 0 zpool-datastore sd107 194 6848 W 77914112 0 4550 0 zpool-datastore sd106 194 6784 W 77918208 0 4550 0 zpool-datastore sd109 194 6976 W 77926400 0 4550 0 zpool-datastore sd128 194 8192 W 78938112 0 4550 0 zpool-datastore sd118 194 7552 W 78979072 0 4550 0 zpool-datastore sd125 194 8000 W 78991360 0 4550 0 zpool-datastore sd120 194 7680 W 78999552 0 4550 0 zpool-datastore sd127 194 8128 W 79007744 0 4550 0 zpool-datastore sd126 194 8064 W 79011840 0 4550 0 zpool-datastore sd119 194 7616 W 79020032 0 4550 0 zpool-datastore sd123 194 7872 W 79020032 0 4550 0 zpool-datastore sd122 194 7808 W 79048704 0 4550 0 zpool-datastore sd124 194 7936 W 79056896 0 4550 0 zpool-datastore sd121 194 7744 W 79065088 0 4550 0 zpool-datastore sd105 194 6720 W 82460672 0 4550 0 zpool-datastore sd102 194 6528 W 82468864 0 4550 0 zpool-datastore sd101 194 6464 W 82477056 0 4550 0 zpool-datastore sd104 194 6656 W 82477056 0 4550 0 zpool-datastore sd103 194 6592 W 82481152 0 4550 0 zpool-datastore sd141 194 9024 W 82489344 0 4550 0 zpool-datastore sd99 194 6336 W 82493440 0 4550 0 zpool-datastore sd98 194 6272 W 82501632 0 4550 0 zpool-datastore sd140 194 8960 W 82513920 0 4550 0 zpool-datastore sd97 194 6208 W 82538496 0 4550 0 zpool-datastore sd100 194 6400 W 82542592 ------------------------------------------------------------------------------------------------------ Any thoughts of how to discover and/or overcome the true bottleneck is much appreciated. Thanks Michael > On Sep 29, 2016, at 7:46 PM, Dale Ghent wrote: > > > Awesome that you're using LX Zones in a way with BeeGFS. > > A note on your testing methodology, however: > http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY > >> On Sep 29, 2016, at 10:21 PM, Michael Talbott wrote: >> >> Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first. >> >> For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity. >> >> I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it. >> >> Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :( >> >> When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :( >> >> Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read. >> >> I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it. >> >> Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool. >> >> START=$(date +%s) >> /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k >> sync >> echo $(($(date +%s)-START)) >> >> My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware.. >> >> Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic? >> >> >> Thanks, >> >> Michael >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkateley at kateley.com Fri Sep 30 05:25:41 2016 From: lkateley at kateley.com (Linda Kateley) Date: Fri, 30 Sep 2016 00:25:41 -0500 Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> Message-ID: <0239a6ea-52fb-5e56-1fcf-cddd7c6dcea1@kateley.com> One of the things I do is turn caching off. on a dataset #zfs get primarycache=none dataset. Give me better disk performance lk On 9/30/16 12:19 AM, Michael Talbott wrote: > I'm very aware that dd is not the best tool for measuring disk > performance in most cases. And I know the read throughput number is > way off b/c of zfs caching mechanisms. However it does work in this > case to illustrate the point in my case of a write throttle somewhere > in the system. If anyone needs me to test with some other tool for > illustrative purposes, I can do that too. It's just so odd that 1 card > with a given set of disks attached in a pool provide roughly the same > net throughput as 2 cards with 2 sets of disks in said pool. > > But, for the nay-sayers of dd testing, I'll provide you with this.. > Here's an example of using 2 x 11 disk raidz3 vdevs where all 22 > drives live on one backplane attached to 1 card with an 8x phy sas > connection. And then adding 2 more 11 disk raidz3 vdevs that are > connected to that system on a separate card (also using an 8x phy sas > link). No compression. Running bonnie++ to saturate the disks while I > pull the numbers from iotop. > > The following is an output of iotop (which uses dtrace for measuring > disk io) > http://www.brendangregg.com/DTrace/iotop > > Here's the 22 drive pool (all attached to 1 card): > ------------------------------------------------------------------------------------------------------ > 2016 Sep 30 00:02:42, load: 3.17, disk_r: 0 KB, disk_w: 3394084 KB > > UID PID PPID CMD DEVICE MAJ MIN D BYTES > 0 7630 0 zpool-datastore sd124 194 7936 W 161812480 > 0 7630 0 zpool-datastore sd118 194 7552 W 161820672 > 0 7630 0 zpool-datastore sd127 194 8128 W 161845248 > 0 7630 0 zpool-datastore sd128 194 8192 W 161845248 > 0 7630 0 zpool-datastore sd122 194 7808 W 161849344 > 0 7630 0 zpool-datastore sd119 194 7616 W 161857536 > 0 7630 0 zpool-datastore sd121 194 7744 W 161857536 > 0 7630 0 zpool-datastore sd125 194 8000 W 161865728 > 0 7630 0 zpool-datastore sd123 194 7872 W 161869824 > 0 7630 0 zpool-datastore sd126 194 8064 W 161873920 > 0 7630 0 zpool-datastore sd120 194 7680 W 161906688 > 0 7630 0 zpool-datastore sd136 194 8704 W 165916672 > 0 7630 0 zpool-datastore sd137 194 8768 W 165916672 > 0 7630 0 zpool-datastore sd138 194 8832 W 165933056 > 0 7630 0 zpool-datastore sd135 194 8640 W 165937152 > 0 7630 0 zpool-datastore sd139 194 8896 W 165941248 > 0 7630 0 zpool-datastore sd134 194 8576 W 165945344 > 0 7630 0 zpool-datastore sd130 194 8320 W 165974016 > 0 7630 0 zpool-datastore sd129 194 8256 W 165978112 > 0 7630 0 zpool-datastore sd132 194 8448 W 165994496 > 0 7630 0 zpool-datastore sd133 194 8512 W 165994496 > 0 7630 0 zpool-datastore sd131 194 8384 W 166006784 > > ------------------------------------------------------------------------------------------------------ > > > And here's the pool extended with 2 more raidz3s with 2 cards > notice it's almost LITERALLY cut in HALF per drive! > > > ------------------------------------------------------------------------------------------------------ > > > 2016 Sep 30 00:01:07, load: 4.59, disk_r: 8 KB, disk_w: 3609852 KB > > UID PID PPID CMD DEVICE MAJ MIN D BYTES > 0 4550 0 zpool-datastore sd133 194 8512 W 76558336 > 0 4550 0 zpool-datastore sd132 194 8448 W 76566528 > 0 4550 0 zpool-datastore sd135 194 8640 W 76570624 > 0 4550 0 zpool-datastore sd134 194 8576 W 76574720 > 0 4550 0 zpool-datastore sd136 194 8704 W 76582912 > 0 4550 0 zpool-datastore sd131 194 8384 W 76611584 > 0 4550 0 zpool-datastore sd130 194 8320 W 76644352 > 0 4550 0 zpool-datastore sd137 194 8768 W 76648448 > 0 4550 0 zpool-datastore sd129 194 8256 W 76660736 > 0 4550 0 zpool-datastore sd138 194 8832 W 76713984 > 0 4550 0 zpool-datastore sd139 194 8896 W 77369344 > 0 4550 0 zpool-datastore sd113 194 7232 W 77770752 > 0 4550 0 zpool-datastore sd115 194 7360 W 77832192 > 0 4550 0 zpool-datastore sd114 194 7296 W 77836288 > 0 4550 0 zpool-datastore sd111 194 7104 W 77840384 > 0 4550 0 zpool-datastore sd112 194 7168 W 77840384 > 0 4550 0 zpool-datastore sd108 194 6912 W 77844480 > 0 4550 0 zpool-datastore sd110 194 7040 W 77864960 > 0 4550 0 zpool-datastore sd116 194 7424 W 77873152 > 0 4550 0 zpool-datastore sd107 194 6848 W 77914112 > 0 4550 0 zpool-datastore sd106 194 6784 W 77918208 > 0 4550 0 zpool-datastore sd109 194 6976 W 77926400 > 0 4550 0 zpool-datastore sd128 194 8192 W 78938112 > 0 4550 0 zpool-datastore sd118 194 7552 W 78979072 > 0 4550 0 zpool-datastore sd125 194 8000 W 78991360 > 0 4550 0 zpool-datastore sd120 194 7680 W 78999552 > 0 4550 0 zpool-datastore sd127 194 8128 W 79007744 > 0 4550 0 zpool-datastore sd126 194 8064 W 79011840 > 0 4550 0 zpool-datastore sd119 194 7616 W 79020032 > 0 4550 0 zpool-datastore sd123 194 7872 W 79020032 > 0 4550 0 zpool-datastore sd122 194 7808 W 79048704 > 0 4550 0 zpool-datastore sd124 194 7936 W 79056896 > 0 4550 0 zpool-datastore sd121 194 7744 W 79065088 > 0 4550 0 zpool-datastore sd105 194 6720 W 82460672 > 0 4550 0 zpool-datastore sd102 194 6528 W 82468864 > 0 4550 0 zpool-datastore sd101 194 6464 W 82477056 > 0 4550 0 zpool-datastore sd104 194 6656 W 82477056 > 0 4550 0 zpool-datastore sd103 194 6592 W 82481152 > 0 4550 0 zpool-datastore sd141 194 9024 W 82489344 > 0 4550 0 zpool-datastore sd99 194 6336 W 82493440 > 0 4550 0 zpool-datastore sd98 194 6272 W 82501632 > 0 4550 0 zpool-datastore sd140 194 8960 W 82513920 > 0 4550 0 zpool-datastore sd97 194 6208 W 82538496 > 0 4550 0 zpool-datastore sd100 194 6400 W 82542592 > > ------------------------------------------------------------------------------------------------------ > > > > > Any thoughts of how to discover and/or overcome the true bottleneck is > much appreciated. > > Thanks > > Michael > > > >> On Sep 29, 2016, at 7:46 PM, Dale Ghent > > wrote: >> >> >> Awesome that you're using LX Zones in a way with BeeGFS. >> >> A note on your testing methodology, however: >> http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY >> >> >>> On Sep 29, 2016, at 10:21 PM, Michael Talbott wrote: >>> >>> Hi, I'm trying to find a way to achieve massive write speeds with >>> some decent hardware which will be used for some parallel computing >>> needs (bioinformatics). Eventually if all goes well and my testing >>> succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX >>> zones (THANK YOU LX ZONES!!!) to run some truly massive parallel >>> computing storage happiness, but, I'd like to tune this box as much >>> as possible first. >>> >>> For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six >>> cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x >>> SAS2008 based LSI cards in that box. 1 of those is connected to 8 >>> internal SSDs and the other 2 cards (4 ports) are connected to a 45 >>> bay drive enclosure. And then there's 2 intel x 2 port 10ge cards >>> for connectivity. >>> >>> I've created so many different zpools in different configurations >>> between straight up striped with no redundancy, radiz2, raidz3.. >>> used multipath, non-multipath'd with 8x phy links instead of 4x >>> multipath links, etc, etc in order to find the magic combination for >>> maximum performance, but there's something somewhere capping raw >>> throughput and I can't seem to find it. >>> >>> Now the crazy part is I can for instance, create zpoolA with ~20 >>> drives (via cardA and attached only to backplaneA), create zpoolB >>> with another ~20 drives (via cardB and attached only to backplaneB), >>> and each of them gets the same performance individually (~1GB/s >>> write and 2.5GB/s read). So, my thought is if I destroy zpoolB and >>> attach all those drives to zpoolA via additional vdevs.. It should >>> double the performance or make some sort of significant improvement, >>> but, nope, roughly the same speed.. Then I thought, ok, well maybe >>> it's a slowest vdev sort of thing.. So then I created vdevs such >>> each vdev used half it's drives from backplaneA and the other half >>> from backplaneB.. That would force data distribution between the >>> cards for each vdev and double the speed and get me to 2GB/s write.. >>> but, nope. same deal. 1G/s write and 2.5G read :( >>> >>> When I create the pool from scratch, for each vdev I add I see a >>> linear increase in performance until I hit about 4-5 vdevs.. That's >>> where the performance flatlines and no matter what I do beyond that >>> it just wont go any faster :( >>> >>> Also, if I create a pure SSD pool with cardC, the linear read/write >>> performance of those are hitting the exact same numbers as the >>> others :( Bottom line, no matter what pool configuration I use, no >>> matter what recordsize is set in zfs, I'm always getting capped with >>> roughly 1GB/s write and 2.5GB/s read. >>> >>> I thought maybe there wasn't enough PCIe lanes to run all of those >>> cards at 8x, but, that's not the case, this board can run 6 x 8 lane >>> PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci >>> -vv to make sure of it (since I'm not sure how to view pcie speeds >>> in OmniOS), and sure enough, everything is running with 8x width, so >>> that's not it. >>> >>> Oh, and just fyi, this is my super simple throughput testing script >>> that I run with compression disabled on the tested pool. >>> >>> START=$(date +%s) >>> /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k >>> sync >>> echo $(($(date +%s)-START)) >>> >>> My goal is to find a way to achieve at least 2GB/s write and 4GB/s >>> read which I think is theoretically possbile with this hardware.. >>> >>> Anyone have any ideas of what could be limiting this or how to >>> remedy it? Could it be the mpt_sas driver itself somehow throttling >>> access to all these devices? Or maybe do I need to do some sort of >>> irq-cpu pinning type of magic? >>> >>> >>> Thanks, >>> >>> Michael >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfriesen at simple.dallas.tx.us Fri Sep 30 13:36:02 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 30 Sep 2016 08:36:02 -0500 (CDT) Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> Message-ID: On Thu, 29 Sep 2016, Michael Talbott wrote: > I'm very aware that dd is not the best tool for measuring disk > performance in most cases. And I know the read throughput number is > way off b/c of zfs caching mechanisms. However it does work in this > case to illustrate the point in my case of a write throttle > somewhere in the system. If anyone needs me to test with some other I think that Dale is correct that your benchmark may be only benchmarking 'dd'. The performance of single-threaded 'dd' will be entirely driven by read performance of /dev/zero followed by write latency per operation on writes. In your case, the issue is almost certainly that the write latency is a fixed value (regardless of the number of disks) and so there is a fixed maximum write rate regardless of the performance of the underlying store. If you were to run two 'dd's in parallel, you may see an increase in the write rate. This is the first thing to try. With zfs, a "write" goes into a buffer associated with a transaction group (TXG) which is written to disk based on a maximum elapsed time, or the maximum size of the TXG being hit. The maximum size of the TXG is estimated based on the available total RAM and zfs's guess as to the time to write the TXG (based on throughput), as well as tunables. For a brief time, the writer proces will not be able to write any data at all as the current TXG is finished (resulting in writes) and a new one is started. At the lowest level, the TXG can not be completed until all involved disks have written their data and completed a cache sync operation. The latency of the cache sync is driven by the latency of the slowest disk drive (+HBA) involved. Getting rid of any disks which exhibit abnormally high latency would help with the transaction times. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From mtalbott at lji.org Fri Sep 30 16:46:23 2016 From: mtalbott at lji.org (Michael Talbott) Date: Fri, 30 Sep 2016 09:46:23 -0700 Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> Message-ID: Ah. That explains it. At first I figured the single stream couldn't be the case since a dd from /dev/zero to /dev/null was in excess of 24GB/s. But what you said about TX groups explains it all. I ran several dd's in parallel and now see all of the disks getting fully saturated. Thanks > On Sep 30, 2016, at 6:36 AM, Bob Friesenhahn wrote: > > On Thu, 29 Sep 2016, Michael Talbott wrote: > >> I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other > > I think that Dale is correct that your benchmark may be only benchmarking 'dd'. The performance of single-threaded 'dd' will be entirely driven by read performance of /dev/zero followed by write latency per operation on writes. In your case, the issue is almost certainly that the write latency is a fixed value (regardless of the number of disks) and so there is a fixed maximum write rate regardless of the performance of the underlying store. > > If you were to run two 'dd's in parallel, you may see an increase in the write rate. This is the first thing to try. > > With zfs, a "write" goes into a buffer associated with a transaction group (TXG) which is written to disk based on a maximum elapsed time, or the maximum size of the TXG being hit. The maximum size of the TXG is estimated based on the available total RAM and zfs's guess as to the time to write the TXG (based on throughput), as well as tunables. > > For a brief time, the writer proces will not be able to write any data at all as the current TXG is finished (resulting in writes) and a new one is started. > > At the lowest level, the TXG can not be completed until all involved disks have written their data and completed a cache sync operation. The latency of the cache sync is driven by the latency of the slowest disk drive (+HBA) involved. Getting rid of any disks which exhibit abnormally high latency would help with the transaction times. > > Bob > -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From mtalbott at lji.org Fri Sep 30 21:30:21 2016 From: mtalbott at lji.org (Michael Talbott) Date: Fri, 30 Sep 2016 14:30:21 -0700 Subject: [OmniOS-discuss] extended attributes in LX zone Message-ID: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas? Thanks Michael From daleg at omniti.com Fri Sep 30 22:40:03 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 30 Sep 2016 18:40:03 -0400 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> Message-ID: > On Sep 30, 2016, at 5:30 PM, Michael Talbott wrote: > > So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas? No support for linux get/setxattr syscalls right now in LX brants other than superficial faking to please certain circumstances: https://grok.elemental.org/source/xref/io-lx-public/usr/src/uts/common/brand/lx/syscall/lx_xattr.c#33 /dale From andykinney at advantagecom.net Fri Sep 30 22:52:34 2016 From: andykinney at advantagecom.net (Andrew Kinney) Date: Fri, 30 Sep 2016 15:52:34 -0700 Subject: [OmniOS-discuss] smartctl and SCT ERC on omnios Message-ID: <57EEECB2.5030701@advantagecom.net> Hello, I have a Samsung 845DC Pro SSD that I need to respond more timely during problems with flash cells. We had an incident where it took 2 minutes to fail a cell erase and bring in a spare cell, which is too long for a redundant ZFS log device on a pool with a dataset with property sync=always serving iSCSI volumes. smartctl shows it supports SCT ERC: # smartctl -x -d sat,12 /dev/rdsk/c4t0d0s0 smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: Samsung SSD 845DC PRO 400GB ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) SCT capabilities: (0x003d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. I should be able to read its current SCT ERC settings, but get an error: # smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0 smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org SMART WRITE LOG returns COUNT and LBA_LOW register unchanged SCT (Get) Error Recovery Control command failed I would go read/write the scterc values on a different OS as a workaround, but I've read that Samsung devices do not persist CCTL values across reboots. At present, I don't know what it's set to or if it's being honored. It appears that this is a limitation of smartmontools on Solaris and its derivatives. Any ideas how I can read and set the scterc values on Omnios? -- Andrew From mtalbott at lji.org Fri Sep 30 22:57:26 2016 From: mtalbott at lji.org (Michael Talbott) Date: Fri, 30 Sep 2016 15:57:26 -0700 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> Message-ID: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :( I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to. Any ideas on getting zvols as raw devices in an lx zone? ________________________ Michael Talbott Systems Administrator La Jolla Institute > On Sep 30, 2016, at 3:40 PM, Dale Ghent wrote: > > >> On Sep 30, 2016, at 5:30 PM, Michael Talbott wrote: >> >> So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas? > > No support for linux get/setxattr syscalls right now in LX brants other than superficial faking to please certain circumstances: > > https://grok.elemental.org/source/xref/io-lx-public/usr/src/uts/common/brand/lx/syscall/lx_xattr.c#33 > > /dale -------------- next part -------------- An HTML attachment was scrubbed... URL: