From lists at mcintyreweb.com Mon Dec 1 01:33:00 2014 From: lists at mcintyreweb.com (Hugh McIntyre) Date: Sun, 30 Nov 2014 17:33:00 -0800 Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after bash/openssl patches ... Message-ID: <547BC54C.6060805@mcintyreweb.com> I have a backup system which I'm trying to upgrade from 151006 to 151012 -- I stayed on 151006 for a long time since this is just a backup system, but now want to upgrade because of some stuck zpool IO issues. I'm starting by trying to upgrade from 151006->151008 (based on the release notes because of package signatures) and following the instructions under http://omnios.omniti.com/wiki.php/Upgrade_r151006_r151008. The eventual problem is that when I run "pkg update --be-name=omnios-r151008 entire at 11,5.11-0.151008" it fails because of conflicting package versions. Although I had previously frozen at r151006, the conflicting packages seem to be omnios/shell/bash, omnios/system/library, and omnios/library/security/openssl. Apparently the r151008 version of OmniOS does not work with the bash/heartbleed patches, which I cannot really complain about since r151008 is so old. But is there still a way to perform the upgrade, or am I stuck with needing to do a clean install? Hugh. PS: failing output: # pkg update --be-name=omnios-r151008 entire at 11,5.11-0.151008 Creating Plan / pkg update: No matching version of entire can be installed: Reject: pkg://omnios/entire at 11,5.11-0.151008:20131204T230829Z pkg://omnios/entire at 11,5.11-0.151008:20131205T191306Z Reason: All versions matching 'require' dependency pkg:/compress/p7zip at 9.20.1,5.11-0.151008 are rejected Reject: pkg://omnios/compress/p7zip at 9.20.1,5.11-0.151008:20131204T022414Z Reason: All versions matching 'require' dependency pkg:/shell/bash are rejected Reject: pkg://omnios/shell/bash at 4.2,5.11-0.151002:20120401T175628Z pkg://omnios/shell/bash at 4.2,5.11-0.151004:20121014T211709Z pkg://omnios/shell/bash at 4.2.45,5.11-0.151006:20130506T192652Z pkg://omnios/shell/bash at 4.2.45,5.11-0.151006:20140924T200933Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already installed Reject: pkg://omnios/shell/bash at 4.2.45,5.11-0.151008:20131204T024624Z Reason: Newer version pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already installed Reject: pkg://omnios/shell/bash at 4.2.48,5.11-0.151006:20140924T201441Z pkg://omnios/shell/bash at 4.2.49,5.11-0.151006:20140926T220552Z pkg://omnios/shell/bash at 4.2.49,5.11-0.151006:20140929T154717Z pkg://omnios/shell/bash at 4.2.50,5.11-0.151006:20140929T161532Z pkg://omnios/shell/bash at 4.2.51,5.11-0.151006:20141001T182748Z pkg://omnios/shell/bash at 4.2.52,5.11-0.151006:20141003T191548Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already installed Reject: pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z Reason: All versions matching 'require' dependency pkg:/system/library are rejected Reject: pkg://omnios/system/library at 0.5.11,5.11-0.151002:20120401T180219Z pkg://omnios/system/library at 0.5.11,5.11-0.151002:20120418T225425Z pkg://omnios/system/library at 0.5.11,5.11-0.151004:20121011T224353Z pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130506T161124Z Reason: Excluded by proposed incorporation 'consolidation/osnet/osnet-incorporation' Excluded by proposed incorporation 'incorporation/jeos/illumos-gate' Newer version pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130516T142123Z is already installed Reject: pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130516T142123Z Reason: All versions matching 'require' dependency pkg:/SUNWcs at 0.5.11,5.11-0.151006 are rejected Reject: pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20130506T161045Z Reason: Excluded by proposed incorporation 'consolidation/osnet/osnet-incorporation' Excluded by proposed incorporation 'incorporation/jeos/illumos-gate' Newer version pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20140622T160108Z is already installed Reject: pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20140622T160108Z Reason: Excluded by proposed incorporation 'consolidation/osnet/osnet-incorporation' Excluded by proposed incorporation 'incorporation/jeos/illumos-gate' Reject: pkg://omnios/SUNWcs at 0.5.11,5.11-0.151008:20131204T022241Z Reason: All versions matching 'require' dependency pkg:/install/beadm at 0.5.11,5.11-0.151008 are rejected Reject: pkg://omnios/install/beadm at 0.5.11,5.11-0.151008:20131204T024150Z Reason: All versions matching 'require' dependency pkg:/runtime/python-26 at 2.6.8,5.11-0.151008 are rejected Reject: pkg://omnios/runtime/python-26 at 2.6.8,5.11-0.151008:20131204T024531Z Reason: All versions matching 'require' dependency pkg:/library/security/openssl are rejected Reject: pkg://omnios/library/security/openssl at 1.0.1,5.11-0.151002:20120401T175247Z pkg://omnios/library/security/openssl at 1.0.1.1,5.11-0.151002:20120419T140207Z pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151002:20120510T203327Z pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151004:20121014T200457Z pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T215759Z pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T230413Z pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151006:20130506T185419Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151006:20140110T154549Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151006:20140407T211430Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151006:20140605T131009Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151006:20140807T034412Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Excluded by proposed incorporation 'consolidation/osnet/osnet-incorporation' Excluded by proposed incorporation 'incorporation/jeos/illumos-gate' Reject: pkg://omnios/system/library at 0.5.11,5.11-0.151008:20131204T024825Z Reason: All versions matching 'require' dependency pkg:/SUNWcs at 0.5.11,5.11-0.151008 are rejected Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Reject: pkg://omnios/entire at 11,5.11-0.151008:20131205T195242Z Reason: All versions matching 'require' dependency pkg:/SUNWcs at 0.5.11,5.11-0.151008 are rejected Reject: pkg://omnios/SUNWcs at 0.5.11,5.11-0.151008:20131204T022241Z Reason: All versions matching 'require' dependency pkg:/install/beadm at 0.5.11,5.11-0.151008 are rejected Reject: pkg://omnios/install/beadm at 0.5.11,5.11-0.151008:20131204T024150Z Reason: All versions matching 'require' dependency pkg:/runtime/python-26 at 2.6.8,5.11-0.151008 are rejected Reject: pkg://omnios/runtime/python-26 at 2.6.8,5.11-0.151008:20131204T024531Z Reason: All versions matching 'require' dependency pkg:/library/security/openssl are rejected Reject: pkg://omnios/library/security/openssl at 1.0.1,5.11-0.151002:20120401T175247Z pkg://omnios/library/security/openssl at 1.0.1.1,5.11-0.151002:20120419T140207Z pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151002:20120510T203327Z pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151004:20121014T200457Z pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T215759Z pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T230413Z pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151006:20130506T185419Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151006:20140110T154549Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151006:20140407T211430Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151006:20140605T131009Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151006:20140807T034412Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Reject: pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z Reason: Excluded by proposed incorporation 'incorporation/jeos/omnios-userland' From alexmcwhirter at mojovapes.net Mon Dec 1 15:28:40 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Mon, 1 Dec 2014 10:28:40 -0500 Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI Message-ID: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net> As of now, the network/service/isc-dhcp package uses UDP sockets. The build.sh file is using the options ??enable-use-sockets? and ??enable-ipv4-pktinfo?. More details on this workaround can be found here. https://kb.isc.org/article/AA-01040/0/Building-ISC-DHCP-on-Solaris-11.html The problem with using UDP sockets is that unicast clients cannot correctly obtain an IP without a relay, which really isn?t ideal. I?ve sent this patch to the isc-dhcp team, but i figured it might be found more quickly useful here. I?ve created two patches for isc-dhcp that allow dhcpd to use DLPI with VNIC?s and other vanity named interfaces. the modified files are ?./configure.ac? and ?./common/dlpi.c? Below is the diff info. --- /Users/alexmcwhirter/Downloads/dhcp-4.3.1 2/common/dlpi.c 2014-07-29 18:41:04.000000000 -0400 +++ /Users/alexmcwhirter/Desktop/dhcp-4.3.1/common/dlpi.c 2014-11-30 20:48:39.000000000 -0500 @@ -130,7 +130,14 @@ #define DLPI_MAXDLBUF 8192 /* Buffer size */ #define DLPI_MAXDLADDR 1024 /* Max address size */ -#define DLPI_DEVDIR "/dev/" /* Device directory */ + +/* Solaris 11 / Illumos 11 uses vanity names for ethernet devices + * and places them in /dev/net */ +#ifdef __SOLARIS_2_11 +# define DLPI_DEVDIR "/dev/net/" /* Device directory */ +#else +# define DLPI_DEVDIR "/dev/" /* Device directory */ +#endif static int dlpiopen(const char *ifname); static int dlpiunit (char *ifname); @@ -794,9 +801,13 @@ ep = cp = ifname; while (*ep) ep++; - /* And back up to the first digit (unit number) */ - while ((*(ep - 1) >= '0' && *(ep - 1) <= '9') || *(ep - 1) == ':') - ep--; + /* Solaris 11 / Illumos 11 should not have the number removed from the + * interface name */ + if (__SOLARIS_2_11 != 1) { + /* And back up to the first digit (unit number) */ + while ((*(ep - 1) >= '0' && *(ep - 1) <= '9') || *(ep - 1) == ':') + ep--; + } /* Copy everything up to the unit number */ while (cp < ep) { --- /Users/alexmcwhirter/Downloads/dhcp-4.3.1 2/configure.ac 2014-08-06 18:35:33.000000000 -0400 +++ /Users/alexmcwhirter/Desktop/dhcp-4.3.1/configure.ac 2014-11-30 20:50:11.000000000 -0500 @@ -13,6 +13,13 @@ AC_CANONICAL_HOST +# Solaris 11 / Illumos 11 is identified as solaris2.11 +AM_CONDITIONAL([SOLARIS_2_11], [test x$host_os = xsolaris2.11]) + +# If host_os is solaris2.11 then define __SOLARIS_2_11 +AM_COND_IF([SOLARIS_2_11],[AC_DEFINE([__SOLARIS_2_11], [1], + [Define if host_os is solaris2.11])]) + # We want to turn on warnings if we are using gcc and the user did # not specify CFLAGS. The autoconf check for the C compiler sets the # CFLAGS if gcc is used, so we will save it before we run that check. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Dec 1 15:57:31 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 1 Dec 2014 10:57:31 -0500 Subject: [OmniOS-discuss] Release 12 - KVM poor network performance In-Reply-To: References: Message-ID: <5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com> One other question, which I should've asked earlier: You're using OmniOS as the KVM host, and your performance drops are on Windows guests, right? Thanks, Dan From danmcd at omniti.com Mon Dec 1 16:12:44 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 1 Dec 2014 11:12:44 -0500 Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after bash/openssl patches ... In-Reply-To: <547BC54C.6060805@mcintyreweb.com> References: <547BC54C.6060805@mcintyreweb.com> Message-ID: <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com> You will likely have better luck directly upgrading to 012. You can try using a spare boot environment (BE) to make it less damaging going forward. beadm create 012-be -e 006-be beadm mount 012-be /mnt pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/stable -g http://pkg.omniti.com/omnios/r151012 omnios pkg -R /mnt update beadm umount /mnt beadm activate 012-be We didn't update 008 with the heartbleed fixes because 008 was discontinued. Hope this helps, Dan From danmcd at omniti.com Mon Dec 1 16:19:50 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 1 Dec 2014 11:19:50 -0500 Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI In-Reply-To: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net> References: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net> Message-ID: > On Dec 1, 2014, at 10:28 AM, Alex McWhirter wrote: > > As of now, the network/service/isc-dhcp package uses UDP sockets. The build.sh file is using the options ??enable-use-sockets? and ??enable-ipv4-pktinfo?. More details on this workaround can be found here. https://kb.isc.org/article/AA-01040/0/Building-ISC-DHCP-on-Solaris-11.html Got it. > The problem with using UDP sockets is that unicast clients cannot correctly obtain an IP without a relay, which really isn?t ideal. I?ve sent this patch to the isc-dhcp team, but i figured it might be found more quickly useful here. > > I?ve created two patches for isc-dhcp that allow dhcpd to use DLPI with VNIC?s and other vanity named interfaces. the modified files are ?./configure.ac? and ?./common/dlpi.c? Below is the diff info. Thank you! BTW, it's just "illumos" not "illumos 11", but that's a tiny nit. My question for you is this: Are you suggesting OmniOS build ISC DHCP using the DLPI method/configure-flags instead, provided your fixes are in place? If so, I have a followup question: What testing have you performed? I'm not saying no, but I just want to know what all you've tested alongside these fixes. Thanks, I really appreciate the contribution! Dan From henson at acm.org Mon Dec 1 22:36:03 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 1 Dec 2014 14:36:03 -0800 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <20141201055654.2a3e23f8@punda-mlia> References: <20141128191703.00a9df99@punda-mlia> <20141130025557.GI29549@bender.unx.csupomona.edu> <20141201055654.2a3e23f8@punda-mlia> Message-ID: <234301d00db7$32bdcc20$98396460$@acm.org> > From: Michael Mounteney > Sent: Sunday, November 30, 2014 11:57 AM > > Port {something other than 22} > PasswordAuthentication no > PermitEmptyPasswords no > UsePAM no > PrintMotd no > PrintLastLog no > UsePrivilegeSeparation sandbox # Default for new installations. > ClientAliveInterval 61 > Subsystem sftp /usr/lib/misc/sftp-server > AcceptEnv LANG LC_* Hmm, I restarted my Gentoo sshd with that exact config (other than port=22), and had no problems connecting to it from omnios, using diffie-hellman-group-exchange-sha1 for the key exchange. I'm running net-misc/openssh-6.6_p1-r1... I've got no idea why yours is behaving differently. What use flags do you have set? Mine are: [ebuild R ] net-misc/openssh-6.6_p1-r1 USE="X hpn kerberos pam -X509 -bindist -ldap -ldns -libedit (-selinux) -skey -static -tcpd" 0 kB From gate03 at landcroft.co.uk Mon Dec 1 23:41:49 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 2 Dec 2014 09:41:49 +1000 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <234301d00db7$32bdcc20$98396460$@acm.org> References: <20141128191703.00a9df99@punda-mlia> <20141130025557.GI29549@bender.unx.csupomona.edu> <20141201055654.2a3e23f8@punda-mlia> <234301d00db7$32bdcc20$98396460$@acm.org> Message-ID: <20141202094149.43d9f5d9@punda-mlia> On Mon, 1 Dec 2014 14:36:03 -0800 "Paul B. Henson" wrote: > I've got no idea why yours is behaving differently. What use flags do > you have set? Mine are: > > [ebuild R ] net-misc/openssh-6.6_p1-r1 USE="X hpn kerberos pam > -X509 -bindist -ldap -ldns -libedit (-selinux) -skey -static -tcpd" 0 > kB My use flags are equery u openssh [ Legend : U - final flag setting for installation] [ : I - package is installed with flag ] [ Colors : set, unset ] * Found these USE flags for net-misc/openssh-6.7_p1-r3: U I + + X : Add support for X11 - - X509 : Adds support for X.509 certificate authentication - - bindist : Disable EC/RC5 algorithms in OpenSSL for patent reasons. + + hpn : Enable high performance ssh - - kerberos : Add kerberos support + + ldap : Add support for storing SSH public keys in LDAP - - ldns : Use LDNS for DNSSEC/SSHFP validation. - - libedit : Use the libedit library (replacement for readline) + + pam : Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip + + pie : Build programs as Position Independent Executables (a security hardening technique) - - sctp : Support for Stream Control Transmission Protocol - - skey : Enable S/Key (Single use password) authentication support - - static : !!do not set this during bootstrap!! Causes binaries to be statically linked instead of dynamically but the reason for the problem is that the older algorithms have been removed from openssh-6.7. I just downgraded to 6.6 on one machine and once again was able to ssh in from OmniOS. Upgrade to 6.7 again and the common kex problem re-arose. Michael. From gearboxes at outlook.com Tue Dec 2 02:54:28 2014 From: gearboxes at outlook.com (Machine Man) Date: Mon, 1 Dec 2014 21:54:28 -0500 Subject: [OmniOS-discuss] Release 12 - KVM poor network performance In-Reply-To: <5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com> References: , <5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com> Message-ID: Correct. The copy would also stall quickly and pool utilization would be at 10%. I have not tested NFS for example, but I am sure that would have been discussed if that was happening. I did have to do zfs send/recv between two nodes on the same subnet and did not notice any performance issue with one node was on r12. Both currently are back to running r10. I will build a test system with a fresh install of the latest version and see when time permits. Thanks, > Subject: Re: [OmniOS-discuss] Release 12 - KVM poor network performance > From: danmcd at omniti.com > Date: Mon, 1 Dec 2014 10:57:31 -0500 > CC: omnios-discuss at lists.omniti.com > To: gearboxes at outlook.com > > One other question, which I should've asked earlier: You're using OmniOS as the KVM host, and your performance drops are on Windows guests, right? > > Thanks, > Dan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at mcintyreweb.com Tue Dec 2 07:38:52 2014 From: lists at mcintyreweb.com (Hugh McIntyre) Date: Mon, 01 Dec 2014 23:38:52 -0800 Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after bash/openssl patches ... In-Reply-To: <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com> References: <547BC54C.6060805@mcintyreweb.com> <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com> Message-ID: <547D6C8C.4020307@mcintyreweb.com> Thanks Dan! This seems to have worked very painlessly, modulo needing to use "-G http://pkg.omniti.com/omnios/release" instead of "-G http://pkg.omniti.com/omnios/stable". Hugh. On 12/1/14 8:12 AM, Dan McDonald wrote: > You will likely have better luck directly upgrading to 012. > > You can try using a spare boot environment (BE) to make it less damaging going forward. > > beadm create 012-be -e 006-be > > beadm mount 012-be /mnt > > pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/stable -g http://pkg.omniti.com/omnios/r151012 omnios > > pkg -R /mnt update > > beadm umount /mnt > > beadm activate 012-be > > We didn't update 008 with the heartbleed fixes because 008 was discontinued. > > Hope this helps, > Dan > From piiv at zhaw.ch Tue Dec 2 11:05:10 2014 From: piiv at zhaw.ch (Vincenzo Pii) Date: Tue, 2 Dec 2014 12:05:10 +0100 Subject: [OmniOS-discuss] OmniOS Packaging: override installation PREFIX Message-ID: I am trying to package something on OmniOS following the instructions here: http://omnios.omniti.com/wiki.php/PackagingForOmniOS. The PREFIX in the config.sh file is set to /usr/local: # Default prefix for packages (may be overridden) PREFIX=/usr/local In my build.sh I want to set a different prefix (for this specific package only), so I am doing something like export PREFIX=/opt export CONFIGURE_OPTS="--prefix=$PREFIX" However, this doesn't seem to work, as files are still installed under /usr/local. What is the correct way to override the global PREFIX? Thanks, Vincenzo. -- Vincenzo Pii Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) blog.zhaw.ch/icclab -------------- next part -------------- An HTML attachment was scrubbed... URL: From lotheac at iki.fi Tue Dec 2 11:30:25 2014 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Tue, 2 Dec 2014 13:30:25 +0200 Subject: [OmniOS-discuss] OmniOS Packaging: override installation PREFIX In-Reply-To: References: Message-ID: <20141202113025.GB11049@gutsman.lotheac.fi> On Tue, Dec 02 2014 12:05:10 +0100, Vincenzo Pii wrote: > In my build.sh I want to set a different prefix (for this specific package > only), so I am doing something like > > export PREFIX=/opt > export CONFIGURE_OPTS="--prefix=$PREFIX" > > However, this doesn't seem to work, as files are still installed under > /usr/local. > > What is the correct way to override the global PREFIX? Try this: PREFIX=/opt reset_configure_opts -- Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet From piiv at zhaw.ch Tue Dec 2 12:39:38 2014 From: piiv at zhaw.ch (Vincenzo Pii) Date: Tue, 2 Dec 2014 13:39:38 +0100 Subject: [OmniOS-discuss] OmniOS Packaging: override installation PREFIX In-Reply-To: <03c26df9fdba46c18c505889f8ca095c@SRV-MAIL-001.zhaw.ch> References: <03c26df9fdba46c18c505889f8ca095c@SRV-MAIL-001.zhaw.ch> Message-ID: Many thanks Lauri, that worked (no need for the CONFIGURE_OPTS)! 2014-12-02 12:30 GMT+01:00 Lauri Tirkkonen : > On Tue, Dec 02 2014 12:05:10 +0100, Vincenzo Pii wrote: > > In my build.sh I want to set a different prefix (for this specific > package > > only), so I am doing something like > > > > export PREFIX=/opt > > export CONFIGURE_OPTS="--prefix=$PREFIX" > > > > However, this doesn't seem to work, as files are still installed under > > /usr/local. > > > > What is the correct way to override the global PREFIX? > > Try this: > > PREFIX=/opt > reset_configure_opts > > -- > Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet > -- Vincenzo Pii Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) blog.zhaw.ch/icclab -------------- next part -------------- An HTML attachment was scrubbed... URL: From filip.marvan at aira.cz Tue Dec 2 14:31:55 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Tue, 2 Dec 2014 15:31:55 +0100 Subject: [OmniOS-discuss] zpool import strange informations Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> Hi, I have one storage server on OmniOS 12 with one data pool: pool: raid_10 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM raid_10 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 c3t5d0 ONLINE 0 0 0 logs c3t0d0 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c3t1d0s0 ONLINE 0 0 0 errors: No known data errors There are no other disks in this storage server. Bud if I write command zpool import, I can see: pool: tank id: 4916680739993977334 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://illumos.org/msg/ZFS-8000-EY config: tank UNAVAIL insufficient replicas raidz2-1 UNAVAIL insufficient replicas /dev/label/mfront8 UNAVAIL cannot open /dev/label/mfront9 UNAVAIL cannot open /dev/label/mfront10 UNAVAIL cannot open c3t2d0p0 ONLINE /dev/label/mfront12 UNAVAIL cannot open /dev/label/mfront13 UNAVAIL cannot open /dev/label/mfront14 UNAVAIL cannot open c3t5d0 ONLINE I never created such "tank" pool. Where are stored informations about this (un)importable pool, how can I delete them? Why c3t5d0 seems to be as part of two different pools at the same time? Thank you for any help. Filip Marvan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6247 bytes Desc: not available URL: From cal-s at blue-bolt.com Tue Dec 2 14:47:50 2014 From: cal-s at blue-bolt.com (Cal Sawyer) Date: Tue, 02 Dec 2014 14:47:50 +0000 Subject: [OmniOS-discuss] Upgrading from OI to OmniOS Message-ID: <547DD116.7040401@blue-bolt.com> Hi I have a server hosting 65TB of RAIDZ2 storage that's currently running OI (oi_151a8), which i really want to get off of given that OI has bitten the dust support-wise. I found one brief discussion on the topic but it wasn't terribly detailed and was missing critical steps like configuration protection/retention. I'd like to preserve network and esp LDAP (client) settings as they were kind of a pain to establish in the first instance for quick reapplication. If anyone has or is interested in cobbling together a checklist of steps to upgrade OI to OmniOS that takes configuration into account, i'd very much appreciate seeing it. thank you - cal sawyer From zmalone at omniti.com Tue Dec 2 14:57:59 2014 From: zmalone at omniti.com (Zach Malone) Date: Tue, 2 Dec 2014 09:57:59 -0500 Subject: [OmniOS-discuss] Postgres on Stable In-Reply-To: References: <20141128191821.43382469@punda-mlia> <20141129071835.1daa0799@punda-mlia> Message-ID: A vagrant box for r151012 is now listed on the download page. --Zach On Fri, Nov 28, 2014 at 6:37 PM, Zach Malone wrote: >>> No suitable version of installed package pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z found >>> Reject: pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z >>> Reason: Excluded by proposed incorporation 'omniti/database/postgresql-934' >>> This version is excluded by installed incorporation pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z >> >> I hadn't realized we incorporated against the OS for postgres, I'll >> spin a new postgresql-934 or 935 on a 012 machine when I get a chance. >> It looks like 012 doesn't have a Vagrant box published yet either, >> which will make this take a little longer. > > I built postgresql-935 for you on a r151012 system, and published it > to the omniti-ms repo. Want to give it a shot? Past installing it, I > have not tried it on a production system (I'm planning on moving some > systems to r151012 this month), and I have not rolled any of the other > postgres libraries or modules that we publish for 9.2. > > I also noticed that we're missing a Vagrant box for r151012, I'll see > if I can get that up next week. > --Zach From a.angelov at gmail.com Tue Dec 2 14:57:44 2014 From: a.angelov at gmail.com (Angel Angelov) Date: Tue, 2 Dec 2014 16:57:44 +0200 Subject: [OmniOS-discuss] PCIe dedicated device for ZIL Message-ID: Hi guys, does anyone played with some of these PCIe devices on the market as a cheaper ZIL dedicated device instead of this one for example that would work with OmniOS out of the box? Here's one of the products I have found in the Internet googling around but it's way too big for the purpose: http://www.kingspec.com/en/product_xx187.html I am trying to find a good price/performance balance when full SSD storage based pool is in use. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zmalone at omniti.com Tue Dec 2 15:12:16 2014 From: zmalone at omniti.com (Zach Malone) Date: Tue, 2 Dec 2014 10:12:16 -0500 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> Message-ID: If I was to guess, c3t2d0p0 was once a part of another pool called "tank", and when you do an import, it's attempting to reassemble the rest of tank, and sees that a drive named c3t5d0 used to be in the pool it knew as tank. Tank is a pretty standard pool name, so I'm guessing the drive used to be in another system, or the system itself used to be running a different OS. --Zach On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan wrote: > Hi, > > > > I have one storage server on OmniOS 12 with one data pool: > > > > pool: raid_10 > > state: ONLINE > > scan: none requested > > config: > > > > NAME STATE READ WRITE CKSUM > > raid_10 ONLINE 0 0 0 > > mirror-0 ONLINE 0 0 0 > > c3t2d0 ONLINE 0 0 0 > > c3t3d0 ONLINE 0 0 0 > > mirror-1 ONLINE 0 0 0 > > c3t4d0 ONLINE 0 0 0 > > c3t5d0 ONLINE 0 0 0 > > logs > > c3t0d0 ONLINE 0 0 0 > > > > errors: No known data errors > > > > pool: rpool > > state: ONLINE > > scan: none requested > > config: > > > > NAME STATE READ WRITE CKSUM > > rpool ONLINE 0 0 0 > > c3t1d0s0 ONLINE 0 0 0 > > > > errors: No known data errors > > > > There are no other disks in this storage server. Bud if I write command > zpool import, I can see: > > pool: tank > > id: 4916680739993977334 > > state: UNAVAIL > > status: The pool was last accessed by another system. > > action: The pool cannot be imported due to damaged devices or data. > > see: http://illumos.org/msg/ZFS-8000-EY > > config: > > > > tank UNAVAIL insufficient replicas > > raidz2-1 UNAVAIL insufficient replicas > > /dev/label/mfront8 UNAVAIL cannot open > > /dev/label/mfront9 UNAVAIL cannot open > > /dev/label/mfront10 UNAVAIL cannot open > > c3t2d0p0 ONLINE > > /dev/label/mfront12 UNAVAIL cannot open > > /dev/label/mfront13 UNAVAIL cannot open > > /dev/label/mfront14 UNAVAIL cannot open > > c3t5d0 ONLINE > > > > I never created such "tank" pool. Where are stored informations about this > (un)importable pool, how can I delete them? Why c3t5d0 seems to be as part > of two different pools at the same time? > > > > Thank you for any help. > > Filip Marvan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From danmcd at omniti.com Tue Dec 2 15:19:08 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 2 Dec 2014 10:19:08 -0500 Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after bash/openssl patches ... In-Reply-To: <547D6C8C.4020307@mcintyreweb.com> References: <547BC54C.6060805@mcintyreweb.com> <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com> <547D6C8C.4020307@mcintyreweb.com> Message-ID: <73038663-F1B9-4033-B800-D2643E8BB1B9@omniti.com> > On Dec 2, 2014, at 2:38 AM, Hugh McIntyre wrote: > > Thanks Dan! > > This seems to have worked very painlessly, modulo needing to use "-G http://pkg.omniti.com/omnios/release" instead of "-G http://pkg.omniti.com/omnios/stable". The other solution would've been for me to update 008's appropriate packages to stop pkg(5) from seizing up. I'm glad the 006->012 path works, though, as the next release is also the next LTS, so we MUST be able to support that. Thanks, Dan From filip.marvan at aira.cz Tue Dec 2 15:20:25 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Tue, 2 Dec 2014 16:20:25 +0100 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> Yes, you are probably right. But I'm thinking, why that information wasn't overwriten when I add whole c3t2d0 to a new mirror? Is it possible to use c3t2d0 in one pool, and c3t2d0p0 in another? Thank you, Filip -----Original Message----- From: Zach Malone [mailto:zmalone at omniti.com] Sent: Tuesday, December 02, 2014 4:12 PM To: Filip Marvan Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] zpool import strange informations If I was to guess, c3t2d0p0 was once a part of another pool called "tank", and when you do an import, it's attempting to reassemble the rest of tank, and sees that a drive named c3t5d0 used to be in the pool it knew as tank. Tank is a pretty standard pool name, so I'm guessing the drive used to be in another system, or the system itself used to be running a different OS. --Zach On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan wrote: > Hi, > > > > I have one storage server on OmniOS 12 with one data pool: > > > > pool: raid_10 > > state: ONLINE > > scan: none requested > > config: > > > > NAME STATE READ WRITE CKSUM > > raid_10 ONLINE 0 0 0 > > mirror-0 ONLINE 0 0 0 > > c3t2d0 ONLINE 0 0 0 > > c3t3d0 ONLINE 0 0 0 > > mirror-1 ONLINE 0 0 0 > > c3t4d0 ONLINE 0 0 0 > > c3t5d0 ONLINE 0 0 0 > > logs > > c3t0d0 ONLINE 0 0 0 > > > > errors: No known data errors > > > > pool: rpool > > state: ONLINE > > scan: none requested > > config: > > > > NAME STATE READ WRITE CKSUM > > rpool ONLINE 0 0 0 > > c3t1d0s0 ONLINE 0 0 0 > > > > errors: No known data errors > > > > There are no other disks in this storage server. Bud if I write > command zpool import, I can see: > > pool: tank > > id: 4916680739993977334 > > state: UNAVAIL > > status: The pool was last accessed by another system. > > action: The pool cannot be imported due to damaged devices or data. > > see: http://illumos.org/msg/ZFS-8000-EY > > config: > > > > tank UNAVAIL insufficient replicas > > raidz2-1 UNAVAIL insufficient replicas > > /dev/label/mfront8 UNAVAIL cannot open > > /dev/label/mfront9 UNAVAIL cannot open > > /dev/label/mfront10 UNAVAIL cannot open > > c3t2d0p0 ONLINE > > /dev/label/mfront12 UNAVAIL cannot open > > /dev/label/mfront13 UNAVAIL cannot open > > /dev/label/mfront14 UNAVAIL cannot open > > c3t5d0 ONLINE > > > > I never created such "tank" pool. Where are stored informations about > this (un)importable pool, how can I delete them? Why c3t5d0 seems to > be as part of two different pools at the same time? > > > > Thank you for any help. > > Filip Marvan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6247 bytes Desc: not available URL: From danmcd at omniti.com Tue Dec 2 15:35:36 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 2 Dec 2014 10:35:36 -0500 Subject: [OmniOS-discuss] Release 12 KVM poor network performance In-Reply-To: References: Message-ID: <34BE292D-F302-4E1C-BCAA-6FD226AABA70@omniti.com> > On Nov 29, 2014, at 3:42 PM, Machine Man wrote: > > Experience slow network performance after upgrading. > Copying from another windows system (physical machine) it drops form the 85MB/s on R10 to between 9 and 14MB/s. Two systems experience the exact same symptom. Both make use of igb driver. > Copy tests are done between windows 8.1 and Windows 2012R2 VM. > Changing to virtio for nic has no impact other than causing kvm to crash after copying ~400MB. > > Any ideas? We are behind the illumos-kvm-cmd distribution because it requires changes in illumos that Joyent hasn't pushed upstream just yet (the "vnd" project). I'm hoping to address this for '014, but I have other 014-related things to do first. Sorry, Dan From richard.elling at richardelling.com Tue Dec 2 17:38:23 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Tue, 2 Dec 2014 09:38:23 -0800 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> Message-ID: <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com> On Dec 2, 2014, at 7:20 AM, Filip Marvan wrote: > Yes, you are probably right. But I'm thinking, why that information wasn't > overwriten when I add whole c3t2d0 to a new mirror? p0 is an fdisk partition for the whole disk. c3t2d0 is really a shorthand notation for c3t2d0s0. c3t2d0 is also in a fdisk partition of type "Solaris2" So you'll need to check your partitioning and see where exactly the blocks are allocated to solve the mystery. > Is it possible to use > c3t2d0 in one pool, and c3t2d0p0 in another? In general, no. The p0 fdisk partition overlaps whatever fdisk partition is being used for c3t2d0 -- an invitation to data corruption. In general, it is considered bad practice to use p0 (or any p*) for ZFS pool creation. There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you do this consistently, then you'll never have partition nightmares. HTH -- richard > > Thank you, > Filip > > -----Original Message----- > From: Zach Malone [mailto:zmalone at omniti.com] > Sent: Tuesday, December 02, 2014 4:12 PM > To: Filip Marvan > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] zpool import strange informations > > If I was to guess, c3t2d0p0 was once a part of another pool called "tank", > and when you do an import, it's attempting to reassemble the rest of tank, and > sees that a drive named c3t5d0 used to be in the pool it knew as tank. > > Tank is a pretty standard pool name, so I'm guessing the drive used to be in > another system, or the system itself used to be running a different OS. > --Zach > > On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan wrote: >> Hi, >> >> >> >> I have one storage server on OmniOS 12 with one data pool: >> >> >> >> pool: raid_10 >> >> state: ONLINE >> >> scan: none requested >> >> config: >> >> >> >> NAME STATE READ WRITE CKSUM >> >> raid_10 ONLINE 0 0 0 >> >> mirror-0 ONLINE 0 0 0 >> >> c3t2d0 ONLINE 0 0 0 >> >> c3t3d0 ONLINE 0 0 0 >> >> mirror-1 ONLINE 0 0 0 >> >> c3t4d0 ONLINE 0 0 0 >> >> c3t5d0 ONLINE 0 0 0 >> >> logs >> >> c3t0d0 ONLINE 0 0 0 >> >> >> >> errors: No known data errors >> >> >> >> pool: rpool >> >> state: ONLINE >> >> scan: none requested >> >> config: >> >> >> >> NAME STATE READ WRITE CKSUM >> >> rpool ONLINE 0 0 0 >> >> c3t1d0s0 ONLINE 0 0 0 >> >> >> >> errors: No known data errors >> >> >> >> There are no other disks in this storage server. Bud if I write >> command zpool import, I can see: >> >> pool: tank >> >> id: 4916680739993977334 >> >> state: UNAVAIL >> >> status: The pool was last accessed by another system. >> >> action: The pool cannot be imported due to damaged devices or data. >> >> see: http://illumos.org/msg/ZFS-8000-EY >> >> config: >> >> >> >> tank UNAVAIL insufficient replicas >> >> raidz2-1 UNAVAIL insufficient replicas >> >> /dev/label/mfront8 UNAVAIL cannot open >> >> /dev/label/mfront9 UNAVAIL cannot open >> >> /dev/label/mfront10 UNAVAIL cannot open >> >> c3t2d0p0 ONLINE >> >> /dev/label/mfront12 UNAVAIL cannot open >> >> /dev/label/mfront13 UNAVAIL cannot open >> >> /dev/label/mfront14 UNAVAIL cannot open >> >> c3t5d0 ONLINE >> >> >> >> I never created such "tank" pool. Where are stored informations about >> this (un)importable pool, how can I delete them? Why c3t5d0 seems to >> be as part of two different pools at the same time? >> >> >> >> Thank you for any help. >> >> Filip Marvan >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From sim.ple at live.nl Tue Dec 2 23:17:24 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 00:17:24 +0100 Subject: [OmniOS-discuss] test omnios on OI machine Message-ID: Hi, I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. The machine has a mirrored rpool created on two sas disks which are connected to two seperate LSI SAS9207-4i4e HBA's . Each sas disk is connected to its own LSI HBA. When I boot the machine into OI, the rpool mirror is reported as expected. (Running like this for over a year) When I boot it into Omnios the rpool mirror is degraded because Omnios cannot see one of the HBA's. Lsiutil reports only one hba , sasinfo hba -v , same thing echo "::mptsas -t" | mdb -k = ditto So as far as I can see omnios could not detect one of the hba's I have cleaned out path_to_inst, done a reconfiguration reboot and devfsadm -Cv but no succes. I have also installed omnios on OI systems with only one LSI SAS9207-4i4e with no problem. Can someone tell me if it is known if omnios has problems with a dual hba system or how to force omnios to detect the other HBA ? We have also tested with multipath disabled and enabled. Rgds, R -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Dec 2 23:35:42 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 2 Dec 2014 18:35:42 -0500 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: Message-ID: <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> > On Dec 2, 2014, at 6:17 PM, Randy S wrote: > > Hi, > > I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. Try upgrading r10 to r12 and see if the same thing happens please. Next, utter "dmesg | grep mpt_sas" to see if there are any complaints from the kernel. Dan From sim.ple at live.nl Wed Dec 3 00:01:33 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 01:01:33 +0100 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: , <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>, Message-ID: Sorry should have send this to the list. Subject: RE: [OmniOS-discuss] test omnios on OI machine Date: Wed, 3 Dec 2014 00:59:33 +0100 Hi Dan, It's been a while... I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. forgot to mention, I did do an fmdump -ev and saw Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 I guess this is because the one controller cannot be found. dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online Thanks, R > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 18:35:42 -0500 > CC: omnios-discuss at lists.omniti.com > To: sim.ple at live.nl > > > > On Dec 2, 2014, at 6:17 PM, Randy S wrote: > > > > Hi, > > > > I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. > > Try upgrading r10 to r12 and see if the same thing happens please. > > Next, utter "dmesg | grep mpt_sas" to see if there are any complaints from the kernel. > > Dan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sim.ple at live.nl Wed Dec 3 00:45:12 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com> References: <,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>> , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com> Message-ID: Hi, was indeed intended for the list. dmesg | grep mptsas gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From henson at acm.org Wed Dec 3 02:03:12 2014 From: henson at acm.org (Paul B. Henson) Date: Tue, 2 Dec 2014 18:03:12 -0800 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <4fcd8a9de3d5ab64e565365c78cb3105@kivalo.at> References: <20141128191703.00a9df99@punda-mlia> <4fcd8a9de3d5ab64e565365c78cb3105@kivalo.at> Message-ID: <306f01d00e9d$4d16ab50$e74401f0$@acm.org> > From: Christian Kivalo > Sent: Friday, November 28, 2014 1:55 AM > > I would be interestent in replacing the default ssh package with OpenSSH on > the OmniOS installation, but the ssh package seems to be bound to @entire so > this looks to me as if this is not as easy as uninstall/install to replace the ssh > daemon/client... I'm definitely interested in that as well, although I haven't actually had the time to try and run into the issue you found :). Any official omniti perspective on making openssh the default and legacy sun ssh the alternative option? Thanks. From henson at acm.org Wed Dec 3 02:05:54 2014 From: henson at acm.org (Paul B. Henson) Date: Tue, 2 Dec 2014 18:05:54 -0800 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <20141202094149.43d9f5d9@punda-mlia> References: <20141128191703.00a9df99@punda-mlia> <20141130025557.GI29549@bender.unx.csupomona.edu> <20141201055654.2a3e23f8@punda-mlia> <234301d00db7$32bdcc20$98396460$@acm.org> <20141202094149.43d9f5d9@punda-mlia> Message-ID: <314901d00e9d$ade49f00$09addd00$@acm.org> > From: Michael Mounteney > Sent: Monday, December 01, 2014 3:42 PM > > but the reason for the problem is that the older algorithms have been > removed from openssh-6.7. I just downgraded to 6.6 on one machine and > once again was able to ssh in from OmniOS. Upgrade to 6.7 again and the > common kex problem re-arose. Ah, that must have gone stable right after I did my last update. Guess I will have this issue to look forward to for our next update cycle 8-/... From gate03 at landcroft.co.uk Wed Dec 3 03:16:57 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 3 Dec 2014 13:16:57 +1000 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <314901d00e9d$ade49f00$09addd00$@acm.org> References: <20141128191703.00a9df99@punda-mlia> <20141130025557.GI29549@bender.unx.csupomona.edu> <20141201055654.2a3e23f8@punda-mlia> <234301d00db7$32bdcc20$98396460$@acm.org> <20141202094149.43d9f5d9@punda-mlia> <314901d00e9d$ade49f00$09addd00$@acm.org> Message-ID: <20141203131657.11875145@punda-mlia> On Tue, 2 Dec 2014 18:05:54 -0800 "Paul B. Henson" wrote: > Ah, that must have gone stable right after I did my last update. > Guess I will have this issue to look forward to for our next update > cycle 8-/... I did consider holding on to 6.6 but there must be a good reason for the issuance of 6.7. As I said originally, I'd rather go up with OmniOS than down with Gentoo. Michael. From henson at acm.org Wed Dec 3 03:51:14 2014 From: henson at acm.org (Paul B. Henson) Date: Tue, 2 Dec 2014 19:51:14 -0800 Subject: [OmniOS-discuss] common-factor key exchange In-Reply-To: <20141203131657.11875145@punda-mlia> References: <20141128191703.00a9df99@punda-mlia> <20141130025557.GI29549@bender.unx.csupomona.edu> <20141201055654.2a3e23f8@punda-mlia> <234301d00db7$32bdcc20$98396460$@acm.org> <20141202094149.43d9f5d9@punda-mlia> <314901d00e9d$ade49f00$09addd00$@acm.org> <20141203131657.11875145@punda-mlia> Message-ID: <20141203035114.GL29549@bender.unx.csupomona.edu> On Wed, Dec 03, 2014 at 01:16:57PM +1000, Michael Mounteney wrote: > I did consider holding on to 6.6 but there must be a good reason for the > issuance of 6.7. As I said originally, I'd rather go up with OmniOS > than down with Gentoo. Looks like it fixes this security bug: https://bugs.gentoo.org/show_bug.cgi?id=505942 Don't think many people are using SSHFP DNS records, so probably not a big deal at this point. I think the odds of illumos ssh getting updated are pretty low, so we'll have to wait and see what the omnios powers that be think of switching the distribution to openssh... From moo at wuffers.net Wed Dec 3 06:02:40 2014 From: moo at wuffers.net (wuffers) Date: Wed, 3 Dec 2014 01:02:40 -0500 Subject: [OmniOS-discuss] Bad ZeusRAM? How to tell if another component is causing issues? Message-ID: I'm at home just looking into the health of our SAN and came across a bunch of errors on the Stec ZeusRAM (in a mirrored log configuration): # iostat -En c12t5000A72B300780FFd0 Soft Errors: 0 Hard Errors: 1 Transport Errors: 5224 Vendor: STEC Product: ZeusRAM Revision: C018 Serial No: STM000170C98 Size: 8.00GB <8000000000 bytes> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 Illegal Request: 391 Predictive Failure Analysis: 0 #fmdump -eV Dec 03 2014 00:26:22.592888816 ereport.io.scsi.cmd.disk.recovered nvlist version: 0 class = ereport.io.scsi.cmd.disk.recovered ena = 0xd38b237e7ed02001 detector = (embedded nvlist) nvlist version: 0 version = 0x0 scheme = dev device-path = /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f /disk at w5000a72b300780ff,0 devid = id1,sd at n5000a720300780ff (end detector) devid = id1,sd at n5000a720300780ff driver-assessment = recovered op-code = 0x2a cdb = 0x2a 0x0 0x0 0x2d 0xda 0x0 0x0 0x0 0xf8 0x0 pkt-reason = 0x0 pkt-state = 0x1f pkt-stats = 0x0 __ttl = 0x1 __tod = 0x547e9efe 0x2356c3f0 # dmesg Dec 3 00:28:24 san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci8086,3c08 at 3 /pci1000,3030 at 0 (mpt_sas1): Dec 3 00:28:24 san1 Log info 0x31120303 received for target 10 w5000a72b300780ff. Dec 3 00:28:24 san1 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc from format: 57. c12t5000A72B300780FFd0 /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f /disk at w5000a72b300780ff,0 Both fmdump and dmesg has these errors repeating over and over. Everything seems to point to the drive. I suppose I would have to physically move the drive to eliminate cable, backplane or controller issues. Is there another way to tell just by these error logs or is the physical test the way to go? Are logs enough to justify an RMA? -------------- next part -------------- An HTML attachment was scrubbed... URL: From moo at wuffers.net Wed Dec 3 06:11:59 2014 From: moo at wuffers.net (wuffers) Date: Wed, 3 Dec 2014 01:11:59 -0500 Subject: [OmniOS-discuss] Slow scrub performance In-Reply-To: References: Message-ID: Just wanted to update this thread with my results. I had the scrub started since late July, and checked on it at the end of Oct: pool: tank state: ONLINE scan: scrub in progress since Tue Jul 29 15:41:27 2014 7.65T scanned out of 24.5T at 1024K/s, (scan is slow, no estimated time) 0 repaired, 31.16% done So it was scrubbing away for ~90 days and got a bit over 31% with vdev_scrub_max set to 5. In anticipation of having to power off the head nodes for the building's power maintenance and not wanting to waste the scrub so far, I pushed the vdev_scrub_max to 10. The scrub then proceeded to accelerate to about 8% per day, and ended: pool: tank state: ONLINE scan: scrub repaired 0 in 2372h54m with 0 errors on Wed Nov 5 11:36:00 2014 My pool wasn't busy so I was okay with it having so much time slice, but this obviously isn't the best thing to do. Is this really normal, or is there something wrong with my config? On Thu, Jul 31, 2014 at 3:44 PM, wuffers wrote: > This is going to be long winded as well (apologies!).. lots of pasted data. > > On Thu, Jul 31, 2014 at 1:37 AM, Richard Elling < > richard.elling at richardelling.com> wrote: > >> >> The %busy for controllers is a sum of the %busy for all disks on the >> controller, so >> is can be large, but overall isn't interesting. With HDDs, there is no >> way you can >> saturate the controller, so we don't really care what the %busy shows. >> >> The more important item is that the number of read ops is fairly low for >> all but 4 disks. >> Since you didn't post the pool configuration, we can only guess that they >> might be a >> souce of the bottleneck. >> >> You're seeing a lot of reads from the cache devices. How much RAM does >> this system >> have? >> >> > I realized that the busy % was a sum after looking through some of that > data, but good to know that it isn't very relevant. > > The pool configuration was in the original post, but here it is again > (after re-attaching the mirror log device). Just saw your edit, but this > has been updated from the original post anyways. > > pool: tank > state: ONLINE > scan: scrub in progress since Tue Jul 29 15:41:27 2014 > 82.5G scanned out of 24.5T at 555K/s, (scan is slow, no estimated time) > 0 repaired, 0.33% done > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > c1t5000C50055F9F637d0 ONLINE 0 0 0 > c1t5000C50055F9EF2Fd0 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > c1t5000C50055F87D97d0 ONLINE 0 0 0 > c1t5000C50055F9D3B3d0 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > c1t5000C50055E6606Fd0 ONLINE 0 0 0 > c1t5000C50055F9F92Bd0 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > c1t5000C50055F856CFd0 ONLINE 0 0 0 > c1t5000C50055F9FE87d0 ONLINE 0 0 0 > mirror-4 ONLINE 0 0 0 > c1t5000C50055F84A97d0 ONLINE 0 0 0 > c1t5000C50055FA0AF7d0 ONLINE 0 0 0 > mirror-5 ONLINE 0 0 0 > c1t5000C50055F9D3E3d0 ONLINE 0 0 0 > c1t5000C50055F9F0B3d0 ONLINE 0 0 0 > mirror-6 ONLINE 0 0 0 > c1t5000C50055F8A46Fd0 ONLINE 0 0 0 > c1t5000C50055F9FB8Bd0 ONLINE 0 0 0 > mirror-7 ONLINE 0 0 0 > c1t5000C50055F8B21Fd0 ONLINE 0 0 0 > c1t5000C50055F9F89Fd0 ONLINE 0 0 0 > mirror-8 ONLINE 0 0 0 > c1t5000C50055F8BE3Fd0 ONLINE 0 0 0 > c1t5000C50055F9E123d0 ONLINE 0 0 0 > mirror-9 ONLINE 0 0 0 > c1t5000C50055F9379Bd0 ONLINE 0 0 0 > c1t5000C50055F9E7D7d0 ONLINE 0 0 0 > mirror-10 ONLINE 0 0 0 > c1t5000C50055E65F0Fd0 ONLINE 0 0 0 > c1t5000C50055F9F80Bd0 ONLINE 0 0 0 > mirror-11 ONLINE 0 0 0 > c1t5000C50055F8A22Bd0 ONLINE 0 0 0 > c1t5000C50055F8D48Fd0 ONLINE 0 0 0 > mirror-12 ONLINE 0 0 0 > c1t5000C50055E65807d0 ONLINE 0 0 0 > c1t5000C50055F8BFA3d0 ONLINE 0 0 0 > mirror-13 ONLINE 0 0 0 > c1t5000C50055E579F7d0 ONLINE 0 0 0 > c1t5000C50055E65877d0 ONLINE 0 0 0 > mirror-14 ONLINE 0 0 0 > c1t5000C50055F9FA1Fd0 ONLINE 0 0 0 > c1t5000C50055F8CDA7d0 ONLINE 0 0 0 > mirror-15 ONLINE 0 0 0 > c1t5000C50055F8BF9Bd0 ONLINE 0 0 0 > c1t5000C50055F9A607d0 ONLINE 0 0 0 > mirror-16 ONLINE 0 0 0 > c1t5000C50055E66503d0 ONLINE 0 0 0 > c1t5000C50055E4FDE7d0 ONLINE 0 0 0 > mirror-17 ONLINE 0 0 0 > c1t5000C50055F8E017d0 ONLINE 0 0 0 > c1t5000C50055F9F3EBd0 ONLINE 0 0 0 > mirror-18 ONLINE 0 0 0 > c1t5000C50055F8B80Fd0 ONLINE 0 0 0 > c1t5000C50055F9F63Bd0 ONLINE 0 0 0 > mirror-19 ONLINE 0 0 0 > c1t5000C50055F84FB7d0 ONLINE 0 0 0 > c1t5000C50055F9FEABd0 ONLINE 0 0 0 > mirror-20 ONLINE 0 0 0 > c1t5000C50055F8CCAFd0 ONLINE 0 0 0 > c1t5000C50055F9F91Bd0 ONLINE 0 0 0 > mirror-21 ONLINE 0 0 0 > c1t5000C50055E65ABBd0 ONLINE 0 0 0 > c1t5000C50055F8905Fd0 ONLINE 0 0 0 > mirror-22 ONLINE 0 0 0 > c1t5000C50055E57A5Fd0 ONLINE 0 0 0 > c1t5000C50055F87E73d0 ONLINE 0 0 0 > mirror-23 ONLINE 0 0 0 > c1t5000C50055E66053d0 ONLINE 0 0 0 > c1t5000C50055E66B63d0 ONLINE 0 0 0 > mirror-24 ONLINE 0 0 0 > c1t5000C50055F8723Bd0 ONLINE 0 0 0 > c1t5000C50055F8C3ABd0 ONLINE 0 0 0 > logs > mirror-25 ONLINE 0 0 0 > c2t5000A72A3007811Dd0 ONLINE 0 0 0 > c12t5000A72B300780FFd0 ONLINE 0 0 0 > cache > c2t500117310015D579d0 ONLINE 0 0 0 > c2t50011731001631FDd0 ONLINE 0 0 0 > c12t500117310015D59Ed0 ONLINE 0 0 0 > c12t500117310015D54Ed0 ONLINE 0 0 0 > spares > c1t5000C50055FA2AEFd0 AVAIL > c1t5000C50055E595B7d0 AVAIL > > Basically, this is 2 head nodes (Supermicro 826BE26) connected to a > Supermicro 847E26 JBOD, using LSI 9207s. There are 52 Seagate ST4000NM0023s > (4TB SAS drives) in 25 mirror pairs plus 2 which are spares. There are 4 > Smart Optimus 400GB SSDs as cache drives, and 2 Stec ZeusRAMs for slogs. > They're wired in such a way that both nodes can see all the drives (data, > cache and log), and the data drives are on separate controllers than the > cache/slog devices. RSF-1 was also specced in here but not in use at the > moment. All the SAN traffic is through InfiniBand (SRP). Each head unit has > 256GB of RAM. Dedupe is not in use and all the latest feature flags are > enabled. > > An arc_summary output: > > System Memory: > Physical RAM: 262103 MB > Free Memory : 10273 MB > LotsFree: 4095 MB > > ZFS Tunables (/etc/system): > set zfs:zfs_arc_shrink_shift = 10 > > ARC Size: > Current Size: 225626 MB (arcsize) > Target Size (Adaptive): 225626 MB (c) > Min Size (Hard Limit): 8190 MB (zfs_arc_min) > Max Size (Hard Limit): 261079 MB (zfs_arc_max) > > ARC Size Breakdown: > Most Recently Used Cache Size: 10% 23290 MB (p) > Most Frequently Used Cache Size: 89% 202335 MB (c-p) > > ARC Efficency: > Cache Access Total: 27377320465 > Cache Hit Ratio: 93% 25532510784 [Defined State for > buffer] > Cache Miss Ratio: 6% 1844809681 [Undefined State > for Buffer] > REAL Hit Ratio: 92% 25243933796 [MRU/MFU Hits Only] > > Data Demand Efficiency: 95% > Data Prefetch Efficiency: 40% > > CACHE HITS BY CACHE LIST: > Anon: --% Counter Rolled. > Most Recently Used: 18% 4663226393 (mru) [ > Return Customer ] > Most Frequently Used: 80% 20580707403 (mfu) > [ Frequent Customer ] > Most Recently Used Ghost: 0% 176686906 (mru_ghost) [ > Return Customer Evicted, Now Back ] > Most Frequently Used Ghost: 0% 126286869 (mfu_ghost) [ > Frequent Customer Evicted, Now Back ] > CACHE HITS BY DATA TYPE: > Demand Data: 95% 24413671342 > Prefetch Data: 1% 358419705 > Demand Metadata: 2% 698314899 > Prefetch Metadata: 0% 62104838 > CACHE MISSES BY DATA TYPE: > Demand Data: 69% 1277347273 > Prefetch Data: 28% 519579788 > Demand Metadata: 2% 39512363 > Prefetch Metadata: 0% 8370257 > > And a sample of arcstat (deleted first line of output): > > # arcstat -f > read,hits,miss,hit%,l2read,l2hits,l2miss,l2hit%,arcsz,l2size,l2asize 1 > > read hits miss hit% l2read l2hits l2miss l2hit% arcsz l2size > l2asize > 5.9K 4.6K 1.3K 78 1.3K 1.2K 80 93 220G 1.6T > 901G > 6.7K 5.2K 1.5K 76 1.5K 1.3K 250 83 220G 1.6T > 901G > 7.0K 5.3K 1.7K 76 1.7K 1.4K 316 81 220G 1.6T > 901G > 6.5K 5.3K 1.2K 80 1.2K 1.1K 111 91 220G 1.6T > 901G > 6.4K 5.2K 1.2K 81 1.2K 1.1K 100 91 220G 1.6T > 901G > 7.2K 5.6K 1.6K 78 1.6K 1.3K 289 81 220G 1.6T > 901G > 8.5K 6.8K 1.7K 80 1.7K 1.3K 351 79 220G 1.6T > 901G > 7.5K 5.9K 1.6K 78 1.6K 1.3K 282 82 220G 1.6T > 901G > 6.7K 5.6K 1.1K 83 1.1K 991 123 88 220G 1.6T > 901G > 6.8K 5.5K 1.3K 80 1.3K 1.1K 234 82 220G 1.6T > 901G > > Interesting to see only an l2asize of 901G even though I should have > more.. 373G x 4 is just under 1.5TB of raw storage. The compressed l2arc > size is 1.6TB, while actual used space is 901G. I expect more to be used. > Perhaps Saso can comment on this portion, if he's following this thread > (snipped from "zpool iostat -v"): > > cache - - - - - - > c2t500117310015D579d0 373G 8M 193 16 2.81M 394K > c2t50011731001631FDd0 373G 5.29M 194 15 2.85M 360K > c12t500117310015D59Ed0 373G 5.50M 191 17 2.74M 368K > c12t500117310015D54Ed0 373G 5.57M 200 14 2.89M 300K > > (from this discussion here: > http://lists.omniti.com/pipermail/omnios-discuss/2014-February/002287.html), > and the uptime on this is currently around ~58 days, so it should have had > enough time to rotate through the l2arc "rotor". > > >> methinks the scrub I/Os are getting starved and since they are low >> priority, they >> could get very starved. In general, I wouldn't worry about it, but I >> understand >> why you might be nervous. Keep in mind that in ZFS scrubs are intended to >> find >> errors on idle data, not frequently accessed data. >> >> more far below... >> >> > I'm worried because there's no way the scrub will ever complete before the > next reboot. Regular scrubs are important, right? > > >> ok, so the pool is issuing 720 read iops, including resilver workload, vs >> 1298 write iops. >> There is plenty of I/O capacity left on the table here, as you can see by >> the %busy being >> so low. >> >> So I think the pool is not scheduling scrub I/Os very well. You can >> increase the number of >> scrub I/Os in the scheduler by adjusting the zfs_vdev_scrub_max_active >> tunable. The >> default is 2, but you'll have to consider that a share (in the stock >> market sense) where >> the active sync reads and writes are getting 10 each. You can try bumping >> up the value >> and see what happens over some time, perhaps 10 minutes or so -- too >> short of a time >> and you won't get a good feeling for the impact (try this in off-peak >> time). >> echo zfs_vdev_scrub_max_active/W0t5 | mdb -kw >> will change the value from 2 to 5, increasing its share of the total I/O >> workload. >> >> You can see the progress of scan (scrubs do scan) workload by looking at >> the ZFS >> debug messages. >> echo ::zfs_dbgmsg | mdb -k >> These will look mysterious... they are. But the interesting bits are >> about how many blocks >> are visited in some amount of time (txg sync interval). Ideally, this >> will change as you >> adjust zfs_vdev_scrub_max_active. >> -- richard >> >> > Actually, you used the data from before the resilver. During resilver this > was the activity on the pool: > > 3883.5 1357.7 40141.6 60739.5 22.8 38.6 4.4 7.4 54 100 tank > > Are you looking at an individual drive's busy % or the pool's busy % to > determine whether it's "busy"? During the resilver this was the activity on > the drives (a sample - between 38-45%, whereas during the scrub the > individual drives were 2-5% busy): > > 59.5 30.0 553.2 741.8 0.0 0.9 0.0 9.8 0 45 > c1t5000C50055F8A46Fd0 > 57.4 22.5 504.0 724.5 0.0 0.8 0.0 9.6 0 41 > c1t5000C50055F856CFd0 > 58.4 24.6 531.4 786.9 0.0 0.7 0.0 8.4 0 38 > c1t5000C50055E6606Fd0 > > But yes, without the resilver the busy % was much less (during the scrub > each individual drive was 2-4% busy). I've pasted the current iostat output > further below. > > With the zfs_vdev_scrub_max_active at the default of 2, it was doing an > average of 162 blocks: > > doing scan sync txg 26678243; bm=897/1/0/15785978 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; > pausing=1 > visited 162 blocks in 6090ms > doing scan sync txg 26678244; bm=897/1/0/15786126 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; > pausing=1 > visited 162 blocks in 6094ms > > After changing it to 5, and waiting about 20 mins, I'm not seeing anything > significantly different: > > doing scan sync txg 26678816; bm=897/1/0/37082043 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; > pausing=1 > visited 163 blocks in 6154ms > doing scan sync txg 26678817; bm=897/1/0/37082193 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; > pausing=1 > visited 162 blocks in 6128ms > > pool: tank > state: ONLINE > scan: scrub in progress since Tue Jul 29 15:41:27 2014 > 97.0G scanned out of 24.5T at 599K/s, (scan is slow, no estimated time) > 0 repaired, 0.39% done > > I'll keep the zfs_vdev_scrub_max_active tunable to 5, as it doesn't > appear to be impacting too much, and monitor for changes. What's strange to > me is that it was "humming" along at 5.5MB/s at the 2 week mark but is now > 10x slower (compared to before reattaching the mirror log device). It > *seems* marginally faster, from 541K/s to almost 600K/s.. > > This is the current activity from "iostat -xnCz 60 2": > > extended device statistics > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > 158.8 1219.2 3717.8 39969.8 0.0 1.6 0.0 1.1 0 139 c1 > 3.6 35.1 86.2 730.9 0.0 0.0 0.0 0.9 0 3 > c1t5000C50055F8723Bd0 > 3.7 19.9 83.7 789.9 0.0 0.0 0.0 1.4 0 3 > c1t5000C50055E66B63d0 > 2.7 22.5 60.8 870.9 0.0 0.0 0.0 1.1 0 2 > c1t5000C50055F87E73d0 > 2.4 27.9 66.0 765.8 0.0 0.0 0.0 0.8 0 2 > c1t5000C50055F8BFA3d0 > 2.8 17.9 64.9 767.0 0.0 0.0 0.0 0.8 0 1 > c1t5000C50055F9E123d0 > 3.1 26.1 73.8 813.3 0.0 0.0 0.0 0.9 0 2 > c1t5000C50055F9F0B3d0 > 3.1 15.5 79.4 783.4 0.0 0.0 0.0 1.3 0 2 > c1t5000C50055F9D3B3d0 > 3.8 38.6 86.2 826.8 0.0 0.1 0.0 1.2 0 4 > c1t5000C50055E4FDE7d0 > 3.8 15.4 93.0 822.3 0.0 0.0 0.0 1.5 0 3 > c1t5000C50055F9A607d0 > 3.0 25.7 79.4 719.7 0.0 0.0 0.0 0.9 0 2 > c1t5000C50055F8CDA7d0 > 3.2 26.5 69.0 824.3 0.0 0.0 0.0 1.1 0 3 > c1t5000C50055E65877d0 > 3.7 42.6 79.2 834.1 0.0 0.1 0.0 1.3 0 5 > c1t5000C50055F9E7D7d0 > 3.3 23.2 79.5 778.0 0.0 0.0 0.0 1.2 0 3 > c1t5000C50055FA0AF7d0 > 3.4 30.2 77.0 805.9 0.0 0.0 0.0 0.9 0 3 > c1t5000C50055F9FE87d0 > 3.0 15.4 72.6 795.0 0.0 0.0 0.0 1.6 0 3 > c1t5000C50055F9F91Bd0 > 2.5 38.1 61.1 859.4 0.0 0.1 0.0 1.6 0 5 > c1t5000C50055F9FEABd0 > 2.1 13.2 42.7 801.6 0.0 0.0 0.0 1.6 0 2 > c1t5000C50055F9F63Bd0 > 3.0 20.0 62.6 766.6 0.0 0.0 0.0 1.1 0 2 > c1t5000C50055F9F3EBd0 > 3.7 24.3 80.2 807.9 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055F9F80Bd0 > 3.2 35.2 66.1 852.4 0.0 0.0 0.0 1.2 0 4 > c1t5000C50055F9FB8Bd0 > 3.9 30.6 84.7 845.7 0.0 0.0 0.0 0.8 0 3 > c1t5000C50055F9F92Bd0 > 2.7 18.1 68.8 831.4 0.0 0.0 0.0 1.4 0 2 > c1t5000C50055F8905Fd0 > 2.7 17.7 61.4 762.1 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055F8D48Fd0 > 3.5 17.5 87.8 749.7 0.0 0.0 0.0 1.7 0 3 > c1t5000C50055F9F89Fd0 > 2.6 13.7 58.6 780.9 0.0 0.0 0.0 1.7 0 3 > c1t5000C50055F9EF2Fd0 > 3.3 34.9 74.5 730.9 0.0 0.0 0.0 0.8 0 3 > c1t5000C50055F8C3ABd0 > 3.1 19.3 64.7 789.9 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055E66053d0 > 3.8 38.5 82.9 826.8 0.0 0.1 0.0 1.3 0 4 > c1t5000C50055E66503d0 > 3.7 25.8 91.4 813.3 0.0 0.0 0.0 0.8 0 2 > c1t5000C50055F9D3E3d0 > 2.2 37.9 52.5 859.4 0.0 0.0 0.0 1.1 0 4 > c1t5000C50055F84FB7d0 > 2.8 20.0 62.8 766.6 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055F8E017d0 > 3.9 26.1 86.5 824.3 0.0 0.0 0.0 1.1 0 3 > c1t5000C50055E579F7d0 > 3.1 27.7 79.9 765.8 0.0 0.0 0.0 1.2 0 3 > c1t5000C50055E65807d0 > 2.9 22.8 76.3 778.0 0.0 0.0 0.0 1.1 0 3 > c1t5000C50055F84A97d0 > 3.6 15.3 89.0 783.4 0.0 0.0 0.0 1.7 0 3 > c1t5000C50055F87D97d0 > 2.8 13.8 77.9 780.9 0.0 0.0 0.0 1.5 0 2 > c1t5000C50055F9F637d0 > 2.1 18.3 51.4 831.4 0.0 0.0 0.0 1.1 0 2 > c1t5000C50055E65ABBd0 > 3.1 15.4 70.9 822.3 0.0 0.0 0.0 1.2 0 2 > c1t5000C50055F8BF9Bd0 > 3.2 17.9 75.5 762.1 0.0 0.0 0.0 1.2 0 2 > c1t5000C50055F8A22Bd0 > 3.7 42.4 83.3 834.1 0.0 0.1 0.0 1.1 0 5 > c1t5000C50055F9379Bd0 > 4.0 22.7 86.8 870.9 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055E57A5Fd0 > 2.6 15.5 67.5 795.0 0.0 0.0 0.0 1.4 0 2 > c1t5000C50055F8CCAFd0 > 2.9 13.2 65.4 801.6 0.0 0.0 0.0 1.9 0 3 > c1t5000C50055F8B80Fd0 > 3.3 25.7 82.7 719.7 0.0 0.0 0.0 1.1 0 3 > c1t5000C50055F9FA1Fd0 > 4.0 24.0 84.9 807.9 0.0 0.0 0.0 1.1 0 3 > c1t5000C50055E65F0Fd0 > 2.8 18.4 69.5 767.0 0.0 0.0 0.0 1.0 0 2 > c1t5000C50055F8BE3Fd0 > 3.3 17.6 81.6 749.7 0.0 0.0 0.0 1.4 0 3 > c1t5000C50055F8B21Fd0 > 3.3 35.1 64.2 852.4 0.0 0.0 0.0 1.1 0 4 > c1t5000C50055F8A46Fd0 > 3.5 30.0 82.1 805.9 0.0 0.0 0.0 0.9 0 3 > c1t5000C50055F856CFd0 > 3.9 30.4 89.5 845.7 0.0 0.0 0.0 0.9 0 3 > c1t5000C50055E6606Fd0 > 429.4 133.6 5933.0 8163.0 0.0 0.2 0.0 0.3 0 12 c2 > 215.8 28.4 2960.4 677.7 0.0 0.1 0.0 0.2 0 5 > c2t500117310015D579d0 > 213.7 27.4 2972.6 654.1 0.0 0.1 0.0 0.2 0 5 > c2t50011731001631FDd0 > 0.0 77.8 0.0 6831.2 0.0 0.1 0.0 0.8 0 2 > c2t5000A72A3007811Dd0 > 0.0 12.3 0.0 46.8 0.0 0.0 0.0 0.0 0 0 c4 > 0.0 6.2 0.0 23.4 0.0 0.0 0.0 0.0 0 0 c4t0d0 > 0.0 6.1 0.0 23.4 0.0 0.0 0.0 0.0 0 0 c4t1d0 > 418.4 134.8 5663.1 8197.8 0.0 0.2 0.0 0.3 0 11 c12 > 0.0 77.8 0.0 6831.2 0.0 0.1 0.0 0.8 0 2 > c12t5000A72B300780FFd0 > 203.5 29.7 2738.0 715.8 0.0 0.1 0.0 0.2 0 5 > c12t500117310015D59Ed0 > 214.9 27.2 2925.2 650.8 0.0 0.1 0.0 0.2 0 5 > c12t500117310015D54Ed0 > 0.0 11.3 0.0 46.8 0.0 0.0 0.7 0.1 0 0 rpool > 1006.7 1478.2 15313.9 56330.7 30.4 2.0 12.2 0.8 6 64 tank > > Seems the pool is busy at 64% but the individual drives are not taxed at > all (this load is virtually identical to when the scrub was running before > the resilver was triggered). Still not too sure how to interpret this data. > Is the system over-stressed? Is there really a bottleneck somewhere, or > just need to fine tune some settings? > > Going to try some iometer results via the VM I/O Analyzer. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.elling at richardelling.com Wed Dec 3 06:27:59 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Tue, 2 Dec 2014 22:27:59 -0800 Subject: [OmniOS-discuss] Slow scrub performance In-Reply-To: References: Message-ID: On Dec 2, 2014, at 10:11 PM, wuffers wrote: > Just wanted to update this thread with my results. I had the scrub started since late July, and checked on it at the end of Oct: > > pool: tank > state: ONLINE > scan: scrub in progress since Tue Jul 29 15:41:27 2014 > 7.65T scanned out of 24.5T at 1024K/s, (scan is slow, no estimated time) > 0 repaired, 31.16% done > > So it was scrubbing away for ~90 days and got a bit over 31% with vdev_scrub_max set to 5. > > In anticipation of having to power off the head nodes for the building's power maintenance and not wanting to waste the scrub so far, I pushed the vdev_scrub_max to 10. The scrub then proceeded to accelerate to about 8% per day, and ended: > > pool: tank > state: ONLINE > scan: scrub repaired 0 in 2372h54m with 0 errors on Wed Nov 5 11:36:00 2014 > > My pool wasn't busy so I was okay with it having so much time slice, but this obviously isn't the best thing to do. Is this really normal, or is there something wrong with my config? The default settings for zfs_vdev_scrub_max_active are (mostly) a guess. It is expected that you can tune. It would not be surprising for the default to change as we get more, varied field experience. Thanks for sharing your results! -- richard > > On Thu, Jul 31, 2014 at 3:44 PM, wuffers wrote: > This is going to be long winded as well (apologies!).. lots of pasted data. > > On Thu, Jul 31, 2014 at 1:37 AM, Richard Elling wrote: > > The %busy for controllers is a sum of the %busy for all disks on the controller, so > is can be large, but overall isn't interesting. With HDDs, there is no way you can > saturate the controller, so we don't really care what the %busy shows. > > The more important item is that the number of read ops is fairly low for all but 4 disks. > Since you didn't post the pool configuration, we can only guess that they might be a > souce of the bottleneck. > > You're seeing a lot of reads from the cache devices. How much RAM does this system > have? > > > I realized that the busy % was a sum after looking through some of that data, but good to know that it isn't very relevant. > > The pool configuration was in the original post, but here it is again (after re-attaching the mirror log device). Just saw your edit, but this has been updated from the original post anyways. > > pool: tank > state: ONLINE > scan: scrub in progress since Tue Jul 29 15:41:27 2014 > 82.5G scanned out of 24.5T at 555K/s, (scan is slow, no estimated time) > 0 repaired, 0.33% done > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > c1t5000C50055F9F637d0 ONLINE 0 0 0 > c1t5000C50055F9EF2Fd0 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > c1t5000C50055F87D97d0 ONLINE 0 0 0 > c1t5000C50055F9D3B3d0 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > c1t5000C50055E6606Fd0 ONLINE 0 0 0 > c1t5000C50055F9F92Bd0 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > c1t5000C50055F856CFd0 ONLINE 0 0 0 > c1t5000C50055F9FE87d0 ONLINE 0 0 0 > mirror-4 ONLINE 0 0 0 > c1t5000C50055F84A97d0 ONLINE 0 0 0 > c1t5000C50055FA0AF7d0 ONLINE 0 0 0 > mirror-5 ONLINE 0 0 0 > c1t5000C50055F9D3E3d0 ONLINE 0 0 0 > c1t5000C50055F9F0B3d0 ONLINE 0 0 0 > mirror-6 ONLINE 0 0 0 > c1t5000C50055F8A46Fd0 ONLINE 0 0 0 > c1t5000C50055F9FB8Bd0 ONLINE 0 0 0 > mirror-7 ONLINE 0 0 0 > c1t5000C50055F8B21Fd0 ONLINE 0 0 0 > c1t5000C50055F9F89Fd0 ONLINE 0 0 0 > mirror-8 ONLINE 0 0 0 > c1t5000C50055F8BE3Fd0 ONLINE 0 0 0 > c1t5000C50055F9E123d0 ONLINE 0 0 0 > mirror-9 ONLINE 0 0 0 > c1t5000C50055F9379Bd0 ONLINE 0 0 0 > c1t5000C50055F9E7D7d0 ONLINE 0 0 0 > mirror-10 ONLINE 0 0 0 > c1t5000C50055E65F0Fd0 ONLINE 0 0 0 > c1t5000C50055F9F80Bd0 ONLINE 0 0 0 > mirror-11 ONLINE 0 0 0 > c1t5000C50055F8A22Bd0 ONLINE 0 0 0 > c1t5000C50055F8D48Fd0 ONLINE 0 0 0 > mirror-12 ONLINE 0 0 0 > c1t5000C50055E65807d0 ONLINE 0 0 0 > c1t5000C50055F8BFA3d0 ONLINE 0 0 0 > mirror-13 ONLINE 0 0 0 > c1t5000C50055E579F7d0 ONLINE 0 0 0 > c1t5000C50055E65877d0 ONLINE 0 0 0 > mirror-14 ONLINE 0 0 0 > c1t5000C50055F9FA1Fd0 ONLINE 0 0 0 > c1t5000C50055F8CDA7d0 ONLINE 0 0 0 > mirror-15 ONLINE 0 0 0 > c1t5000C50055F8BF9Bd0 ONLINE 0 0 0 > c1t5000C50055F9A607d0 ONLINE 0 0 0 > mirror-16 ONLINE 0 0 0 > c1t5000C50055E66503d0 ONLINE 0 0 0 > c1t5000C50055E4FDE7d0 ONLINE 0 0 0 > mirror-17 ONLINE 0 0 0 > c1t5000C50055F8E017d0 ONLINE 0 0 0 > c1t5000C50055F9F3EBd0 ONLINE 0 0 0 > mirror-18 ONLINE 0 0 0 > c1t5000C50055F8B80Fd0 ONLINE 0 0 0 > c1t5000C50055F9F63Bd0 ONLINE 0 0 0 > mirror-19 ONLINE 0 0 0 > c1t5000C50055F84FB7d0 ONLINE 0 0 0 > c1t5000C50055F9FEABd0 ONLINE 0 0 0 > mirror-20 ONLINE 0 0 0 > c1t5000C50055F8CCAFd0 ONLINE 0 0 0 > c1t5000C50055F9F91Bd0 ONLINE 0 0 0 > mirror-21 ONLINE 0 0 0 > c1t5000C50055E65ABBd0 ONLINE 0 0 0 > c1t5000C50055F8905Fd0 ONLINE 0 0 0 > mirror-22 ONLINE 0 0 0 > c1t5000C50055E57A5Fd0 ONLINE 0 0 0 > c1t5000C50055F87E73d0 ONLINE 0 0 0 > mirror-23 ONLINE 0 0 0 > c1t5000C50055E66053d0 ONLINE 0 0 0 > c1t5000C50055E66B63d0 ONLINE 0 0 0 > mirror-24 ONLINE 0 0 0 > c1t5000C50055F8723Bd0 ONLINE 0 0 0 > c1t5000C50055F8C3ABd0 ONLINE 0 0 0 > logs > mirror-25 ONLINE 0 0 0 > c2t5000A72A3007811Dd0 ONLINE 0 0 0 > c12t5000A72B300780FFd0 ONLINE 0 0 0 > cache > c2t500117310015D579d0 ONLINE 0 0 0 > c2t50011731001631FDd0 ONLINE 0 0 0 > c12t500117310015D59Ed0 ONLINE 0 0 0 > c12t500117310015D54Ed0 ONLINE 0 0 0 > spares > c1t5000C50055FA2AEFd0 AVAIL > c1t5000C50055E595B7d0 AVAIL > > Basically, this is 2 head nodes (Supermicro 826BE26) connected to a Supermicro 847E26 JBOD, using LSI 9207s. There are 52 Seagate ST4000NM0023s (4TB SAS drives) in 25 mirror pairs plus 2 which are spares. There are 4 Smart Optimus 400GB SSDs as cache drives, and 2 Stec ZeusRAMs for slogs. They're wired in such a way that both nodes can see all the drives (data, cache and log), and the data drives are on separate controllers than the cache/slog devices. RSF-1 was also specced in here but not in use at the moment. All the SAN traffic is through InfiniBand (SRP). Each head unit has 256GB of RAM. Dedupe is not in use and all the latest feature flags are enabled. > > An arc_summary output: > > System Memory: > Physical RAM: 262103 MB > Free Memory : 10273 MB > LotsFree: 4095 MB > > ZFS Tunables (/etc/system): > set zfs:zfs_arc_shrink_shift = 10 > > ARC Size: > Current Size: 225626 MB (arcsize) > Target Size (Adaptive): 225626 MB (c) > Min Size (Hard Limit): 8190 MB (zfs_arc_min) > Max Size (Hard Limit): 261079 MB (zfs_arc_max) > > ARC Size Breakdown: > Most Recently Used Cache Size: 10% 23290 MB (p) > Most Frequently Used Cache Size: 89% 202335 MB (c-p) > > ARC Efficency: > Cache Access Total: 27377320465 > Cache Hit Ratio: 93% 25532510784 [Defined State for buffer] > Cache Miss Ratio: 6% 1844809681 [Undefined State for Buffer] > REAL Hit Ratio: 92% 25243933796 [MRU/MFU Hits Only] > > Data Demand Efficiency: 95% > Data Prefetch Efficiency: 40% > > CACHE HITS BY CACHE LIST: > Anon: --% Counter Rolled. > Most Recently Used: 18% 4663226393 (mru) [ Return Customer ] > Most Frequently Used: 80% 20580707403 (mfu) [ Frequent Customer ] > Most Recently Used Ghost: 0% 176686906 (mru_ghost) [ Return Customer Evicted, Now Back ] > Most Frequently Used Ghost: 0% 126286869 (mfu_ghost) [ Frequent Customer Evicted, Now Back ] > CACHE HITS BY DATA TYPE: > Demand Data: 95% 24413671342 > Prefetch Data: 1% 358419705 > Demand Metadata: 2% 698314899 > Prefetch Metadata: 0% 62104838 > CACHE MISSES BY DATA TYPE: > Demand Data: 69% 1277347273 > Prefetch Data: 28% 519579788 > Demand Metadata: 2% 39512363 > Prefetch Metadata: 0% 8370257 > > And a sample of arcstat (deleted first line of output): > > # arcstat -f read,hits,miss,hit%,l2read,l2hits,l2miss,l2hit%,arcsz,l2size,l2asize 1 > > read hits miss hit% l2read l2hits l2miss l2hit% arcsz l2size l2asize > 5.9K 4.6K 1.3K 78 1.3K 1.2K 80 93 220G 1.6T 901G > 6.7K 5.2K 1.5K 76 1.5K 1.3K 250 83 220G 1.6T 901G > 7.0K 5.3K 1.7K 76 1.7K 1.4K 316 81 220G 1.6T 901G > 6.5K 5.3K 1.2K 80 1.2K 1.1K 111 91 220G 1.6T 901G > 6.4K 5.2K 1.2K 81 1.2K 1.1K 100 91 220G 1.6T 901G > 7.2K 5.6K 1.6K 78 1.6K 1.3K 289 81 220G 1.6T 901G > 8.5K 6.8K 1.7K 80 1.7K 1.3K 351 79 220G 1.6T 901G > 7.5K 5.9K 1.6K 78 1.6K 1.3K 282 82 220G 1.6T 901G > 6.7K 5.6K 1.1K 83 1.1K 991 123 88 220G 1.6T 901G > 6.8K 5.5K 1.3K 80 1.3K 1.1K 234 82 220G 1.6T 901G > > Interesting to see only an l2asize of 901G even though I should have more.. 373G x 4 is just under 1.5TB of raw storage. The compressed l2arc size is 1.6TB, while actual used space is 901G. I expect more to be used. Perhaps Saso can comment on this portion, if he's following this thread (snipped from "zpool iostat -v"): > > cache - - - - - - > c2t500117310015D579d0 373G 8M 193 16 2.81M 394K > c2t50011731001631FDd0 373G 5.29M 194 15 2.85M 360K > c12t500117310015D59Ed0 373G 5.50M 191 17 2.74M 368K > c12t500117310015D54Ed0 373G 5.57M 200 14 2.89M 300K > > (from this discussion here: http://lists.omniti.com/pipermail/omnios-discuss/2014-February/002287.html), and the uptime on this is currently around ~58 days, so it should have had enough time to rotate through the l2arc "rotor". > > > methinks the scrub I/Os are getting starved and since they are low priority, they > could get very starved. In general, I wouldn't worry about it, but I understand > why you might be nervous. Keep in mind that in ZFS scrubs are intended to find > errors on idle data, not frequently accessed data. > > more far below... > > > I'm worried because there's no way the scrub will ever complete before the next reboot. Regular scrubs are important, right? > > > ok, so the pool is issuing 720 read iops, including resilver workload, vs 1298 write iops. > There is plenty of I/O capacity left on the table here, as you can see by the %busy being > so low. > > So I think the pool is not scheduling scrub I/Os very well. You can increase the number of > scrub I/Os in the scheduler by adjusting the zfs_vdev_scrub_max_active tunable. The > default is 2, but you'll have to consider that a share (in the stock market sense) where > the active sync reads and writes are getting 10 each. You can try bumping up the value > and see what happens over some time, perhaps 10 minutes or so -- too short of a time > and you won't get a good feeling for the impact (try this in off-peak time). > echo zfs_vdev_scrub_max_active/W0t5 | mdb -kw > will change the value from 2 to 5, increasing its share of the total I/O workload. > > You can see the progress of scan (scrubs do scan) workload by looking at the ZFS > debug messages. > echo ::zfs_dbgmsg | mdb -k > These will look mysterious... they are. But the interesting bits are about how many blocks > are visited in some amount of time (txg sync interval). Ideally, this will change as you > adjust zfs_vdev_scrub_max_active. > -- richard > > > Actually, you used the data from before the resilver. During resilver this was the activity on the pool: > > 3883.5 1357.7 40141.6 60739.5 22.8 38.6 4.4 7.4 54 100 tank > > Are you looking at an individual drive's busy % or the pool's busy % to determine whether it's "busy"? During the resilver this was the activity on the drives (a sample - between 38-45%, whereas during the scrub the individual drives were 2-5% busy): > > 59.5 30.0 553.2 741.8 0.0 0.9 0.0 9.8 0 45 c1t5000C50055F8A46Fd0 > 57.4 22.5 504.0 724.5 0.0 0.8 0.0 9.6 0 41 c1t5000C50055F856CFd0 > 58.4 24.6 531.4 786.9 0.0 0.7 0.0 8.4 0 38 c1t5000C50055E6606Fd0 > > But yes, without the resilver the busy % was much less (during the scrub each individual drive was 2-4% busy). I've pasted the current iostat output further below. > > With the zfs_vdev_scrub_max_active at the default of 2, it was doing an average of 162 blocks: > > doing scan sync txg 26678243; bm=897/1/0/15785978 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1 > visited 162 blocks in 6090ms > doing scan sync txg 26678244; bm=897/1/0/15786126 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1 > visited 162 blocks in 6094ms > > After changing it to 5, and waiting about 20 mins, I'm not seeing anything significantly different: > > doing scan sync txg 26678816; bm=897/1/0/37082043 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1 > visited 163 blocks in 6154ms > doing scan sync txg 26678817; bm=897/1/0/37082193 > scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1 > visited 162 blocks in 6128ms > > pool: tank > state: ONLINE > scan: scrub in progress since Tue Jul 29 15:41:27 2014 > 97.0G scanned out of 24.5T at 599K/s, (scan is slow, no estimated time) > 0 repaired, 0.39% done > > I'll keep the zfs_vdev_scrub_max_active tunable to 5, as it doesn't appear to be impacting too much, and monitor for changes. What's strange to me is that it was "humming" along at 5.5MB/s at the 2 week mark but is now 10x slower (compared to before reattaching the mirror log device). It *seems* marginally faster, from 541K/s to almost 600K/s.. > > This is the current activity from "iostat -xnCz 60 2": > > extended device statistics > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > 158.8 1219.2 3717.8 39969.8 0.0 1.6 0.0 1.1 0 139 c1 > 3.6 35.1 86.2 730.9 0.0 0.0 0.0 0.9 0 3 c1t5000C50055F8723Bd0 > 3.7 19.9 83.7 789.9 0.0 0.0 0.0 1.4 0 3 c1t5000C50055E66B63d0 > 2.7 22.5 60.8 870.9 0.0 0.0 0.0 1.1 0 2 c1t5000C50055F87E73d0 > 2.4 27.9 66.0 765.8 0.0 0.0 0.0 0.8 0 2 c1t5000C50055F8BFA3d0 > 2.8 17.9 64.9 767.0 0.0 0.0 0.0 0.8 0 1 c1t5000C50055F9E123d0 > 3.1 26.1 73.8 813.3 0.0 0.0 0.0 0.9 0 2 c1t5000C50055F9F0B3d0 > 3.1 15.5 79.4 783.4 0.0 0.0 0.0 1.3 0 2 c1t5000C50055F9D3B3d0 > 3.8 38.6 86.2 826.8 0.0 0.1 0.0 1.2 0 4 c1t5000C50055E4FDE7d0 > 3.8 15.4 93.0 822.3 0.0 0.0 0.0 1.5 0 3 c1t5000C50055F9A607d0 > 3.0 25.7 79.4 719.7 0.0 0.0 0.0 0.9 0 2 c1t5000C50055F8CDA7d0 > 3.2 26.5 69.0 824.3 0.0 0.0 0.0 1.1 0 3 c1t5000C50055E65877d0 > 3.7 42.6 79.2 834.1 0.0 0.1 0.0 1.3 0 5 c1t5000C50055F9E7D7d0 > 3.3 23.2 79.5 778.0 0.0 0.0 0.0 1.2 0 3 c1t5000C50055FA0AF7d0 > 3.4 30.2 77.0 805.9 0.0 0.0 0.0 0.9 0 3 c1t5000C50055F9FE87d0 > 3.0 15.4 72.6 795.0 0.0 0.0 0.0 1.6 0 3 c1t5000C50055F9F91Bd0 > 2.5 38.1 61.1 859.4 0.0 0.1 0.0 1.6 0 5 c1t5000C50055F9FEABd0 > 2.1 13.2 42.7 801.6 0.0 0.0 0.0 1.6 0 2 c1t5000C50055F9F63Bd0 > 3.0 20.0 62.6 766.6 0.0 0.0 0.0 1.1 0 2 c1t5000C50055F9F3EBd0 > 3.7 24.3 80.2 807.9 0.0 0.0 0.0 1.0 0 2 c1t5000C50055F9F80Bd0 > 3.2 35.2 66.1 852.4 0.0 0.0 0.0 1.2 0 4 c1t5000C50055F9FB8Bd0 > 3.9 30.6 84.7 845.7 0.0 0.0 0.0 0.8 0 3 c1t5000C50055F9F92Bd0 > 2.7 18.1 68.8 831.4 0.0 0.0 0.0 1.4 0 2 c1t5000C50055F8905Fd0 > 2.7 17.7 61.4 762.1 0.0 0.0 0.0 1.0 0 2 c1t5000C50055F8D48Fd0 > 3.5 17.5 87.8 749.7 0.0 0.0 0.0 1.7 0 3 c1t5000C50055F9F89Fd0 > 2.6 13.7 58.6 780.9 0.0 0.0 0.0 1.7 0 3 c1t5000C50055F9EF2Fd0 > 3.3 34.9 74.5 730.9 0.0 0.0 0.0 0.8 0 3 c1t5000C50055F8C3ABd0 > 3.1 19.3 64.7 789.9 0.0 0.0 0.0 1.0 0 2 c1t5000C50055E66053d0 > 3.8 38.5 82.9 826.8 0.0 0.1 0.0 1.3 0 4 c1t5000C50055E66503d0 > 3.7 25.8 91.4 813.3 0.0 0.0 0.0 0.8 0 2 c1t5000C50055F9D3E3d0 > 2.2 37.9 52.5 859.4 0.0 0.0 0.0 1.1 0 4 c1t5000C50055F84FB7d0 > 2.8 20.0 62.8 766.6 0.0 0.0 0.0 1.0 0 2 c1t5000C50055F8E017d0 > 3.9 26.1 86.5 824.3 0.0 0.0 0.0 1.1 0 3 c1t5000C50055E579F7d0 > 3.1 27.7 79.9 765.8 0.0 0.0 0.0 1.2 0 3 c1t5000C50055E65807d0 > 2.9 22.8 76.3 778.0 0.0 0.0 0.0 1.1 0 3 c1t5000C50055F84A97d0 > 3.6 15.3 89.0 783.4 0.0 0.0 0.0 1.7 0 3 c1t5000C50055F87D97d0 > 2.8 13.8 77.9 780.9 0.0 0.0 0.0 1.5 0 2 c1t5000C50055F9F637d0 > 2.1 18.3 51.4 831.4 0.0 0.0 0.0 1.1 0 2 c1t5000C50055E65ABBd0 > 3.1 15.4 70.9 822.3 0.0 0.0 0.0 1.2 0 2 c1t5000C50055F8BF9Bd0 > 3.2 17.9 75.5 762.1 0.0 0.0 0.0 1.2 0 2 c1t5000C50055F8A22Bd0 > 3.7 42.4 83.3 834.1 0.0 0.1 0.0 1.1 0 5 c1t5000C50055F9379Bd0 > 4.0 22.7 86.8 870.9 0.0 0.0 0.0 1.0 0 2 c1t5000C50055E57A5Fd0 > 2.6 15.5 67.5 795.0 0.0 0.0 0.0 1.4 0 2 c1t5000C50055F8CCAFd0 > 2.9 13.2 65.4 801.6 0.0 0.0 0.0 1.9 0 3 c1t5000C50055F8B80Fd0 > 3.3 25.7 82.7 719.7 0.0 0.0 0.0 1.1 0 3 c1t5000C50055F9FA1Fd0 > 4.0 24.0 84.9 807.9 0.0 0.0 0.0 1.1 0 3 c1t5000C50055E65F0Fd0 > 2.8 18.4 69.5 767.0 0.0 0.0 0.0 1.0 0 2 c1t5000C50055F8BE3Fd0 > 3.3 17.6 81.6 749.7 0.0 0.0 0.0 1.4 0 3 c1t5000C50055F8B21Fd0 > 3.3 35.1 64.2 852.4 0.0 0.0 0.0 1.1 0 4 c1t5000C50055F8A46Fd0 > 3.5 30.0 82.1 805.9 0.0 0.0 0.0 0.9 0 3 c1t5000C50055F856CFd0 > 3.9 30.4 89.5 845.7 0.0 0.0 0.0 0.9 0 3 c1t5000C50055E6606Fd0 > 429.4 133.6 5933.0 8163.0 0.0 0.2 0.0 0.3 0 12 c2 > 215.8 28.4 2960.4 677.7 0.0 0.1 0.0 0.2 0 5 c2t500117310015D579d0 > 213.7 27.4 2972.6 654.1 0.0 0.1 0.0 0.2 0 5 c2t50011731001631FDd0 > 0.0 77.8 0.0 6831.2 0.0 0.1 0.0 0.8 0 2 c2t5000A72A3007811Dd0 > 0.0 12.3 0.0 46.8 0.0 0.0 0.0 0.0 0 0 c4 > 0.0 6.2 0.0 23.4 0.0 0.0 0.0 0.0 0 0 c4t0d0 > 0.0 6.1 0.0 23.4 0.0 0.0 0.0 0.0 0 0 c4t1d0 > 418.4 134.8 5663.1 8197.8 0.0 0.2 0.0 0.3 0 11 c12 > 0.0 77.8 0.0 6831.2 0.0 0.1 0.0 0.8 0 2 c12t5000A72B300780FFd0 > 203.5 29.7 2738.0 715.8 0.0 0.1 0.0 0.2 0 5 c12t500117310015D59Ed0 > 214.9 27.2 2925.2 650.8 0.0 0.1 0.0 0.2 0 5 c12t500117310015D54Ed0 > 0.0 11.3 0.0 46.8 0.0 0.0 0.7 0.1 0 0 rpool > 1006.7 1478.2 15313.9 56330.7 30.4 2.0 12.2 0.8 6 64 tank > > Seems the pool is busy at 64% but the individual drives are not taxed at all (this load is virtually identical to when the scrub was running before the resilver was triggered). Still not too sure how to interpret this data. Is the system over-stressed? Is there really a bottleneck somewhere, or just need to fine tune some settings? > > Going to try some iometer results via the VM I/O Analyzer. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.elling at richardelling.com Wed Dec 3 06:31:23 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Tue, 2 Dec 2014 22:31:23 -0800 Subject: [OmniOS-discuss] Bad ZeusRAM? How to tell if another component is causing issues? In-Reply-To: References: Message-ID: <5601F3FA-4BD6-4E2B-B365-8640605BAECC@richardelling.com> On Dec 2, 2014, at 10:02 PM, wuffers wrote: > I'm at home just looking into the health of our SAN and came across a bunch of errors on the Stec ZeusRAM (in a mirrored log configuration): > > # iostat -En > c12t5000A72B300780FFd0 Soft Errors: 0 Hard Errors: 1 Transport Errors: 5224 ZeusRAMs are more sensitive to noisy fabric or cables than some other drives. At the OS level, one symptom is transport errors, but that is only one view of the fabric and you'll need more than one view to reach root cause. Check the drive's health using something like sg3_utils: sg_logs -a /dev/rdsk/c#t#d# and look for link stats, especially loss of DWORD sync and running disparity errors. -- richard > Vendor: STEC Product: ZeusRAM Revision: C018 Serial No: STM000170C98 > Size: 8.00GB <8000000000 bytes> > Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 > Illegal Request: 391 Predictive Failure Analysis: 0 > > #fmdump -eV > Dec 03 2014 00:26:22.592888816 ereport.io.scsi.cmd.disk.recovered > nvlist version: 0 > class = ereport.io.scsi.cmd.disk.recovered > ena = 0xd38b237e7ed02001 > detector = (embedded nvlist) > nvlist version: 0 > version = 0x0 > scheme = dev > device-path = /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f/disk at w5000a72b300780ff,0 > devid = id1,sd at n5000a720300780ff > (end detector) > > devid = id1,sd at n5000a720300780ff > driver-assessment = recovered > op-code = 0x2a > cdb = 0x2a 0x0 0x0 0x2d 0xda 0x0 0x0 0x0 0xf8 0x0 > pkt-reason = 0x0 > pkt-state = 0x1f > pkt-stats = 0x0 > __ttl = 0x1 > __tod = 0x547e9efe 0x2356c3f0 > > # dmesg > Dec 3 00:28:24 san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0 (mpt_sas1): > Dec 3 00:28:24 san1 Log info 0x31120303 received for target 10 w5000a72b300780ff. > Dec 3 00:28:24 san1 scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc > > from format: > > 57. c12t5000A72B300780FFd0 > /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f/disk at w5000a72b300780ff,0 > > Both fmdump and dmesg has these errors repeating over and over. Everything seems to point to the drive. I suppose I would have to physically move the drive to eliminate cable, backplane or controller issues. Is there another way to tell just by these error logs or is the physical test the way to go? > > Are logs enough to justify an RMA? > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From piiv at zhaw.ch Wed Dec 3 09:33:30 2014 From: piiv at zhaw.ch (Vincenzo Pii) Date: Wed, 3 Dec 2014 10:33:30 +0100 Subject: [OmniOS-discuss] packaging - unwanted directory group owner Message-ID: I am on my way to package some source code for OmniOS. For one package I have this problem, as seen from the res file: dir path=etc/init.d owner=root group=bin mode=0755 This is changing the owner of the /etc/init.d dir to bin (should be sys) and so conflicts with other packages. If I try to install this, I get pkg install: The requested change to the system attempts to install multiple actions for dir 'etc/init.d' with conflicting attributes: 1 package delivers 'dir group=bin mode=0755 owner=root path=etc/init.d': [...] 5 packages deliver 'dir group=sys mode=0755 owner=root path=etc/init.d': [...] Now, I can't find where in the source code this change is happening (e.g., chgrp'ing or chown'ing the folder), but I don't even know if this "bin" group comes from some default setting of the packaging template (so not directly from one of the install scripts in the code). My questions are: 1) Is there a way to manually edit the res file (.p5m.int.3.res) and just rebuild the package? The edit would consist in changing "group=bin" to "group=sys" (I am 100% sure this won't be a problem for this package). 2) If "group=bin" comes from some default, can I override this setting? Many thanks! -- Vincenzo Pii Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) blog.zhaw.ch/icclab -------------- next part -------------- An HTML attachment was scrubbed... URL: From piiv at zhaw.ch Wed Dec 3 09:48:05 2014 From: piiv at zhaw.ch (Vincenzo Pii) Date: Wed, 3 Dec 2014 10:48:05 +0100 Subject: [OmniOS-discuss] packaging - unwanted directory group owner In-Reply-To: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch> References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch> Message-ID: Ok, problem solved :D! I added a "local.mog" file to my build directory (besides build.sh), with the following content: set group sys> I realized I could do that by browsing the omnios-build repo, for the bash package: https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog 2014-12-03 10:33 GMT+01:00 Pii Vincenzo Maria (piiv) : > I am on my way to package some source code for OmniOS. > > For one package I have this problem, as seen from the res file: > > dir path=etc/init.d owner=root group=bin mode=0755 > > This is changing the owner of the /etc/init.d dir to bin (should be sys) > and so conflicts with other packages. > > If I try to install this, I get > > pkg install: The requested change to the system attempts to install > multiple actions > for dir 'etc/init.d' with conflicting attributes: > > 1 package delivers 'dir group=bin mode=0755 owner=root > path=etc/init.d': > [...] > 5 packages deliver 'dir group=sys mode=0755 owner=root > path=etc/init.d': > [...] > > Now, I can't find where in the source code this change is happening > (e.g., chgrp'ing or chown'ing the folder), but I don't even know if this > "bin" group comes from some default setting of the packaging template (so > not directly from one of the install scripts in the code). > > My questions are: > > 1) Is there a way to manually edit the res file > (.p5m.int.3.res) and just rebuild the package? The edit would > consist in changing "group=bin" to "group=sys" (I am 100% sure this won't > be a problem for this package). > > 2) If "group=bin" comes from some default, can I override this setting? > > Many thanks! > > -- > Vincenzo Pii > Researcher, InIT Cloud Computing Lab > Zurich University of Applied Sciences (ZHAW) > blog.zhaw.ch/icclab > -- Vincenzo Pii Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) blog.zhaw.ch/icclab -------------- next part -------------- An HTML attachment was scrubbed... URL: From lotheac at iki.fi Wed Dec 3 10:08:11 2014 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Wed, 3 Dec 2014 12:08:11 +0200 Subject: [OmniOS-discuss] packaging - unwanted directory group owner In-Reply-To: References: Message-ID: <20141203100811.GA17394@gutsman.lotheac.fi> On Wed, Dec 03 2014 10:33:30 +0100, Vincenzo Pii wrote: > If I try to install this, I get > > pkg install: The requested change to the system attempts to install > multiple actions > for dir 'etc/init.d' with conflicting attributes: > > 1 package delivers 'dir group=bin mode=0755 owner=root path=etc/init.d': > [...] > 5 packages deliver 'dir group=sys mode=0755 owner=root path=etc/init.d': > [...] pkglint(1) will also warn you about this. I'm under the impression pkglint was integrated into omnios-build:template, so you could also catch this before publishing the package. -- Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet From sim.ple at live.nl Wed Dec 3 10:37:31 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 11:37:31 +0100 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: , <, <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>, , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, Message-ID: Anybody else had problems with two or more sas hba's in an omnios system ? How was this solved? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Hi, was indeed intended for the list. dmesg | grep mptsas gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Wed Dec 3 20:28:47 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Thu, 4 Dec 2014 06:28:47 +1000 Subject: [OmniOS-discuss] help with ipfilter rule for DHCP client Message-ID: <20141204062847.6d1b24d9@punda-mlia> Hello, I tightened up my ipfilter rules and somehow cut out DHCP. Could someone who knows DHCP better than me please have a look and tell me what's missing. I pulled ipf.conf from an Oracle page so it must be slightly different in some respect. What I have now is pass in on e1000g1 all pass in on e1000g0 all pass out on e1000g1 all pass out on e1000g0 all where: e1000g0 is connected to my ISP's cable modem and picks up its address via DHCP in the range 192.168.0.2 to 192.168.0.254 e1000g1 is the local network interface, statically set to 192.168.1.1 It was working after I changed the rules to those below, even after restarting service ipfilter, but I suppose it didn't have to renew the lease at that time. Dec 3 16:47:34 world /sbin/dhcpagent[106]: [ID 490758 daemon.error] send_pkt_internal: cannot send REQUEST packet to server (will retry in 63 seconds): Network is unreachable Dec 3 16:48:34 world ipf: [ID 774698 kern.info] IP Filter: v4.1.9, running. Dec 3 16:48:39 world /sbin/dhcpagent[106]: [ID 778557 daemon.warning] configure_v4_lease: no IP broadcast specified for e1000g0, making best guess The rules were as at http://pastebin.com/4aYyZhJ8 -- posted there to avoid a long message. Can I also suggest that for the sake of anyone searching for this later, that you copy and repost any relevant lines from the rule set. Thanks in anticipation, Michael. From sim.ple at live.nl Wed Dec 3 10:59:11 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 11:59:11 +0100 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: , <,,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>,<>>, , , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, , , Message-ID: when I do cfgadm -al Ap_Id Type Receptacle Occupant Condition c7 scsi-sas connected configured unknown c7::dsk/c7t500000E118E31592d0 disk connected configured unknown c9 scsi-sas connected configured unknown c9::dsk/c9t5000C50055B2A5B1d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A9A9d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A139d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A245d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B081d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B249d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B255d0 disk connected configured unknown c9::dsk/c9t5000C50055B2BD6Dd0 disk connected configured unknown c9::es/ses0 ESI connected configured unknown c9::es/ses1 ESI connected configured unknown c9::es/ses2 ESI connected configured unknown c9::smp/expd0 smp connected configured unknown c9::smp/expd1 smp connected configured unknown c9::smp/expd2 smp connected configured unknown c10 scsi-sas connected configured unknown c10::dsk/c10t5001517BB27F79F7 disk connected configured unknown c11 scsi-sas connected unconfigured unknown usb2/1 usb-hub connected configured ok usb2/1.1 unknown empty unconfigured ok usb2/1.2 unknown empty unconfigured ok usb2/1.3 unknown empty unconfigured ok usb2/1.4 unknown empty unconfigured ok usb2/1.5 unknown empty unconfigured ok usb2/1.6 unknown empty unconfigured ok usb2/2 unknown empty unconfigured ok usb3/1 usb-hub connected configured ok usb3/1.1 unknown empty unconfigured ok usb3/1.2 unknown empty unconfigured ok usb3/1.3 unknown empty unconfigured ok usb3/1.4 usb-device connected configured ok usb3/1.5 unknown empty unconfigured ok usb3/1.6 unknown empty unconfigured ok usb3/1.7 unknown empty unconfigured ok usb3/1.8 unknown empty unconfigured ok usb3/2 unknown empty unconfigured ok i see c11 as unconfigured. Could this be the cause ? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 11:37:31 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Anybody else had problems with two or more sas hba's in an omnios system ? How was this solved? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Hi, was indeed intended for the list. dmesg | grep mptsas gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Wed Dec 3 12:05:46 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 3 Dec 2014 22:05:46 +1000 Subject: [OmniOS-discuss] help with ipfilter rule for DHCP client In-Reply-To: <20141204062847.6d1b24d9@punda-mlia> References: <20141204062847.6d1b24d9@punda-mlia> Message-ID: <20141203220546.1eed4dcb@punda-mlia> Answering my own question: # Allow access to ISP's specified DHCP server for cable or DSL networks. # The first rule obtains the IP of the server. # The second obtains an address from it. pass out log quick on e1000g0 proto udp from any to any port = 67 keep state pass out quick on e1000g0 proto udp from any to 192.168.0.1 port = 67 keep state How about that? Anything better? From jimklimov at cos.ru Wed Dec 3 12:02:26 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Wed, 03 Dec 2014 13:02:26 +0100 Subject: [OmniOS-discuss] KVM compatibility with Linux Message-ID: <9BA9C418-73C1-4AB1-82D0-AA1A99F48594@cos.ru> A team I know uses a Linux server hosting some KVM machines for continuous integration and testing of their work. Now there is discussion about backing it up and/or replicating, and I proposed to set up an omnios box for dependable storage and VM capabilities (at least vbox, or maybe kvm). A question popped up - how comparable (features, hw support) and compatible (copy-paste the disk image and config) are the kvm implementations in linux and illumos/omnios today? And what is the state of libvirt in illumos/omnios regarding management of VMs and similar environments (zones, vboxes, maybe BEs, etc.)? Thanks in advance, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From sim.ple at live.nl Wed Dec 3 12:44:35 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 13:44:35 +0100 Subject: [OmniOS-discuss] Upgrading from OI to OmniOS In-Reply-To: References: <547DD116.7040401@blue-bolt.com>, Message-ID: should have send this to the list.... again.... ;-) Hi, The safest way I could think of to do this is as follows: create a bootenv in OI en install Omnios fresh into this Start omnios and if you need any settings you can mount the be of OI to e.g. /mnt and copy the config files from there to the Omnios install. I had to keep track of config files in /etc and /kernel/drv I have noticed that both oses are quite compatible Above procedure can first be tested with vm's Do not upgrade any pools untill you are satisfied with the new situation since there might be a diff in zfs version. If you do you won't be able to go back to your OI install. Please first test with vm's! Rgds R > Date: Tue, 2 Dec 2014 14:47:50 +0000 > From: cal-s at blue-bolt.com > To: omnios-discuss at lists.omniti.com > Subject: [OmniOS-discuss] Upgrading from OI to OmniOS > > Hi > > I have a server hosting 65TB of RAIDZ2 storage that's currently running > OI (oi_151a8), which i really want to get off of given that OI has > bitten the dust support-wise. I found one brief discussion on the topic > but it wasn't terribly detailed and was missing critical steps like > configuration protection/retention. I'd like to preserve network and esp > LDAP (client) settings as they were kind of a pain to establish in the > first instance for quick reapplication. If anyone has or is interested > in cobbling together a checklist of steps to upgrade OI to OmniOS that > takes configuration into account, i'd very much appreciate seeing it. > > thank you > > - cal sawyer > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.sproul at circonus.com Wed Dec 3 14:49:04 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Wed, 3 Dec 2014 09:49:04 -0500 Subject: [OmniOS-discuss] packaging - unwanted directory group owner In-Reply-To: References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch> Message-ID: On Wed, Dec 3, 2014 at 4:48 AM, Vincenzo Pii wrote: > Ok, problem solved :D! > > I added a "local.mog" file to my build directory (besides build.sh), with > the following content: > > set group sys> > > I realized I could do that by browsing the omnios-build repo, for the bash > package: > > https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog Vincenzo, If you're interested, the IPS Developer's Guide, linked from http://omnios.omniti.com/wiki.php/MoreInfo was very helpful to me when I was first learning how to package for IPS. Among many other things, it describes the various ways you can programmatically transform pkg manifests to fine-tune your packages. Eric From filip.marvan at aira.cz Wed Dec 3 14:53:36 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Wed, 3 Dec 2014 15:53:36 +0100 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com> Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local> Hi, thank you for clarification. Yes, it is possible, that these disk was used in some ZFS pool before, they aren't new and I don't know their history. So I checked both, c3t2d0 and c3t5d0 disks (disks, that are "online" in importable "tank" pool) and I found nothing strange. Partition Status Type Start End Length % ========= ====== ============ ===== === ====== === 1 EFI 0 60800 60801 100 There is only one partition on both disk, which should be online and used by my "raid_10" pool (as I posted before). So do you have any idea, where on disks are stored information, that they are part of some another pool and how can get rid of it? Thank you very much, Filip -----Original Message----- From: Richard Elling [mailto:richard.elling at richardelling.com] Sent: Tuesday, December 02, 2014 6:38 PM To: Filip Marvan Cc: Zach Malone; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] zpool import strange informations On Dec 2, 2014, at 7:20 AM, Filip Marvan wrote: > Yes, you are probably right. But I'm thinking, why that information > wasn't overwriten when I add whole c3t2d0 to a new mirror? p0 is an fdisk partition for the whole disk. c3t2d0 is really a shorthand notation for c3t2d0s0. c3t2d0 is also in a fdisk partition of type "Solaris2" So you'll need to check your partitioning and see where exactly the blocks are allocated to solve the mystery. > Is it possible to use > c3t2d0 in one pool, and c3t2d0p0 in another? In general, no. The p0 fdisk partition overlaps whatever fdisk partition is being used for c3t2d0 -- an invitation to data corruption. In general, it is considered bad practice to use p0 (or any p*) for ZFS pool creation. There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you do this consistently, then you'll never have partition nightmares. HTH -- richard > > Thank you, > Filip > > -----Original Message----- > From: Zach Malone [mailto:zmalone at omniti.com] > Sent: Tuesday, December 02, 2014 4:12 PM > To: Filip Marvan > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] zpool import strange informations > > If I was to guess, c3t2d0p0 was once a part of another pool called > "tank", and when you do an import, it's attempting to reassemble the > rest of tank, and sees that a drive named c3t5d0 used to be in the pool it knew as tank. > > Tank is a pretty standard pool name, so I'm guessing the drive used to > be in another system, or the system itself used to be running a different OS. > --Zach > > On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan wrote: >> Hi, >> >> >> >> I have one storage server on OmniOS 12 with one data pool: >> >> >> >> pool: raid_10 >> >> state: ONLINE >> >> scan: none requested >> >> config: >> >> >> >> NAME STATE READ WRITE CKSUM >> >> raid_10 ONLINE 0 0 0 >> >> mirror-0 ONLINE 0 0 0 >> >> c3t2d0 ONLINE 0 0 0 >> >> c3t3d0 ONLINE 0 0 0 >> >> mirror-1 ONLINE 0 0 0 >> >> c3t4d0 ONLINE 0 0 0 >> >> c3t5d0 ONLINE 0 0 0 >> >> logs >> >> c3t0d0 ONLINE 0 0 0 >> >> >> >> errors: No known data errors >> >> >> >> pool: rpool >> >> state: ONLINE >> >> scan: none requested >> >> config: >> >> >> >> NAME STATE READ WRITE CKSUM >> >> rpool ONLINE 0 0 0 >> >> c3t1d0s0 ONLINE 0 0 0 >> >> >> >> errors: No known data errors >> >> >> >> There are no other disks in this storage server. Bud if I write >> command zpool import, I can see: >> >> pool: tank >> >> id: 4916680739993977334 >> >> state: UNAVAIL >> >> status: The pool was last accessed by another system. >> >> action: The pool cannot be imported due to damaged devices or data. >> >> see: http://illumos.org/msg/ZFS-8000-EY >> >> config: >> >> >> >> tank UNAVAIL insufficient replicas >> >> raidz2-1 UNAVAIL insufficient replicas >> >> /dev/label/mfront8 UNAVAIL cannot open >> >> /dev/label/mfront9 UNAVAIL cannot open >> >> /dev/label/mfront10 UNAVAIL cannot open >> >> c3t2d0p0 ONLINE >> >> /dev/label/mfront12 UNAVAIL cannot open >> >> /dev/label/mfront13 UNAVAIL cannot open >> >> /dev/label/mfront14 UNAVAIL cannot open >> >> c3t5d0 ONLINE >> >> >> >> I never created such "tank" pool. Where are stored informations about >> this (un)importable pool, how can I delete them? Why c3t5d0 seems to >> be as part of two different pools at the same time? >> >> >> >> Thank you for any help. >> >> Filip Marvan >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6247 bytes Desc: not available URL: From piiv at zhaw.ch Wed Dec 3 14:54:09 2014 From: piiv at zhaw.ch (Vincenzo Pii) Date: Wed, 3 Dec 2014 15:54:09 +0100 Subject: [OmniOS-discuss] packaging - unwanted directory group owner In-Reply-To: <75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch> References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch> <75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch> Message-ID: Wow, many thanks Eric. Could someone with super powers add the link in [1] at the end of [2]? I wish that link was there since 2 days ago :D! (I cannot edit the wiki.) [1] http://omnios.omniti.com/wiki.php/MoreInfo#IPSpkg5 [2] http://omnios.omniti.com/wiki.php/PackagingForOmniOS 2014-12-03 15:49 GMT+01:00 Eric Sproul : > On Wed, Dec 3, 2014 at 4:48 AM, Vincenzo Pii wrote: > > Ok, problem solved :D! > > > > I added a "local.mog" file to my build directory (besides build.sh), with > > the following content: > > > > set group sys> > > > > I realized I could do that by browsing the omnios-build repo, for the > bash > > package: > > > > > https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog > > Vincenzo, > If you're interested, the IPS Developer's Guide, linked from > http://omnios.omniti.com/wiki.php/MoreInfo was very helpful to me when > I was first learning how to package for IPS. Among many other things, > it describes the various ways you can programmatically transform pkg > manifests to fine-tune your packages. > > Eric > -- Vincenzo Pii Researcher, InIT Cloud Computing Lab Zurich University of Applied Sciences (ZHAW) blog.zhaw.ch/icclab -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.sproul at circonus.com Wed Dec 3 15:02:59 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Wed, 3 Dec 2014 10:02:59 -0500 Subject: [OmniOS-discuss] packaging - unwanted directory group owner In-Reply-To: References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch> <75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch> Message-ID: On Wed, Dec 3, 2014 at 9:54 AM, Vincenzo Pii wrote: > Wow, many thanks Eric. > > Could someone with super powers add the link in [1] at the end of [2]? I > wish that link was there since 2 days ago :D! > (I cannot edit the wiki.) > > [1] http://omnios.omniti.com/wiki.php/MoreInfo#IPSpkg5 > [2] http://omnios.omniti.com/wiki.php/PackagingForOmniOS Done. I put it at the top. :) From sim.ple at live.nl Wed Dec 3 16:11:32 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 17:11:32 +0100 Subject: [OmniOS-discuss] test omnios on OI machine / solved In-Reply-To: References: , , <, , <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>, <>>, , , , , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, , , , , , Message-ID: As a last resort I just turned off the machine and kept it off for a few minutes. Turned it back on and both controllers are now properly detected. R From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 11:59:11 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine when I do cfgadm -al Ap_Id Type Receptacle Occupant Condition c7 scsi-sas connected configured unknown c7::dsk/c7t500000E118E31592d0 disk connected configured unknown c9 scsi-sas connected configured unknown c9::dsk/c9t5000C50055B2A5B1d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A9A9d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A139d0 disk connected configured unknown c9::dsk/c9t5000C50055B2A245d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B081d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B249d0 disk connected configured unknown c9::dsk/c9t5000C50055B2B255d0 disk connected configured unknown c9::dsk/c9t5000C50055B2BD6Dd0 disk connected configured unknown c9::es/ses0 ESI connected configured unknown c9::es/ses1 ESI connected configured unknown c9::es/ses2 ESI connected configured unknown c9::smp/expd0 smp connected configured unknown c9::smp/expd1 smp connected configured unknown c9::smp/expd2 smp connected configured unknown c10 scsi-sas connected configured unknown c10::dsk/c10t5001517BB27F79F7 disk connected configured unknown c11 scsi-sas connected unconfigured unknown usb2/1 usb-hub connected configured ok usb2/1.1 unknown empty unconfigured ok usb2/1.2 unknown empty unconfigured ok usb2/1.3 unknown empty unconfigured ok usb2/1.4 unknown empty unconfigured ok usb2/1.5 unknown empty unconfigured ok usb2/1.6 unknown empty unconfigured ok usb2/2 unknown empty unconfigured ok usb3/1 usb-hub connected configured ok usb3/1.1 unknown empty unconfigured ok usb3/1.2 unknown empty unconfigured ok usb3/1.3 unknown empty unconfigured ok usb3/1.4 usb-device connected configured ok usb3/1.5 unknown empty unconfigured ok usb3/1.6 unknown empty unconfigured ok usb3/1.7 unknown empty unconfigured ok usb3/1.8 unknown empty unconfigured ok usb3/2 unknown empty unconfigured ok i see c11 as unconfigured. Could this be the cause ? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 11:37:31 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Anybody else had problems with two or more sas hba's in an omnios system ? How was this solved? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Hi, was indeed intended for the list. dmesg | grep mptsas gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.elling at richardelling.com Wed Dec 3 16:58:43 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 3 Dec 2014 08:58:43 -0800 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: , <, <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>, , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, Message-ID: I'm glad you got it running ok. Helpful hint below... On Dec 3, 2014, at 2:37 AM, Randy S wrote: > > Anybody else had problems with two or more sas hba's in an omnios system ? > How was this solved? > > From: sim.ple at live.nl > To: omnios-discuss at lists.omniti.com > Date: Wed, 3 Dec 2014 01:45:12 +0100 > Subject: Re: [OmniOS-discuss] test omnios on OI machine > > Hi, was indeed intended for the list. > > dmesg | grep mptsas dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg): /usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \ /var/adm/messages | /usr/bin/tail -200 to show you the last 200 lines in the messages file. For a large system, there can easily be 1000+ lines of boot-time messages. You're better off looking at the messages file directly: grep mptsas /var/adm/messages For a better method of identifying loaded drivers, use prtconf: prtconf -D shows the device tree and the attached drivers. HTH -- richard > > gives nothing (no output). > > Hope you have an idea of what to do next. > > Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. > > Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > > > > > Subject: Re: [OmniOS-discuss] test omnios on OI machine > > From: danmcd at omniti.com > > Date: Tue, 2 Dec 2014 19:22:15 -0500 > > CC: danmcd at omniti.com > > To: sim.ple at live.nl > > > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > > > Hi Dan, > > > > > > It's been a while... > > > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > > > forgot to mention, I did do an fmdump -ev and saw > > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > > > I guess this is because the one controller cannot be found. > > > > That's a fair assessment. > > > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > > > Dan > > > > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.elling at richardelling.com Wed Dec 3 17:02:08 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 3 Dec 2014 09:02:08 -0800 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local> Message-ID: <6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com> On Dec 3, 2014, at 6:53 AM, Filip Marvan wrote: > Hi, > > thank you for clarification. Yes, it is possible, that these disk was used > in some ZFS pool before, they aren't new and I don't know their history. > > So I checked both, c3t2d0 and c3t5d0 disks (disks, that are "online" in > importable "tank" pool) and I found nothing strange. > > Partition Status Type Start End Length % > ========= ====== ============ ===== === ====== === > 1 EFI 0 60800 60801 100 > > There is only one partition on both disk, which should be online and used by > my "raid_10" pool (as I posted before). So do you have any idea, where on > disks are stored information, that they are part of some another pool and > how can get rid of it? Try this instead: for i in /dev/dsk/c3t2d0* do echo $i zdb -l $i done This will show you the ZFS labels on each and every slice/partition for the disk. Ideally, you'll only see one set of ZFS labels for each disk. -- richard > > Thank you very much, > Filip > > > -----Original Message----- > From: Richard Elling [mailto:richard.elling at richardelling.com] > Sent: Tuesday, December 02, 2014 6:38 PM > To: Filip Marvan > Cc: Zach Malone; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] zpool import strange informations > > > On Dec 2, 2014, at 7:20 AM, Filip Marvan wrote: > >> Yes, you are probably right. But I'm thinking, why that information >> wasn't overwriten when I add whole c3t2d0 to a new mirror? > > p0 is an fdisk partition for the whole disk. > c3t2d0 is really a shorthand notation for c3t2d0s0. > c3t2d0 is also in a fdisk partition of type "Solaris2" > > So you'll need to check your partitioning and see where exactly the blocks > are allocated to solve the mystery. > >> Is it possible to use >> c3t2d0 in one pool, and c3t2d0p0 in another? > > In general, no. The p0 fdisk partition overlaps whatever fdisk partition is > being used for c3t2d0 -- an invitation to data corruption. > > In general, it is considered bad practice to use p0 (or any p*) for ZFS pool > creation. > There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you > do this consistently, then you'll never have partition nightmares. > > HTH > -- richard > >> >> Thank you, >> Filip >> >> -----Original Message----- >> From: Zach Malone [mailto:zmalone at omniti.com] >> Sent: Tuesday, December 02, 2014 4:12 PM >> To: Filip Marvan >> Cc: omnios-discuss at lists.omniti.com >> Subject: Re: [OmniOS-discuss] zpool import strange informations >> >> If I was to guess, c3t2d0p0 was once a part of another pool called >> "tank", and when you do an import, it's attempting to reassemble the >> rest of tank, and sees that a drive named c3t5d0 used to be in the pool it > knew as tank. >> >> Tank is a pretty standard pool name, so I'm guessing the drive used to >> be in another system, or the system itself used to be running a different > OS. >> --Zach >> >> On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan wrote: >>> Hi, >>> >>> >>> >>> I have one storage server on OmniOS 12 with one data pool: >>> >>> >>> >>> pool: raid_10 >>> >>> state: ONLINE >>> >>> scan: none requested >>> >>> config: >>> >>> >>> >>> NAME STATE READ WRITE CKSUM >>> >>> raid_10 ONLINE 0 0 0 >>> >>> mirror-0 ONLINE 0 0 0 >>> >>> c3t2d0 ONLINE 0 0 0 >>> >>> c3t3d0 ONLINE 0 0 0 >>> >>> mirror-1 ONLINE 0 0 0 >>> >>> c3t4d0 ONLINE 0 0 0 >>> >>> c3t5d0 ONLINE 0 0 0 >>> >>> logs >>> >>> c3t0d0 ONLINE 0 0 0 >>> >>> >>> >>> errors: No known data errors >>> >>> >>> >>> pool: rpool >>> >>> state: ONLINE >>> >>> scan: none requested >>> >>> config: >>> >>> >>> >>> NAME STATE READ WRITE CKSUM >>> >>> rpool ONLINE 0 0 0 >>> >>> c3t1d0s0 ONLINE 0 0 0 >>> >>> >>> >>> errors: No known data errors >>> >>> >>> >>> There are no other disks in this storage server. Bud if I write >>> command zpool import, I can see: >>> >>> pool: tank >>> >>> id: 4916680739993977334 >>> >>> state: UNAVAIL >>> >>> status: The pool was last accessed by another system. >>> >>> action: The pool cannot be imported due to damaged devices or data. >>> >>> see: http://illumos.org/msg/ZFS-8000-EY >>> >>> config: >>> >>> >>> >>> tank UNAVAIL insufficient replicas >>> >>> raidz2-1 UNAVAIL insufficient replicas >>> >>> /dev/label/mfront8 UNAVAIL cannot open >>> >>> /dev/label/mfront9 UNAVAIL cannot open >>> >>> /dev/label/mfront10 UNAVAIL cannot open >>> >>> c3t2d0p0 ONLINE >>> >>> /dev/label/mfront12 UNAVAIL cannot open >>> >>> /dev/label/mfront13 UNAVAIL cannot open >>> >>> /dev/label/mfront14 UNAVAIL cannot open >>> >>> c3t5d0 ONLINE >>> >>> >>> >>> I never created such "tank" pool. Where are stored informations about >>> this (un)importable pool, how can I delete them? Why c3t5d0 seems to >>> be as part of two different pools at the same time? >>> >>> >>> >>> Thank you for any help. >>> >>> Filip Marvan >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > From sim.ple at live.nl Wed Dec 3 17:04:43 2014 From: sim.ple at live.nl (Randy S) Date: Wed, 3 Dec 2014 18:04:43 +0100 Subject: [OmniOS-discuss] test omnios on OI machine In-Reply-To: References: , <, <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>, , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, , Message-ID: Thank you very much for the hint, indeed helpful. Subject: Re: [OmniOS-discuss] test omnios on OI machine From: richard.elling at richardelling.com Date: Wed, 3 Dec 2014 08:58:43 -0800 CC: omnios-discuss at lists.omniti.com To: sim.ple at live.nl I'm glad you got it running ok. Helpful hint below... On Dec 3, 2014, at 2:37 AM, Randy S wrote: Anybody else had problems with two or more sas hba's in an omnios system ? How was this solved? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Hi, was indeed intended for the list. dmesg | grep mptsas dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg): /usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \ /var/adm/messages | /usr/bin/tail -200 to show you the last 200 lines in the messages file. For a large system, there can easily be 1000+ lines of boot-time messages. You're better off looking at the messagesfile directly: grep mptsas /var/adm/messages For a better method of identifying loaded drivers, use prtconf: prtconf -Dshows the device tree and the attached drivers. HTH -- richard gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at linuxsuite.org Wed Dec 3 17:06:40 2014 From: steve at linuxsuite.org (steve at linuxsuite.org) Date: Wed, 3 Dec 2014 12:06:40 -0500 Subject: [OmniOS-discuss] Missing device but format shows it exists? Message-ID: <5ec7cb4824606ee7cd7c719281608b1c.squirrel@emailmg.netfirms.com> Howdy! This seems wierd.... I am adding cache and log mirrors and everything works fine but then their is a missing device, but format shows the partition is there. devfsadmn does not add the device file. d0s0 - d0s6 and d0s8-d015 are there but d0s7 is missing? thoughts? -steve # zpool add dfs2_data8 cache c1t5d0s7 cannot open 'c1t5d0s7': no such device in /dev/dsk must be a full path or shorthand device name root at live-dfs-2:~# ls /dev/dsk/c1t5d0s7 ls: cannot access /dev/dsk/c1t5d0s7: No such file or directory root at live-dfs-2:~# ls /dev/dsk/c1t5d0s* /dev/dsk/c1t5d0s0 /dev/dsk/c1t5d0s1 /dev/dsk/c1t5d0s2 /dev/dsk/c1t5d0s3 /dev/dsk/c1t5d0s4 /dev/dsk/c1t5d0s5 /dev/dsk/c1t5d0s6 /dev/dsk/c1t5d0s8 /dev/dsk/c1t5d0s9 /dev/dsk/c1t5d0s10 /dev/dsk/c1t5d0s11 /dev/dsk/c1t5d0s12 /dev/dsk/c1t5d0s13 /dev/dsk/c1t5d0s14 /dev/dsk/c1t5d0s15 # format -e Specify disk (enter its number): 4 selecting c1t5d0 [disk formatted] /dev/dsk/c1t5d0s0 is in use as a cache device for ZFS pool dfs2_data1. Please see zpool(1M). /dev/dsk/c1t5d0s1 is in use as a cache device for ZFS pool dfs2_data2. Please see zpool(1M). /dev/dsk/c1t5d0s2 is in use as a cache device for ZFS pool dfs2_data3. Please see zpool(1M). /dev/dsk/c1t5d0s3 is in use as a cache device for ZFS pool dfs2_data4. Please see zpool(1M). /dev/dsk/c1t5d0s4 is in use as a cache device for ZFS pool dfs2_data5. Please see zpool(1M). /dev/dsk/c1t5d0s5 is in use as a cache device for ZFS pool dfs2_data6. Please see zpool(1M). /dev/dsk/c1t5d0s6 is in use as a cache device for ZFS pool dfs2_data7. Please see zpool(1M). format> ver Volume name = < > ascii name = bytes/sector = 512 sectors = 233308159 accessible sectors = 233308126 Part Tag Flag First Sector Size Last Sector 0 usr wm 256 12.00GB 25166079 1 usr wm 25166080 12.00GB 50331903 2 usr wm 50331904 12.00GB 75497727 3 usr wm 75497728 12.00GB 100663551 4 usr wm 100663552 12.00GB 125829375 5 usr wm 125829376 12.00GB 150995199 6 usr wm 150995200 12.00GB 176161023 7 usr wm 176161024 12.00GB 201326847 8 reserved wm 233291743 8.00MB 233308126 format> From jboren at drakecooper.com Thu Dec 4 00:10:29 2014 From: jboren at drakecooper.com (Joseph Boren) Date: Wed, 3 Dec 2014 17:10:29 -0700 Subject: [OmniOS-discuss] anyone doing DirectPath I/O? Message-ID: Greetings, I was wondering if anyone was using DirectPath, specifically for exclusive use of a drive controller and attached drives to a specific VM. I have a use case that would seem to be a good fit for this, so I played around with a couple of RAID controllers I had, and was able to get one (3ware 9650se) configured for directpath, but none of the drives attached would show to OmniOS, regardless of how I configured them in the raid bois (JBOD, individual disks, etc). I know that controller is poorly supported and I was curious if anyone was using DirectPath this way in production and what kind of drive controller/HBA/whatever was working. Also any "for the love of god don't do it this way" scenarios? I seem to be really adept at finding and trying those out first.... Thanks a ton and best regards, -jb- *Joseph Boren* IT Specialist *DRAKE COOPER* + c: (208) 891-2128 + o: (208) 342-0925 + 416 S. 8th St., Boise, ID 83702 + w: drakecooper.com + f: /drakecooper + t: @drakecooper -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Thu Dec 4 03:01:51 2014 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 3 Dec 2014 21:01:51 -0600 Subject: [OmniOS-discuss] anyone doing DirectPath I/O? In-Reply-To: References: Message-ID: I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years to an OpenIndiana server. It's been very stable: root at zfs01:~# uptime 21:01pm up 649 days 6:58, 2 users, load average: 0.46, 0.30, 0.25 On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren wrote: > Greetings, > > I was wondering if anyone was using DirectPath, specifically for exclusive > use of a drive controller and attached drives to a specific VM. I have a > use case that would seem to be a good fit for this, so I played around with > a couple of RAID controllers I had, and was able to get one (3ware 9650se) > configured for directpath, but none of the drives attached would show to > OmniOS, regardless of how I configured them in the raid bois (JBOD, > individual disks, etc). I know that controller is poorly supported and I > was curious if anyone was using DirectPath this way in production and what > kind of drive controller/HBA/whatever was working. Also any "for the love > of god don't do it this way" scenarios? I seem to be really adept at > finding and trying those out first.... > > Thanks a ton and best regards, > > -jb- > *Joseph Boren* > > IT Specialist > *DRAKE COOPER* > + c: (208) 891-2128 + o: (208) 342-0925 > + 416 S. 8th St., Boise, ID 83702 > + w: drakecooper.com + f: /drakecooper + > t: @drakecooper > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Thu Dec 4 05:30:51 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Thu, 4 Dec 2014 15:30:51 +1000 Subject: [OmniOS-discuss] adding cua/a as a second login Message-ID: <20141204153051.3e17ac8f@punda-mlia> There's plenty of stuff about configuring the serial port as the console on login, but I would like to ADD the serial port as a second login; that is, keep the normal console login but have cua/a as a second login access point. Is this possible ? Something to do with SAF on Solaris, IIRC. Michael. From tim at multitalents.net Thu Dec 4 07:21:54 2014 From: tim at multitalents.net (Tim Rice) Date: Wed, 3 Dec 2014 23:21:54 -0800 (PST) Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: <20141204153051.3e17ac8f@punda-mlia> References: <20141204153051.3e17ac8f@punda-mlia> Message-ID: On Thu, 4 Dec 2014, Michael Mounteney wrote: > There's plenty of stuff about configuring the serial port as the > console on login, but I would like to ADD the serial port as a second > login; that is, keep the normal console login but have cua/a as a > second login access point. Is this possible ? Something to do with > SAF on Solaris, IIRC. > > Michael. Yes, read the saf, sacadm, and pmadm man pages. The default install will probably already have a ttymon running. ...... # sacadm -l PMTAG PMTYPE FLGS RCNT STATUS COMMAND zsmon ttymon - 0 ENABLED /usr/lib/saf/ttymon # ...... And here are the ports in zsmon ....... # pmadm -l PMTAG PMTYPE SVCTAG FLGS ID zsmon ttymon ttya u root /dev/term/a I - /usr/bin/login - 9600 ldterm,ttcompat ttya login: - tvi925 y # zsmon ttymon ttyb u root /dev/term/b I - /usr/bin/login - 9600 ldterm,ttcompat ttyb login: - tvi925 y # ....... -- Tim Rice Multitalents (707) 456-1146 tim at multitalents.net From tim at multitalents.net Thu Dec 4 07:11:04 2014 From: tim at multitalents.net (Tim Rice) Date: Wed, 3 Dec 2014 23:11:04 -0800 (PST) Subject: [OmniOS-discuss] anyone doing DirectPath I/O? In-Reply-To: References: Message-ID: On Wed, 3 Dec 2014, Schweiss, Chip wrote: | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years | to an OpenIndiana server. It's been very stable: My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS storage to the ESXi host for (currently) 15 running VMS as well as well as NFS home directories for various other computers around the office. Other than drives failing, it's been working reasonably well for what it is. | | root at zfs01:~# uptime | 21:01pm up 649 days 6:58, 2 users, load average: 0.46, 0.30, 0.25 | | | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren wrote: | | > Greetings, | > | > I was wondering if anyone was using DirectPath, specifically for exclusive | > use of a drive controller and attached drives to a specific VM. I have a | > use case that would seem to be a good fit for this, so I played around with | > a couple of RAID controllers I had, and was able to get one (3ware 9650se) | > configured for directpath, but none of the drives attached would show to | > OmniOS, regardless of how I configured them in the raid bois (JBOD, | > individual disks, etc). I know that controller is poorly supported and I | > was curious if anyone was using DirectPath this way in production and what | > kind of drive controller/HBA/whatever was working. Also any "for the love | > of god don't do it this way" scenarios? I seem to be really adept at | > finding and trying those out first.... | > | > Thanks a ton and best regards, | > | > -jb- | > *Joseph Boren* | > | > IT Specialist | > *DRAKE COOPER* | > + c: (208) 891-2128 + o: (208) 342-0925 | > + 416 S. 8th St., Boise, ID 83702 | > + w: drakecooper.com + f: /drakecooper + | > t: @drakecooper | > | > | > _______________________________________________ | > OmniOS-discuss mailing list | > OmniOS-discuss at lists.omniti.com | > http://lists.omniti.com/mailman/listinfo/omnios-discuss | > | > | -- Tim Rice Multitalents (707) 456-1146 tim at multitalents.net From sjorge+ml at blackdot.be Thu Dec 4 08:35:04 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Thu, 04 Dec 2014 09:35:04 +0100 Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: <20141204153051.3e17ac8f@punda-mlia> References: <20141204153051.3e17ac8f@punda-mlia> Message-ID: Hey, I have this setup on my SuperMicro server to have a 2nd console on the SoL. I documented it here: https://docu.blackdot.be/snipets/solaris/misc-serial-console Hopefully this is helpful. Regards Jorge On 2014-12-04 06:30, Michael Mounteney wrote: > There's plenty of stuff about configuring the serial port as the > console on login, but I would like to ADD the serial port as a second > login; that is, keep the normal console login but have cua/a as a > second login access point. Is this possible ? Something to do with > SAF on Solaris, IIRC. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From sim.ple at live.nl Thu Dec 4 10:51:53 2014 From: sim.ple at live.nl (Randy S) Date: Thu, 4 Dec 2014 11:51:53 +0100 Subject: [OmniOS-discuss] test omnios on OI machine / ehh In-Reply-To: References: , <,,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>,,, , , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, , , , , , Message-ID: Did a cold reboot and controller = seen I did my stuff. After a while I did a devfsadm -Cv , removed path_to_inst and a reboot -- -r and controller = gone It's indeed the c11 (see previous message), prtconf -c configure c11 , does nothing (causes no log messages either). Is there a way to configure this manually without having to reboot? The strange thing is that the OI has no problem with this at all. R From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 18:04:43 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Thank you very much for the hint, indeed helpful. Subject: Re: [OmniOS-discuss] test omnios on OI machine From: richard.elling at richardelling.com Date: Wed, 3 Dec 2014 08:58:43 -0800 CC: omnios-discuss at lists.omniti.com To: sim.ple at live.nl I'm glad you got it running ok. Helpful hint below... On Dec 3, 2014, at 2:37 AM, Randy S wrote: Anybody else had problems with two or more sas hba's in an omnios system ? How was this solved? From: sim.ple at live.nl To: omnios-discuss at lists.omniti.com Date: Wed, 3 Dec 2014 01:45:12 +0100 Subject: Re: [OmniOS-discuss] test omnios on OI machine Hi, was indeed intended for the list. dmesg | grep mptsas dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg): /usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \ /var/adm/messages | /usr/bin/tail -200 to show you the last 200 lines in the messages file. For a large system, there can easily be 1000+ lines of boot-time messages. You're better off looking at the messagesfile directly: grep mptsas /var/adm/messages For a better method of identifying loaded drivers, use prtconf: prtconf -Dshows the device tree and the attached drivers. HTH -- richard gives nothing (no output). Hope you have an idea of what to do next. Going to test a current omnios with one hba by adding another to it to see what happens tomorrow. Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. > Subject: Re: [OmniOS-discuss] test omnios on OI machine > From: danmcd at omniti.com > Date: Tue, 2 Dec 2014 19:22:15 -0500 > CC: danmcd at omniti.com > To: sim.ple at live.nl > > Please share these with the community in the future, unless you're sharing confidential data of some sort. > > > > On Dec 2, 2014, at 6:59 PM, Randy S wrote: > > > > Hi Dan, > > > > It's been a while... > > > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. > > > > forgot to mention, I did do an fmdump -ev and saw > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001 > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801 > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801 > > > > I guess this is because the one controller cannot be found. > > That's a fair assessment. > > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives : > > > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0 > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0 > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0 > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online > > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case? > > Dan > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From sim.ple at live.nl Thu Dec 4 12:16:47 2014 From: sim.ple at live.nl (Randy S) Date: Thu, 4 Dec 2014 13:16:47 +0100 Subject: [OmniOS-discuss] Upgrading from OI to OmniOS In-Reply-To: References: <547DD116.7040401@blue-bolt.com>, , , , <547F4668.9040309@osn.de>, Message-ID: From: sim.ple at live.nl To: jg at osn.de Subject: RE: [OmniOS-discuss] Upgrading from OI to OmniOS Date: Thu, 4 Dec 2014 12:45:10 +0100 Hi, Not a stupid question at all. I fought with this for quite a while. I did it in quite an elaborate way and I am certain this can be done in fewer steps so I hope that one of the guru's in here can step in to correct my way of doing this. Perhaps with a pkg install entire sort of a command ? But anyway the following works for me : On a vm I install the Omnios vm and configure it In this vm I do a beadm create BEname. Do: beadm mount BEname /mnt create a compressed file of this BE tar --create --bzip2 --absolute-names --preserve-permissions --preserve-order --verbose --file=/tmp/omni.tar.bz2 /mnt Copy this to the machine where you want to install to e.g. /tmp On the destination (vm)machine create a BE ( beadm create whatever and also mount it on e.g /mnt) cd to /mnt on the destination and rm -rf * untar the bz2 file (cd /tmp) : tar --extract --bzip2 --absolute-names --preserve-permissions --same-owner --verbose --file=omni.tar.bz2 This will populate the /mnt with the omnios be On this machine (destination) I also do a :;zpool status rpool to identify the disks used by rpool and then do e.g. ls -l /dev/dsk/c4t50014EE002EC010Bd0s0 and register the physical device in the bootenv.rc file of the omnios :;vi /mnt/boot/solaris/bootenv.rc and add e.g. lines like: setprop bootpath /scsi_vhci/disk at g50014ee002ec010b:asetprop altbootpath /scsi_vhci/disk at g50014ee0ad96d82a:a <== in case of mirrorsetprop bootfile kernel/amd64/unix After this do: :;/usr/sbin/devfsadm -r /mnt Now you can reboot into your new Omni bootenv and clean stuff up. Again do not upgrade zfs version until satiesfied. I am certain this can be a lot better but this works 4 me (being an autodidact) HTH And as others usually end their helpful hints,... typo's courtesy of ehh my keyboard... R > Date: Wed, 3 Dec 2014 18:20:40 +0100 > From: jg at osn.de > To: sim.ple at live.nl > Subject: Re: [OmniOS-discuss] Upgrading from OI to OmniOS > > Hi Randy, > > sorry this may be a stupid question, but how can I install > a fresh Omnios in an existing bootenv? Or how do you create > the bootenv? bootadm create ? > > Kind regards, > > - Joerg > > On 03.12.2014 13:44, Randy S wrote: > > should have send this to the list.... again.... ;-) > > > > ------------------------------------------------------------------------ > > > > > > Hi, > > > > The safest way I could think of to do this is as follows: > > create a bootenv in OI en install Omnios fresh into this > > Start omnios and if you need any settings you can mount the be of OI to > > e.g. /mnt and copy the config files from there to the Omnios install. > > > > I had to keep track of config files in /etc and /kernel/drv > > I have noticed that both oses are quite compatible > > > > Above procedure can first be tested with vm's > > > > Do not upgrade any pools untill you are satisfied with the new situation > > since there might be a diff in zfs version. If you do you won't be able > > to go back to your OI install. > > > > Please first test with vm's! > > > > Rgds > > > > R > > > > > > > >> Date: Tue, 2 Dec 2014 14:47:50 +0000 > >> From: cal-s at blue-bolt.com > >> To: omnios-discuss at lists.omniti.com > >> Subject: [OmniOS-discuss] Upgrading from OI to OmniOS > >> > >> Hi > >> > >> I have a server hosting 65TB of RAIDZ2 storage that's currently running > >> OI (oi_151a8), which i really want to get off of given that OI has > >> bitten the dust support-wise. I found one brief discussion on the topic > >> but it wasn't terribly detailed and was missing critical steps like > >> configuration protection/retention. I'd like to preserve network and esp > >> LDAP (client) settings as they were kind of a pain to establish in the > >> first instance for quick reapplication. If anyone has or is interested > >> in cobbling together a checklist of steps to upgrade OI to OmniOS that > >> takes configuration into account, i'd very much appreciate seeing it. > >> > >> thank you > >> > >> - cal sawyer > >> _______________________________________________ > >> OmniOS-discuss mailing list > >> OmniOS-discuss at lists.omniti.com > >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > -- > OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg > Tel: +49 911 39905-0 - Fax: +49 911 39905-55 - http://www.osn.de > HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklimek at gmail.com Thu Dec 4 18:20:04 2014 From: jklimek at gmail.com (John Klimek) Date: Thu, 4 Dec 2014 13:20:04 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? Message-ID: I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client. I'm only using sys (?) authentication and not Kerberos. I've correctly set my NFS domain name but it's only partially working. Here's an example: Server: user 'media', uid 1000. Client: user 'media', uid 2000. If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user. However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid. What can I have setup wrong? I can't find any debugging or logging options for nfsmapid... Also, does the uidmap and gidmap share.nfs options only work for NFSv3? -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Dec 4 18:29:50 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 4 Dec 2014 13:29:50 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: Message-ID: > On Dec 4, 2014, at 1:20 PM, John Klimek wrote: > > I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client. I'm only using sys (?) authentication and not Kerberos. > > I've correctly set my NFS domain name but it's only partially working. > > Here's an example: > > Server: user 'media', uid 1000. > Client: user 'media', uid 2000. > > If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user. However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid. > > What can I have setup wrong? > Are you sharing using zfs(1M) sharing properties? (zfs set sharenfs=on ) You need to make sure you set the uidmap property when you share your filesystem. I believe, you want this: zfs set sharenfs='uidmap=2000:media' Try that out for size. Dan From danmcd at omniti.com Thu Dec 4 18:32:28 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 4 Dec 2014 13:32:28 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: Message-ID: > On Dec 4, 2014, at 1:29 PM, Dan McDonald wrote: > > >> On Dec 4, 2014, at 1:20 PM, John Klimek wrote: >> >> I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client. I'm only using sys (?) authentication and not Kerberos. >> >> I've correctly set my NFS domain name but it's only partially working. >> >> Here's an example: >> >> Server: user 'media', uid 1000. >> Client: user 'media', uid 2000. >> >> If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user. However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid. >> >> What can I have setup wrong? >> > > Are you sharing using zfs(1M) sharing properties? (zfs set sharenfs=on ) > > You need to make sure you set the uidmap property when you share your filesystem. I believe, you want this: > > zfs set sharenfs='uidmap=2000:media' You may need to also specify the IP address of the particular client: zfs set sharenfs='uidmap=2000:media:' Dan From jboren at drakecooper.com Thu Dec 4 18:32:35 2014 From: jboren at drakecooper.com (Joseph Boren) Date: Thu, 4 Dec 2014 11:32:35 -0700 Subject: [OmniOS-discuss] anyone doing DirectPath I/O? In-Reply-To: References: Message-ID: Thanks guys, that's very helpful. Tim that sounds very close to the use-case i had in my head. I've run into a couple of situations where a VM sitting on ESXi on particular hardware was much more performant than the exact same system, bare-metal on the exact same hardware. Once I actually P2V'd a server to a temp location, loaded ESXi on the physical server, migrated the P2V'd guest back and it was crazy how much faster and stabler it was. So now days I tend to want to run ESXi on the bare-metal, but I didn't have any experience with DirectPath, and it looked perfect for this situation. Thanks again gentlemen, much appreciated. Best regards, -jb- *Joseph Boren* IT Specialist *DRAKE COOPER* + c: (208) 891-2128 + o: (208) 342-0925 + 416 S. 8th St., Boise, ID 83702 + w: drakecooper.com + f: /drakecooper + t: @drakecooper On Thu, Dec 4, 2014 at 12:11 AM, Tim Rice wrote: > On Wed, 3 Dec 2014, Schweiss, Chip wrote: > > | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 > years > | to an OpenIndiana server. It's been very stable: > > My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for > about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since > updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS > storage to the ESXi host for (currently) 15 running VMS as well > as well as NFS home directories for various other computers around > the office. Other than drives failing, it's been working reasonably > well for what it is. > > | > | root at zfs01:~# uptime > | 21:01pm up 649 days 6:58, 2 users, load average: 0.46, 0.30, 0.25 > | > | > | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren > wrote: > | > | > Greetings, > | > > | > I was wondering if anyone was using DirectPath, specifically for > exclusive > | > use of a drive controller and attached drives to a specific VM. I > have a > | > use case that would seem to be a good fit for this, so I played around > with > | > a couple of RAID controllers I had, and was able to get one (3ware > 9650se) > | > configured for directpath, but none of the drives attached would show > to > | > OmniOS, regardless of how I configured them in the raid bois (JBOD, > | > individual disks, etc). I know that controller is poorly supported > and I > | > was curious if anyone was using DirectPath this way in production and > what > | > kind of drive controller/HBA/whatever was working. Also any "for the > love > | > of god don't do it this way" scenarios? I seem to be really adept at > | > finding and trying those out first.... > | > > | > Thanks a ton and best regards, > | > > | > -jb- > | > *Joseph Boren* > | > > | > IT Specialist > | > *DRAKE COOPER* > | > + c: (208) 891-2128 + o: (208) 342-0925 > | > + 416 S. 8th St., Boise, ID 83702 > | > + w: drakecooper.com + f: /drakecooper < > http://facebook.com/drakecooper> + > | > t: @drakecooper > | > > | > > | > _______________________________________________ > | > OmniOS-discuss mailing list > | > OmniOS-discuss at lists.omniti.com > | > http://lists.omniti.com/mailman/listinfo/omnios-discuss > | > > | > > | > > -- > Tim Rice Multitalents (707) 456-1146 > tim at multitalents.net > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Thu Dec 4 18:37:11 2014 From: mir at miras.org (Michael Rasmussen) Date: Thu, 4 Dec 2014 19:37:11 +0100 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: Message-ID: <20141204193711.03524183@sleipner.datanom.net> On Thu, 4 Dec 2014 13:20:04 -0500 John Klimek wrote: > > What can I have setup wrong? > > I can't find any debugging or logging options for nfsmapid... > > Also, does the uidmap and gidmap share.nfs options only work for NFSv3? Solaris and derivatives implementation of NFS ACL is not compliant to the Linux NFS ACL. More directly it is the ACL in Linux which is not POSIX conformant so to avoid problems you should add the mount option noacl in your Linux fstab file. Noacl will instruct Omnios NFS to revert to plain old uid/gid. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Don't just echo the code with comments - make every comment count. - The Elements of Programming Style (Kernighan & Plaugher) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From jklimek at gmail.com Thu Dec 4 18:55:11 2014 From: jklimek at gmail.com (John Klimek) Date: Thu, 4 Dec 2014 13:55:11 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <20141204193711.03524183@sleipner.datanom.net> References: <20141204193711.03524183@sleipner.datanom.net> Message-ID: Thanks, I will give this a try. However, when I create a file if I check the acl on Omni, it does have extended permissions and looks correct. It's only the uid and gid that I'm trying to get mapped correctly. Do you still think that noacl will solve that problem? (I'm migrating some virtual machines so I can't try out the setting you mentioned until a few more minutes...) On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen wrote: > On Thu, 4 Dec 2014 13:20:04 -0500 > John Klimek wrote: > > > > > What can I have setup wrong? > > > > I can't find any debugging or logging options for nfsmapid... > > > > Also, does the uidmap and gidmap share.nfs options only work for NFSv3? > Solaris and derivatives implementation of NFS ACL is not compliant to the > Linux NFS ACL. > More directly it is the ACL in Linux which is not POSIX conformant so to > avoid problems you > should add the mount option noacl in your Linux fstab file. Noacl will > instruct Omnios NFS > to revert to plain old uid/gid. > > -- > Hilsen/Regards > Michael Rasmussen > > Get my public GnuPG keys: > michael rasmussen cc > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > mir datanom net > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > mir miras org > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > -------------------------------------------------------------- > /usr/games/fortune -es says: > Don't just echo the code with comments - make every comment count. > - The Elements of Programming Style (Kernighan & Plaugher) > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mir at miras.org Thu Dec 4 19:11:07 2014 From: mir at miras.org (Michael Rasmussen) Date: Thu, 4 Dec 2014 20:11:07 +0100 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> Message-ID: <20141204201107.43ad5928@sleipner.datanom.net> On Thu, 4 Dec 2014 13:55:11 -0500 John Klimek wrote: > Thanks, I will give this a try. > > However, when I create a file if I check the acl on Omni, it does have > extended permissions and looks correct. It's only the uid and gid that I'm > trying to get mapped correctly. > > Do you still think that noacl will solve that problem? > Yes, because you want to avoid Omnios presents ACL which is incompatible with Linux ACL. Read my explanation of fix here: http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-Share?p=81530#post81530 -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Fairlight: udp is the light margarine of tcp/ip transport protocols :) -- Seen on #Linux -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From henson at acm.org Thu Dec 4 20:38:19 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 4 Dec 2014 12:38:19 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <20141204193711.03524183@sleipner.datanom.net> References: <20141204193711.03524183@sleipner.datanom.net> Message-ID: <4ce401d01002$3efbe920$bcf3bb60$@acm.org> > From: Michael Rasmussen > Sent: Thursday, December 04, 2014 10:37 AM > > > Also, does the uidmap and gidmap share.nfs options only work for NFSv3? > Solaris and derivatives implementation of NFS ACL is not compliant to the > Linux NFS ACL. Linux supports NFSv4 ACLs perfectly fine, unless you have some requirement to use an older version of the protocol, just mount with v4. You can use the nfs4_editfacl, nfs4_getfacl, and nfs4_setfacl commands under linux to manipulate ZFS ACLs over NFS. From danmcd at omniti.com Thu Dec 4 20:50:29 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 4 Dec 2014 15:50:29 -0500 Subject: [OmniOS-discuss] OmniOS Bloody December 4th update Message-ID: <879935CC-27DE-4F12-BB23-50DE4AC815B2@omniti.com> New with this release (r151013-20141204): - No change on the omnios-build master branch, still revision 31f07b8 - illumos-omnios master branch, revision aeccc33 - Many man page updates. - TCP change that will positively affect off-link-peer performance (see illumos 5295) - ZFS improvements and bugfixes, including better metadata caching in the ARC. - mpt_sas(7d) now uses 64-bit DMA. - SMB/CIFS bugfixes - Built in Sun SSH supports EOW extension. Install media and Kayak root & miniroot are also available. Dan From henson at acm.org Thu Dec 4 20:59:56 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 4 Dec 2014 12:59:56 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <20141204201107.43ad5928@sleipner.datanom.net> References: <20141204193711.03524183@sleipner.datanom.net> <20141204201107.43ad5928@sleipner.datanom.net> Message-ID: <4ced01d01005$44363af0$cca2b0d0$@acm.org> > From: Michael Rasmussen > Sent: Thursday, December 04, 2014 11:11 AM > > Yes, because you want to avoid Omnios presents ACL which is incompatible > with Linux ACL. I don't believe the ACL has anything to do with NFSv4 id mapping? And a ZFS ACL presented over NFSv4 is perfectly compatible with Linux. It's not a Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need to use the nfs4 ACL tools, but it works fine. > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS- > Share?p=81530#post81530 In that thread, the user fails to chmod via NFS: chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': Operation not permitted The root cause of which was a setting of restricted for aclmode: vdev1/proxCT aclmode restricted local Per the man page "An aclmode property of restricted will cause the chmod(2) operation to return an error when used on any file or directory which has a non-trivial ACL whose entries can not be represented by a mode." The user could have set the inherited ACL on the initial filesystem to a trivial ACL, in which case chmod would've worked fine over NFS. In any case, I don't see anything in that thread that seems relevant to NFSv4 id mapping, which unless I misunderstand is the problem the OP is trying to resolve. On that subject, NFSv4 id mapping seems to be working fine for me between an omnios client and server. On the server, the file system is mounted as: /export/user/henson on export/user/henson read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c And exported as: /export/user/henson - nfs nosuid,sec=krb5i,sec=krb5p with the domain set: $ sharectl get -p nfsmapid_domain nfs nfsmapid_domain=csupomona.edu if I create a file on the server, it has the correct ownership: $ touch test_server $ ls -l test_server -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server on the client, the NFS export is mounted as: /mnt on files-www.csupomona.edu:/export/user/henson remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec 4 12:50:01 2014 the client has the same domain: $ sharectl get -p nfsmapid_domain nfs nfsmapid_domain=csupomona.edu The file created on the server shows up with the correct ownership: $ ls -l test_server -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server A file created on the client has the correct ownership: $ touch test_client $ ls -l test_client -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client And viewed back on the server, still correct: $ ls -l test_client -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client From gate03 at landcroft.co.uk Thu Dec 4 21:31:11 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Fri, 5 Dec 2014 07:31:11 +1000 Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: References: <20141204153051.3e17ac8f@punda-mlia> Message-ID: <20141205073111.0762e2b1@punda-mlia> On Thu, 04 Dec 2014 09:35:04 +0100 Jorge Schrauwen wrote: > Hey, > > I have this setup on my SuperMicro server to have a 2nd console on > the SoL. > I documented it here: > https://docu.blackdot.be/snipets/solaris/misc-serial-console > > Hopefully this is helpful. It certainly is, Jorge; thanks very much. Working very well. The only thing I'd like to change is to limit root to logging in on the two known ports /dev/console and /dev/ttya, but that's a small point. Thank you for a very clear document. Michael. From henson at acm.org Thu Dec 4 22:22:55 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 4 Dec 2014 14:22:55 -0800 Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: <20141205073111.0762e2b1@punda-mlia> References: <20141204153051.3e17ac8f@punda-mlia> <20141205073111.0762e2b1@punda-mlia> Message-ID: <4cff01d01010$dc508510$94f18f30$@acm.org> > From: Michael Mounteney > Sent: Thursday, December 04, 2014 1:31 PM > > It certainly is, Jorge; thanks very much. Working very well. The > only thing I'd like to change is to limit root to logging in on the two > known ports /dev/console and /dev/ttya, but that's a small point. Hmm, reviewing the source to login, if CONSOLE is set to the default /dev/console, root login is allowed on either /dev/console or /dev/vt/*. If it is set to anything else, root login is allowed only from the device it is set to. If not set, root login is allowed on any device. It would be pretty trivial to extend the current code: } else { if (strcmp(ttyn, Console) == 0) return; } To allow CONSOLE to be a list of devices rather than a single device: char *state; char *test_console; for (test_console = strtok_r(Console, ",", &state); test_console != NULL, test_console = strtok_r(NULL, ",", &state)) { if (strcmp(ttyn, test_console) == 0) return; } I'm not sure if anything else pays attention to the CONSOLE definition in /etc/default/login that might get confused though. If you open a ticket in the illumos issue tracker requesting this feature, I might take a shot at implementing it :). I'm hoping to get an illumos-gate development environment going under omnios stable over the Christmas break (the OI vm I was using got trashed quite a while ago, and I just can't bring myself to install another OI box ), and this would be a simple test of it. From gate03 at landcroft.co.uk Thu Dec 4 22:50:57 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Fri, 5 Dec 2014 08:50:57 +1000 Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: <4cff01d01010$dc508510$94f18f30$@acm.org> References: <20141204153051.3e17ac8f@punda-mlia> <20141205073111.0762e2b1@punda-mlia> <4cff01d01010$dc508510$94f18f30$@acm.org> Message-ID: <20141205085057.5e000d9e@punda-mlia> On Thu, 4 Dec 2014 14:22:55 -0800 "Paul B. Henson" wrote: > If you open a ticket in the illumos issue tracker requesting this > feature, I might take a shot at implementing it :). I'm hoping to get > an illumos-gate development environment going under omnios stable > over the Christmas break (the OI vm I was using got trashed quite a > while ago, and I just can't bring myself to install another OI box > ), and this would be a simple test of it. Paul: https://www.illumos.org/issues/5391 Thanks, Michael. From gate03 at landcroft.co.uk Fri Dec 5 01:08:42 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Fri, 5 Dec 2014 11:08:42 +1000 Subject: [OmniOS-discuss] UPS serial non-standard, WHY? Message-ID: <20141205110842.2da41bea@punda-mlia> OK, not an OmniOS matter but as a lot of people here will be using UPSs, I hope it's alright. I made what is apparently the common but incorrect assumption that the serial monitoring port on the back of my APC 700 UPS had a standard serial pinout but for some reason the TxD and RxD are swapped. Is this just some random brain-&*^# on the part of an android within APC, or is there some actual reason for it? Don't plug a standard serial cable into the UPS: it instantly powers off. No fooling. This is the first step in getting UPS software monitoring in place. I presume that the software of choice is NUT? I can't find it in a repo so I suppose it's a build-it-yourself job. Michael. From protonwrangler at gmail.com Fri Dec 5 03:07:00 2014 From: protonwrangler at gmail.com (Warren Marts) Date: Thu, 4 Dec 2014 20:07:00 -0700 Subject: [OmniOS-discuss] UPS serial non-standard, WHY? In-Reply-To: <20141205110842.2da41bea@punda-mlia> References: <20141205110842.2da41bea@punda-mlia> Message-ID: Serial ports have two wiring standards DTE (data Terminal equipment) and DCE (data Communication equipment), with the data pins and transmit enables swapped. Connecting DTE to DCE you use a straight-through cable; here both are DTE so you need a crossover or "null modem" cable. On Thu, Dec 4, 2014 at 6:08 PM, Michael Mounteney wrote: > OK, not an OmniOS matter but as a lot of people here will be using > UPSs, I hope it's alright. > > I made what is apparently the common but incorrect assumption that the > serial monitoring port on the back of my APC 700 UPS had a standard > serial pinout but for some reason the TxD and RxD are swapped. Is this > just some random brain-&*^# on the part of an android within APC, or is > there some actual reason for it? > > Don't plug a standard serial cable into the UPS: it instantly powers > off. No fooling. > > This is the first step in getting UPS software monitoring in place. I > presume that the software of choice is NUT? I can't find it in a > repo so I suppose it's a build-it-yourself job. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From orilali at evogene.com Fri Dec 5 06:56:34 2014 From: orilali at evogene.com (Kobi Schwartz) Date: Fri, 05 Dec 2014 08:56:34 +0200 Subject: [OmniOS-discuss] Kernel stability. Message-ID: <54815722.3020909@evogene.com> Hi, My name is Kobi and I work with a company called Evogene. We have a storage server based on a Supermicro X9DRD-7LN4F (Intel(R) Xeon(R) CPU E5-2609, 64GB RAM) equipped with Intel X520-DA2 10Gb Ethernet card. Two weeks ago we encountered a problem which regards Omnios (r15012) and OpenIndiana (151_a8) kernel stability. The system is a clean install of r151012, only configuration is networking and zpool (raidz3 1 vdev of 8 disks of data and 3 parity). The storage has a single NFS share mounted (nfsvers=3) on an 8 grid nodes (running CentOS6.6) with the following mount options : async,tcp,vers=3,noacl,timeo=20,rw,bg,hard,intr,rsize=65536,wsize=65536 0 0. Under normal conditions (random reads and writes) we get great performance and stability but if we increase nfs usage the Kernel memory usage increases until the machine eventually halts. To test it we used on each node iozone ( Version $Revision: 3.394 ) which simultaneously write and reads 12 files with a size of 500MB. (iozone -r 64k -i 0 -i 1 -s 500m -t 12 -F /evolon2/iozone/`hostname`.{1..12}). While the iozone's are running the memory of Kernel (as seen by echo '::memstat'| mdb -k) is steadily increasing until the machine is dead. For clarification, when the machine is dead/halts: There is no crash files. There is no new messages in /var/adm/messages. It doesn't respond to pings (and obviously other network services). Waiting doesn't help. It can only be brought to life by power cycling it. I attached 3 files: memstat, top and arc-kstat which I hope will assist to understand why the storage died/halted Thank you in advance, Kobi. memstat -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- Thu Dec 4 16:14:39 IST 2014 l2_size = 0 other_size = 16487240 c_max = 67601473536 hash_elements = 36191 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12438 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97172664 hash_elements_max = 41359 size = 5312841912 hits = 750510 mfu_hits = 507924 prefetch_metadata_hits = 10297 hash_chains = 54 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 519 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5283795456 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1052.750004426 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 28989 arc_meta_max = 97686696 c_min = 2146100480 mru_hits = 231752 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 710687 l2_free_on_write = 0 l2_feeds = 0 misses = 19974 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:42 IST 2014 l2_size = 0 other_size = 16557200 c_max = 67601473536 hash_elements = 35929 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12446 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97848832 hash_elements_max = 41359 size = 5242345984 hits = 765628 mfu_hits = 519125 prefetch_metadata_hits = 10297 hash_chains = 53 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 544 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5213229568 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1055.756679096 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 28999 arc_meta_max = 97815864 c_min = 2146100480 mru_hits = 235669 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 725795 l2_free_on_write = 0 l2_feeds = 0 misses = 19982 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:45 IST 2014 l2_size = 0 other_size = 16815440 c_max = 67601473536 hash_elements = 34070 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12446 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97619648 hash_elements_max = 41359 size = 5487483584 hits = 791669 mfu_hits = 537122 prefetch_metadata_hits = 10297 hash_chains = 50 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 581 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5458108928 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1058.767796546 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29003 arc_meta_max = 97968040 c_min = 2146100480 mru_hits = 243713 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 751832 l2_free_on_write = 0 l2_feeds = 0 misses = 19982 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:48 IST 2014 l2_size = 0 other_size = 16858360 c_max = 67601473536 hash_elements = 36794 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12454 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98313832 hash_elements_max = 41359 size = 5462880872 hits = 802717 mfu_hits = 544723 prefetch_metadata_hits = 10297 hash_chains = 75 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 626 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5433463296 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1061.787519516 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29022 arc_meta_max = 98312872 c_min = 2146100480 mru_hits = 247160 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 762861 l2_free_on_write = 0 l2_feeds = 0 misses = 19990 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:51 IST 2014 l2_size = 0 other_size = 16888240 c_max = 67601473536 hash_elements = 40590 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12458 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98425632 hash_elements_max = 42641 size = 5669168928 hits = 815255 mfu_hits = 552397 prefetch_metadata_hits = 10297 hash_chains = 86 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 664 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5639721472 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1064.791199302 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29029 arc_meta_max = 98388464 c_min = 2146100480 mru_hits = 252024 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 775392 l2_free_on_write = 0 l2_feeds = 0 misses = 19994 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:54 IST 2014 l2_size = 0 other_size = 17262640 c_max = 67601473536 hash_elements = 36737 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12462 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98304416 hash_elements_max = 42641 size = 6396890528 hits = 840144 mfu_hits = 570407 prefetch_metadata_hits = 10297 hash_chains = 80 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 690 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6367068672 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1067.797220708 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29038 arc_meta_max = 98730168 c_min = 2146100480 mru_hits = 258903 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 800272 l2_free_on_write = 0 l2_feeds = 0 misses = 19998 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:57 IST 2014 l2_size = 0 other_size = 17330840 c_max = 67601473536 hash_elements = 39702 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12466 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99036168 hash_elements_max = 42641 size = 6258292744 hits = 851791 mfu_hits = 578190 prefetch_metadata_hits = 10297 hash_chains = 63 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 710 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6228402688 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1070.806236777 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29044 arc_meta_max = 99036168 c_min = 2146100480 mru_hits = 262767 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 811913 l2_free_on_write = 0 l2_feeds = 0 misses = 20002 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:00 IST 2014 l2_size = 0 other_size = 17276800 c_max = 67601473536 hash_elements = 39938 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12470 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98994416 hash_elements_max = 42641 size = 5947348208 hits = 854591 mfu_hits = 580209 prefetch_metadata_hits = 10297 hash_chains = 64 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 713 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5917512192 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1073.811610852 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29051 arc_meta_max = 99051168 c_min = 2146100480 mru_hits = 263548 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 814706 l2_free_on_write = 0 l2_feeds = 0 misses = 20006 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:03 IST 2014 l2_size = 0 other_size = 17471000 c_max = 67601473536 hash_elements = 38061 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12470 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98656136 hash_elements_max = 42641 size = 6162492296 hits = 881460 mfu_hits = 598502 prefetch_metadata_hits = 10297 hash_chains = 77 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 756 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6132462080 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1076.82642884 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29052 arc_meta_max = 99051168 c_min = 2146100480 mru_hits = 272124 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 841574 l2_free_on_write = 0 l2_feeds = 0 misses = 20006 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:06 IST 2014 l2_size = 0 other_size = 17531840 c_max = 67601473536 hash_elements = 38754 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98766128 hash_elements_max = 42641 size = 6131800368 hits = 891251 mfu_hits = 605449 prefetch_metadata_hits = 10297 hash_chains = 83 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 774 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6101709312 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1079.87068534 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 274968 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 851351 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:09 IST 2014 l2_size = 0 other_size = 17593040 c_max = 67601473536 hash_elements = 40255 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98860096 hash_elements_max = 42641 size = 6205425728 hits = 902579 mfu_hits = 612627 prefetch_metadata_hits = 10297 hash_chains = 81 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 794 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6175273472 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1082.875826687 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 279118 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 862679 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:12 IST 2014 l2_size = 0 other_size = 17598240 c_max = 67601473536 hash_elements = 43117 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98865296 hash_elements_max = 43118 size = 6013541520 hits = 905362 mfu_hits = 613511 prefetch_metadata_hits = 10297 hash_chains = 111 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 831 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5983384064 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1085.879514131 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 281017 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 865462 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:15 IST 2014 l2_size = 0 other_size = 17603440 c_max = 67601473536 hash_elements = 43174 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12482 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99480800 hash_elements_max = 43929 size = 5807325408 hits = 908319 mfu_hits = 615797 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 832 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5777162752 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1088.884358936 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29075 arc_meta_max = 99480800 c_min = 2146100480 mru_hits = 281688 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 868410 l2_free_on_write = 0 l2_feeds = 0 misses = 20018 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:18 IST 2014 l2_size = 0 other_size = 17668200 c_max = 67601473536 hash_elements = 41509 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12488 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98984408 hash_elements_max = 43929 size = 5884816856 hits = 927065 mfu_hits = 630052 prefetch_metadata_hits = 10297 hash_chains = 93 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 843 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5854589440 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1091.888514904 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29087 arc_meta_max = 99568328 c_min = 2146100480 mru_hits = 286179 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 887144 l2_free_on_write = 0 l2_feeds = 0 misses = 20024 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:21 IST 2014 l2_size = 0 other_size = 17646400 c_max = 67601473536 hash_elements = 39031 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12488 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98978992 hash_elements_max = 43929 size = 5865412784 hits = 955131 mfu_hits = 649821 prefetch_metadata_hits = 10297 hash_chains = 76 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 857 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5835207168 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1094.945818674 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29087 arc_meta_max = 99568328 c_min = 2146100480 mru_hits = 294476 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 915210 l2_free_on_write = 0 l2_feeds = 0 misses = 20024 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:24 IST 2014 l2_size = 0 other_size = 17752040 c_max = 67601473536 hash_elements = 41863 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12492 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99137880 hash_elements_max = 43929 size = 6192858456 hits = 968809 mfu_hits = 659725 prefetch_metadata_hits = 10297 hash_chains = 87 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 880 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6162547200 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1097.957144074 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29094 arc_meta_max = 99660136 c_min = 2146100480 mru_hits = 298250 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 928881 l2_free_on_write = 0 l2_feeds = 0 misses = 20028 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:27 IST 2014 l2_size = 0 other_size = 17651840 c_max = 67601473536 hash_elements = 44052 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12496 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99623408 hash_elements_max = 44527 size = 5948239344 hits = 972940 mfu_hits = 661821 prefetch_metadata_hits = 10297 hash_chains = 92 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 891 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5918028288 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1100.964341965 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29100 arc_meta_max = 99660136 c_min = 2146100480 mru_hits = 300285 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 933006 l2_free_on_write = 0 l2_feeds = 0 misses = 20032 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:30 IST 2014 l2_size = 0 other_size = 17736840 c_max = 67601473536 hash_elements = 39286 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12500 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99134968 hash_elements_max = 44527 size = 5866748408 hits = 999296 mfu_hits = 681359 prefetch_metadata_hits = 10297 hash_chains = 89 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 922 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5836452352 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1103.989100171 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29108 arc_meta_max = 99718496 c_min = 2146100480 mru_hits = 307103 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 959354 l2_free_on_write = 0 l2_feeds = 0 misses = 20036 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:33 IST 2014 l2_size = 0 other_size = 17708240 c_max = 67601473536 hash_elements = 43427 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12500 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99106368 hash_elements_max = 44527 size = 5953882688 hits = 1009876 mfu_hits = 687985 prefetch_metadata_hits = 10297 hash_chains = 107 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 959 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5923615232 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1106.99462468 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29109 arc_meta_max = 99718496 c_min = 2146100480 mru_hits = 311057 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 969933 l2_free_on_write = 0 l2_feeds = 0 misses = 20036 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:36 IST 2014 l2_size = 0 other_size = 17820440 c_max = 67601473536 hash_elements = 38756 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12508 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99222664 hash_elements_max = 44527 size = 5931061384 hits = 1034158 mfu_hits = 705727 prefetch_metadata_hits = 10297 hash_chains = 79 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 970 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5900681728 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1109.998397046 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29126 arc_meta_max = 99816784 c_min = 2146100480 mru_hits = 317597 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 994198 l2_free_on_write = 0 l2_feeds = 0 misses = 20044 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:39 IST 2014 l2_size = 0 other_size = 17808640 c_max = 67601473536 hash_elements = 44706 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12508 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99212592 hash_elements_max = 44706 size = 6196209968 hits = 1039940 mfu_hits = 709638 prefetch_metadata_hits = 10297 hash_chains = 103 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1014 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6165840384 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1113.002157194 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29126 arc_meta_max = 99816784 c_min = 2146100480 mru_hits = 319468 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 999980 l2_free_on_write = 0 l2_feeds = 0 misses = 20044 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:42 IST 2014 l2_size = 0 other_size = 17820240 c_max = 67601473536 hash_elements = 42808 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12516 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99240576 hash_elements_max = 46675 size = 6415390336 hits = 1059255 mfu_hits = 723283 prefetch_metadata_hits = 10297 hash_chains = 108 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1042 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6385009152 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1116.00772062 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29140 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 325138 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1019281 l2_free_on_write = 0 l2_feeds = 0 misses = 20052 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:45 IST 2014 l2_size = 0 other_size = 17674040 c_max = 67601473536 hash_elements = 44891 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12516 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99094376 hash_elements_max = 46675 size = 5900262248 hits = 1068199 mfu_hits = 728474 prefetch_metadata_hits = 10297 hash_chains = 130 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1081 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5870027264 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1119.021235208 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29140 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 328891 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1028225 l2_free_on_write = 0 l2_feeds = 0 misses = 20052 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:48 IST 2014 l2_size = 0 other_size = 17766200 c_max = 67601473536 hash_elements = 39994 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12524 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99751784 hash_elements_max = 46675 size = 5771420520 hits = 1088758 mfu_hits = 743653 prefetch_metadata_hits = 10297 hash_chains = 94 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1092 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5741093376 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1122.026835395 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29147 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 334271 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1048777 l2_free_on_write = 0 l2_feeds = 0 misses = 20060 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:51 IST 2014 l2_size = 0 other_size = 17829240 c_max = 67601473536 hash_elements = 43380 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12524 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99249576 hash_elements_max = 46675 size = 6035421608 hits = 1096297 mfu_hits = 749091 prefetch_metadata_hits = 10297 hash_chains = 87 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1102 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6005031424 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1125.031042354 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29154 arc_meta_max = 99829992 c_min = 2146100480 mru_hits = 336372 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1056309 l2_free_on_write = 0 l2_feeds = 0 misses = 20060 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:54 IST 2014 l2_size = 0 other_size = 17900600 c_max = 67601473536 hash_elements = 39997 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12528 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99390568 hash_elements_max = 46675 size = 5948399720 hits = 1120591 mfu_hits = 765902 prefetch_metadata_hits = 10297 hash_chains = 83 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1118 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5917938176 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1128.034852134 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29163 arc_meta_max = 99971728 c_min = 2146100480 mru_hits = 343855 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1080594 l2_free_on_write = 0 l2_feeds = 0 misses = 20064 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:57 IST 2014 l2_size = 0 other_size = 17931600 c_max = 67601473536 hash_elements = 44425 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12532 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100027776 hash_elements_max = 46675 size = 5889137024 hits = 1123794 mfu_hits = 768068 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1137 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5858644480 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1131.040711699 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29170 arc_meta_max = 99994208 c_min = 2146100480 mru_hits = 344892 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1083790 l2_free_on_write = 0 l2_feeds = 0 misses = 20068 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:00 IST 2014 l2_size = 0 other_size = 17803560 c_max = 67601473536 hash_elements = 42523 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12532 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99334488 hash_elements_max = 46675 size = 5995791704 hits = 1137123 mfu_hits = 776941 prefetch_metadata_hits = 10297 hash_chains = 84 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1145 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5965427200 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1134.044734783 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29171 arc_meta_max = 100042776 c_min = 2146100480 mru_hits = 349348 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1097118 l2_free_on_write = 0 l2_feeds = 0 misses = 20068 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:03 IST 2014 l2_size = 0 other_size = 18005000 c_max = 67601473536 hash_elements = 40668 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12540 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100146232 hash_elements_max = 46675 size = 5979957304 hits = 1157458 mfu_hits = 791193 prefetch_metadata_hits = 10297 hash_chains = 85 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1162 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5949391360 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1137.04862393 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29187 arc_meta_max = 100145632 c_min = 2146100480 mru_hits = 355431 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1117437 l2_free_on_write = 0 l2_feeds = 0 misses = 20076 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:06 IST 2014 l2_size = 0 other_size = 18052040 c_max = 67601473536 hash_elements = 45931 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12540 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99615736 hash_elements_max = 46725 size = 6264246264 hits = 1168001 mfu_hits = 798095 prefetch_metadata_hits = 10297 hash_chains = 98 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1192 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6233633280 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1140.052771613 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29188 arc_meta_max = 100153872 c_min = 2146100480 mru_hits = 359072 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1127979 l2_free_on_write = 0 l2_feeds = 0 misses = 20076 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:09 IST 2014 l2_size = 0 other_size = 18144240 c_max = 67601473536 hash_elements = 44399 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12548 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99752992 hash_elements_max = 47118 size = 6250227744 hits = 1186886 mfu_hits = 811377 prefetch_metadata_hits = 10297 hash_chains = 155 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1302 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6219522560 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1143.058190189 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29202 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 364675 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1146850 l2_free_on_write = 0 l2_feeds = 0 misses = 20084 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:12 IST 2014 l2_size = 0 other_size = 18053440 c_max = 67601473536 hash_elements = 46005 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12552 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100254576 hash_elements_max = 47118 size = 6189538160 hits = 1194015 mfu_hits = 815523 prefetch_metadata_hits = 10297 hash_chains = 129 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1311 l2_writes_hdr_miss = 0 evict_l2_ineligible = 397312 l2_write_bytes = 0 data_size = 6158923776 c = 6190066904 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1146.06243032 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29214 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 367658 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1153967 l2_free_on_write = 0 l2_feeds = 0 misses = 20088 evict_skip = 1214 evict_l2_eligible = 1011712 p = 3096114796 l2_compress_successes = 0 recycle_miss = 1 l2_writes_done = 0 ---------- Thu Dec 4 16:16:15 IST 2014 l2_size = 0 other_size = 17695320 c_max = 67601473536 hash_elements = 44156 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12556 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99370392 hash_elements_max = 47845 size = 6062509464 hits = 1214370 mfu_hits = 831061 prefetch_metadata_hits = 10297 hash_chains = 121 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12583744 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1369 l2_writes_hdr_miss = 0 evict_l2_ineligible = 132255744 l2_write_bytes = 0 data_size = 6032230400 c = 6190066904 l2_asize = 0 class = misc mutex_miss = 1 snaptime = 1149.080481297 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29222 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 372475 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1174314 l2_free_on_write = 0 l2_feeds = 0 misses = 20092 evict_skip = 1214 evict_l2_eligible = 285229056 p = 3401023084 l2_compress_successes = 0 recycle_miss = 219 l2_writes_done = 0 ---------- Thu Dec 4 16:16:18 IST 2014 l2_size = 0 other_size = 17003120 c_max = 67601473536 hash_elements = 49196 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12556 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97914576 hash_elements_max = 49196 size = 5597583056 hits = 1214720 mfu_hits = 831312 prefetch_metadata_hits = 10297 hash_chains = 158 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 11820128 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1414 l2_writes_hdr_miss = 0 evict_l2_ineligible = 132255744 l2_write_bytes = 0 data_size = 5568759808 c = 5859249032 l2_asize = 0 class = misc mutex_miss = 1 snaptime = 1152.093743051 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29222 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 372574 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1174664 l2_free_on_write = 0 l2_feeds = 0 misses = 20092 evict_skip = 1214 evict_l2_eligible = 285491200 p = 3295003257 l2_compress_successes = 0 recycle_miss = 219 l2_writes_done = 0 ---------- Thu Dec 4 16:16:21 IST 2014 l2_size = 0 other_size = 15860920 c_max = 67601473536 hash_elements = 44130 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12564 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 91144424 hash_elements_max = 49670 size = 4804212968 hits = 1244167 mfu_hits = 851345 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 1063 hdr_size = 11332144 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1433 l2_writes_hdr_miss = 0 evict_l2_ineligible = 137154560 l2_write_bytes = 0 data_size = 4777019904 c = 4879960441 l2_asize = 0 class = misc mutex_miss = 9 snaptime = 1155.099409293 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 24 demand_metadata_hits = 29238 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 381988 demand_data_misses = 2639 duplicate_buffers = 0 demand_data_hits = 1204095 l2_free_on_write = 0 l2_feeds = 0 misses = 20124 evict_skip = 1214 evict_l2_eligible = 1330413056 p = 3274886866 l2_compress_successes = 0 recycle_miss = 519 l2_writes_done = 0 ---------- Thu Dec 4 16:16:24 IST 2014 l2_size = 0 other_size = 15703800 c_max = 67601473536 hash_elements = 41125 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12572 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 85702920 hash_elements_max = 49670 size = 4396642568 hits = 1257727 mfu_hits = 860930 prefetch_metadata_hits = 10297 hash_chains = 92 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 3379 hdr_size = 10493456 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3383 l2_evict_lock_retry = 0 hash_collisions = 1442 l2_writes_hdr_miss = 0 evict_l2_ineligible = 142102528 l2_write_bytes = 0 data_size = 4370445312 c = 4396690358 l2_asize = 0 class = misc mutex_miss = 9 snaptime = 1158.10348092 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 56 demand_metadata_hits = 29252 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 385963 demand_data_misses = 2669 duplicate_buffers = 0 demand_data_hits = 1217641 l2_free_on_write = 0 l2_feeds = 0 misses = 20164 evict_skip = 1214 evict_l2_eligible = 1491629056 p = 2997814168 l2_compress_successes = 0 recycle_miss = 821 l2_writes_done = 0 ---------- Thu Dec 4 16:16:27 IST 2014 l2_size = 0 other_size = 14938800 c_max = 67601473536 hash_elements = 39052 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12576 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 80821952 hash_elements_max = 49670 size = 3828283072 hits = 1264949 mfu_hits = 864148 prefetch_metadata_hits = 10297 hash_chains = 75 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 10138 hdr_size = 8953872 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3383 l2_evict_lock_retry = 0 hash_collisions = 1454 l2_writes_hdr_miss = 0 evict_l2_ineligible = 144691200 l2_write_bytes = 0 data_size = 3804390400 c = 3830309700 l2_asize = 0 class = misc mutex_miss = 12 snaptime = 1161.109480472 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 77 demand_metadata_hits = 29256 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 389967 demand_data_misses = 2692 duplicate_buffers = 0 demand_data_hits = 1224859 l2_free_on_write = 0 l2_feeds = 0 misses = 20191 evict_skip = 1214 evict_l2_eligible = 1951429632 p = 2764099257 l2_compress_successes = 0 recycle_miss = 972 l2_writes_done = 0 ---------- -------------- next part -------------- Thu Dec 4 16:14:02 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 5667640 22139 34% ZFS File Data 1171645 4576 7% Anon 26071 101 0% Exec and libs 1181 4 0% Page cache 4609 18 0% Free (cachelist) 7723 30 0% Free (freelist) 9887541 38623 59% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:06 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 6380424 24923 38% ZFS File Data 1171645 4576 7% Anon 26096 101 0% Exec and libs 1175 4 0% Page cache 4610 18 0% Free (cachelist) 7729 30 0% Free (freelist) 9174731 35838 55% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:09 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 6524733 25487 39% ZFS File Data 1184093 4625 7% Anon 26096 101 0% Exec and libs 1181 4 0% Page cache 4612 18 0% Free (cachelist) 7722 30 0% Free (freelist) 9017973 35226 54% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:12 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7290171 28477 43% ZFS File Data 1184093 4625 7% Anon 26122 102 0% Exec and libs 1181 4 0% Page cache 4612 18 0% Free (cachelist) 7723 30 0% Free (freelist) 8252508 32236 49% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:16 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7456599 29127 44% ZFS File Data 1190237 4649 7% Anon 26122 102 0% Exec and libs 1181 4 0% Page cache 4613 18 0% Free (cachelist) 7723 30 0% Free (freelist) 8079935 31562 48% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:19 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7480950 29222 45% ZFS File Data 1513949 5913 9% Anon 26178 102 0% Exec and libs 1180 4 0% Page cache 4615 18 0% Free (cachelist) 7724 30 0% Free (freelist) 7731814 30202 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:25 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7481258 29223 45% ZFS File Data 1513949 5913 9% Anon 26174 102 0% Exec and libs 1181 4 0% Page cache 4616 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7731509 30201 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:28 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482405 29228 45% ZFS File Data 1577181 6160 9% Anon 26200 102 0% Exec and libs 1181 4 0% Page cache 4617 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7667103 29949 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:31 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482519 29228 45% ZFS File Data 1577181 6160 9% Anon 26226 102 0% Exec and libs 1181 4 0% Page cache 4618 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666962 29949 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:35 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482590 29228 45% ZFS File Data 1577181 6160 9% Anon 26226 102 0% Exec and libs 1180 4 0% Page cache 4619 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666891 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:38 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482751 29229 45% ZFS File Data 1577181 6160 9% Anon 26252 102 0% Exec and libs 1181 4 0% Page cache 4619 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666703 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:41 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482755 29229 45% ZFS File Data 1577181 6160 9% Anon 26252 102 0% Exec and libs 1181 4 0% Page cache 4622 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666696 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:44 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7483021 29230 45% ZFS File Data 1577181 6160 9% Anon 26278 102 0% Exec and libs 1181 4 0% Page cache 4623 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666403 29946 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:48 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7483581 29232 45% ZFS File Data 1577181 6160 9% Anon 26303 102 0% Exec and libs 1181 4 0% Page cache 4622 18 0% Free (cachelist) 7724 30 0% Free (freelist) 7665818 29944 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:51 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7645816 29866 46% ZFS File Data 1577181 6160 9% Anon 26328 102 0% Exec and libs 1181 4 0% Page cache 4624 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7503557 29310 45% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:55 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7645904 29866 46% ZFS File Data 1577181 6160 9% Anon 26328 102 0% Exec and libs 1181 4 0% Page cache 4625 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7503468 29310 45% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:58 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7753278 30286 46% ZFS File Data 1577181 6160 9% Anon 26355 102 0% Exec and libs 1181 4 0% Page cache 4626 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7396066 28890 44% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:01 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8085909 31585 48% ZFS File Data 1577181 6160 9% Anon 26380 103 0% Exec and libs 1179 4 0% Page cache 4627 18 0% Free (cachelist) 7725 30 0% Free (freelist) 7063409 27591 42% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:04 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8085951 31585 48% ZFS File Data 1577181 6160 9% Anon 26380 103 0% Exec and libs 1181 4 0% Page cache 4628 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7063366 27591 42% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:08 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8209025 32066 49% ZFS File Data 1577181 6160 9% Anon 26407 103 0% Exec and libs 1181 4 0% Page cache 4630 18 0% Free (cachelist) 7723 30 0% Free (freelist) 6940263 27110 41% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:11 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8763032 34230 52% ZFS File Data 1577181 6160 9% Anon 26432 103 0% Exec and libs 1181 4 0% Page cache 4630 18 0% Free (cachelist) 7723 30 0% Free (freelist) 6386231 24946 38% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:14 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9388716 36674 56% ZFS File Data 1577181 6160 9% Anon 26432 103 0% Exec and libs 1181 4 0% Page cache 4631 18 0% Free (cachelist) 7724 30 0% Free (freelist) 5760545 22502 34% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:17 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9586189 37446 57% ZFS File Data 1577181 6160 9% Anon 26459 103 0% Exec and libs 1181 4 0% Page cache 4632 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5563045 21730 33% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:20 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9967338 38934 59% ZFS File Data 1577181 6160 9% Anon 26459 103 0% Exec and libs 1181 4 0% Page cache 4633 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5181895 20241 31% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:24 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10087818 39405 60% ZFS File Data 1577181 6160 9% Anon 26484 103 0% Exec and libs 1181 4 0% Page cache 4635 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5061388 19771 30% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:27 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10620046 41484 63% ZFS File Data 1577181 6160 9% Anon 26510 103 0% Exec and libs 1157 4 0% Page cache 4633 18 0% Free (cachelist) 7750 30 0% Free (freelist) 4529133 17691 27% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:30 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10728161 41906 64% ZFS File Data 1577181 6160 9% Anon 26510 103 0% Exec and libs 1181 4 0% Page cache 4637 18 0% Free (cachelist) 7723 30 0% Free (freelist) 4421017 17269 26% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:34 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11230924 43870 67% ZFS File Data 1577181 6160 9% Anon 26536 103 0% Exec and libs 1181 4 0% Page cache 4637 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3918228 15305 23% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:38 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11277074 44051 67% ZFS File Data 1577981 6163 9% Anon 26562 103 0% Exec and libs 1181 4 0% Page cache 4638 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3871251 15122 23% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:41 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11779320 46012 70% ZFS File Data 1577981 6163 9% Anon 26562 103 0% Exec and libs 1180 4 0% Page cache 4639 18 0% Free (cachelist) 7724 30 0% Free (freelist) 3369004 13160 20% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:44 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11968305 46751 71% ZFS File Data 1577981 6163 9% Anon 26588 103 0% Exec and libs 1181 4 0% Page cache 4641 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3179991 12421 19% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:47 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12424008 48531 74% ZFS File Data 1577981 6163 9% Anon 26614 103 0% Exec and libs 1181 4 0% Page cache 4642 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2724261 10641 16% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:51 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12708257 49641 76% ZFS File Data 1577981 6163 9% Anon 26614 103 0% Exec and libs 1181 4 0% Page cache 4643 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2440011 9531 15% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:54 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12960558 50627 77% ZFS File Data 1577981 6163 9% Anon 26640 104 0% Exec and libs 1181 4 0% Page cache 4644 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2187683 8545 13% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:58 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 13441434 52505 80% ZFS File Data 1577981 6163 9% Anon 26664 104 0% Exec and libs 1181 4 0% Page cache 4645 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1706782 6667 10% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:01 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 13724075 53609 82% ZFS File Data 1577981 6163 9% Anon 26691 104 0% Exec and libs 1181 4 0% Page cache 4646 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1424113 5562 8% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:05 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14075430 54982 84% ZFS File Data 1577981 6163 9% Anon 26691 104 0% Exec and libs 1181 4 0% Page cache 4647 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1072757 4190 6% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:08 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14302316 55868 85% ZFS File Data 1577981 6163 9% Anon 26723 104 0% Exec and libs 1181 4 0% Page cache 4649 18 0% Free (cachelist) 7723 30 0% Free (freelist) 845837 3304 5% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:12 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14585839 56975 87% ZFS File Data 1541308 6020 9% Anon 26743 104 0% Exec and libs 1181 4 0% Page cache 4649 18 0% Free (cachelist) 7724 30 0% Free (freelist) 598966 2339 4% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:16 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14682403 57353 88% ZFS File Data 1408152 5500 8% Anon 26768 104 0% Exec and libs 1181 4 0% Page cache 4651 18 0% Free (cachelist) 7723 30 0% Free (freelist) 635532 2482 4% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:19 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15056031 58812 90% ZFS File Data 1163626 4545 7% Anon 26778 104 0% Exec and libs 1181 4 0% Page cache 4652 18 0% Free (cachelist) 7723 30 0% Free (freelist) 506419 1978 3% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:23 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15285492 59708 91% ZFS File Data 1008029 3937 6% Anon 26795 104 0% Exec and libs 1181 4 0% Page cache 4653 18 0% Free (cachelist) 7429 29 0% Free (freelist) 432831 1690 3% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:27 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15705417 61349 94% ZFS File Data 857418 3349 5% Anon 21468 83 0% Exec and libs 73 0 0% Page cache 721 2 0% Free (cachelist) 14622 57 0% Free (freelist) 166691 651 1% Total 16766410 65493 Physical 16766409 65493 -------------- next part -------------- last pid: 996; load avg: 5.95, 3.04, 1.31; up 0+00:19:43 16:16:27 59 processes: 58 sleeping, 1 on cpu CPU states: 30.6% idle, 1.6% user, 67.8% kernel, 0.0% iowait, 0.0% swap Kernel: 97556 ctxsw, 2440 trap, 448800 intr, 27430 syscall, 1 fork, 1407 flt Memory: 64G phys mem, 933M free mem, 4096M total swap, 4096M free swap PID USERNAME NLWP PRI NICE SIZE RES STATE TIME CPU COMMAND 675 daemon 10 60 -20 2904K 1968K sleep 2:28 4.35% nfsd 607 root 1 59 0 8308K 6668K sleep 0:00 0.01% perl 631 root 1 59 0 26M 25M cpu/5 0:00 0.01% top 711 root 1 59 0 5888K 4180K sleep 0:00 0.01% perl 705 root 1 59 0 4712K 2628K sleep 0:00 0.00% perl 633 root 1 59 0 7576K 4828K sleep 0:00 0.00% sshd 712 root 1 59 0 1644K 1040K sleep 0:00 0.00% tee 622 root 1 59 0 7576K 4828K sleep 0:00 0.00% sshd 645 root 1 59 0 7576K 4832K sleep 0:00 0.00% sshd 10 root 14 59 0 7684K 5664K sleep 0:01 0.00% svc.startd 580 root 27 59 0 23M 16M sleep 0:01 0.00% fmd 706 root 1 59 0 1644K 1040K sleep 0:00 0.00% tee 12 root 19 59 0 9704K 8372K sleep 0:04 0.00% svc.configd 241 root 7 59 0 4456K 3220K sleep 0:00 0.00% devfsadm 294 root 34 59 0 7480K 4936K sleep 0:00 0.00% nscd 367 root 1 59 0 1656K 940K sleep 0:00 0.00% utmpd 658 daemon 2 60 -20 2888K 1912K sleep 0:00 0.00% lockd 232 root 5 60 -20 2712K 1600K sleep 0:00 0.00% zonestatd 322 root 4 59 0 11M 8224K sleep 0:00 0.00% hald 301 root 5 59 0 5232K 4324K sleep 0:00 0.00% picld 487 root 3 59 0 5272K 3688K sleep 0:00 0.00% inetd 331 root 2 59 0 6660K 3368K sleep 0:00 0.00% hald-runner 225 root 17 59 0 6164K 3312K sleep 0:00 0.00% syseventd 632 root 1 59 0 5804K 3192K sleep 0:00 0.00% sshd 644 root 1 59 0 5804K 3192K sleep 0:00 0.00% sshd 612 root 1 59 0 6508K 3180K sleep 0:00 0.00% hald-addon-netw 614 root 1 59 0 6508K 3180K sleep 0:00 0.00% hald-addon-cpuf 621 root 1 59 0 5804K 3096K sleep 0:00 0.00% sshd 49 netadm 3 59 0 4084K 2836K sleep 0:00 0.00% ipmgmtd 673 root 13 59 0 4588K 2744K sleep 0:00 0.00% mountd From henson at acm.org Fri Dec 5 08:09:17 2014 From: henson at acm.org (Paul B. Henson) Date: Fri, 5 Dec 2014 00:09:17 -0800 Subject: [OmniOS-discuss] Kernel stability. In-Reply-To: <54815722.3020909@evogene.com> References: <54815722.3020909@evogene.com> Message-ID: <20141205080917.GM29549@bender.unx.csupomona.edu> On Fri, Dec 05, 2014 at 08:56:34AM +0200, Kobi Schwartz wrote: > > It can only be brought to life by power cycling it. I'm in no way qualified to diagnose this :), but perhaps you could set your box up to panic on NMI: http://www.cuddletech.com/blog/pivot/entry.php?id=1044 And next time this happens, instead of just resetting, force a panic and get a kernel dump? That would be helpful to someone who actually would be qualified to look at the issue... BTW, on newer boxes you need: set apix:apic_panic_on_nmi = 1 instead of: set pcplusmp:apic_kmdb_on_nmi=1 I usually just set both to cover the bases... From filip.marvan at aira.cz Fri Dec 5 09:10:58 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Fri, 5 Dec 2014 10:10:58 +0100 Subject: [OmniOS-discuss] zpool import strange informations In-Reply-To: <6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local> <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local> <6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com> Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3308DC@AIRA-SRV.aira.local> Hi, great! Thank you. With this command, I can see something like that: /dev/dsk/c3t2d0 LABEL 2 and LABEL 3 -------------------------------------------- version: 15 name: 'tank' state: 0 txg: 121888 pool_guid: 4916680739993977334 hostid: 3848478874 hostname: 'xxx' top_guid: 11305277212649201582 guid: 11245012245677467134 ... And for /dev/dsk/c3t2d0s0 LABEL 0-3 -------------------------------------------- version: 5000 name: 'raid_10' state: 0 txg: 7494567 pool_guid: 1071744107269411238 hostid: 134742016 hostname: 'dnas' top_guid: 565884742309860805 guid: 13206765472826992314 vdev_children: 3 So it seems, that /dev/dsk/c3t2d0 have some bad (old) labels, and /dev/dsk/c3t2d0s0 have correct one. How can I remove the bad unused labels and how can I avoid this in future when I create new pool with formerly used disks? Thank you very much, Filip -----Original Message----- From: Richard Elling [mailto:richard.elling at richardelling.com] Sent: Wednesday, December 03, 2014 6:02 PM To: Filip Marvan Cc: Zach Malone; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] zpool import strange informations Try this instead: for i in /dev/dsk/c3t2d0* do echo $i zdb -l $i done This will show you the ZFS labels on each and every slice/partition for the disk. Ideally, you'll only see one set of ZFS labels for each disk. -- richard -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6247 bytes Desc: not available URL: From orilali at evogene.com Fri Dec 5 10:19:59 2014 From: orilali at evogene.com (Kobi Schwartz) Date: Fri, 05 Dec 2014 12:19:59 +0200 Subject: [OmniOS-discuss] Kernel stability. In-Reply-To: <20141205080917.GM29549@bender.unx.csupomona.edu> References: <54815722.3020909@evogene.com> <20141205080917.GM29549@bender.unx.csupomona.edu> Message-ID: <548186CF.9090207@evogene.com> Hi Paul, Thanks for the tip. I already changed /etc/system. Kobi. On 12/05/2014 10:09 AM, Paul B. Henson wrote: > On Fri, Dec 05, 2014 at 08:56:34AM +0200, Kobi Schwartz wrote: >> It can only be brought to life by power cycling it. > I'm in no way qualified to diagnose this :), but perhaps you could set > your box up to panic on NMI: > > http://www.cuddletech.com/blog/pivot/entry.php?id=1044 > > And next time this happens, instead of just resetting, force a panic and > get a kernel dump? That would be helpful to someone who actually would > be qualified to look at the issue... > > BTW, on newer boxes you need: > > set apix:apic_panic_on_nmi = 1 > > instead of: > > set pcplusmp:apic_kmdb_on_nmi=1 > > I usually just set both to cover the bases... > From gate03 at landcroft.co.uk Fri Dec 5 21:00:52 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Sat, 6 Dec 2014 07:00:52 +1000 Subject: [OmniOS-discuss] UPS serial non-standard, WHY? In-Reply-To: <20141205080411.GI19035@wagner-net.com> References: <20141205110842.2da41bea@punda-mlia> <20141205080411.GI19035@wagner-net.com> Message-ID: <20141206070052.6367a1a8@punda-mlia> On Fri, 5 Dec 2014 09:04:11 +0100 Thomas Wagner wrote: > Michael, > I can't say something on the serial connection. > But I believe that some of the PINs like "carrier detect" > are used to signal power conditions and in the other direction > tell the UPS to power on or off. > Maybe the "null-modem" case where the special cable from APC > swaps everything. It's not a proper 'null modem' (DTE) layout; just pins 2 and 3 are swapped. http://www.networkupstools.org/ups-protocols/apcsmart.html > About NUT, I compiled this for the SFE repository (RPM style > build recipes). Thanks but I'm going to bite the bullet and work out how to build the packages I want in my own repository. So far: OpenLDAP and NUT, and I'm sure there'll be more. I'm going to have to do it at some stage. Michael. From jklimek at gmail.com Fri Dec 5 13:32:01 2014 From: jklimek at gmail.com (John Klimek) Date: Fri, 5 Dec 2014 08:32:01 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <4ced01d01005$44363af0$cca2b0d0$@acm.org> References: <20141204193711.03524183@sleipner.datanom.net> <20141204201107.43ad5928@sleipner.datanom.net> <4ced01d01005$44363af0$cca2b0d0$@acm.org> Message-ID: Paul, thanks! You understand exactly what I'm asking and what my problem is. Something seems to be going wrong (or incorrectly configured) with the NFS id mapping services on my Linux client or OmniOS server. I've setup the nfsmapid_domain and I believe it's working because before configuring it, everything was showing as nobody/nobody, but afterwards it shows the correct names. (on the client) I can also see in the client log and it's correctly mapping user at domain.com in the Linux rpc.idmapd domain. The problem is that when I create a file on the client, the server (OmniOS) is showing the UID of the user that created the file instead of mapping it using the name to the local UID. It looks like I can avoid the problem if I synchronize UIDs and GIDs across all servers (it's a small home network so it's not a big problem and I'm not using LDAP or NIS), but NFSv4 was supposed to avoid this problem by using name mapping so you didn't have to synchronize UIDs/GIDs. On Thu, Dec 4, 2014 at 3:59 PM, Paul B. Henson wrote: > > From: Michael Rasmussen > > Sent: Thursday, December 04, 2014 11:11 AM > > > > Yes, because you want to avoid Omnios presents ACL which is incompatible > > with Linux ACL. > > I don't believe the ACL has anything to do with NFSv4 id mapping? And a ZFS > ACL presented over NFSv4 is perfectly compatible with Linux. It's not a > Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need > to > use the nfs4 ACL tools, but it works fine. > > > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS- > > Share?p=81530#post81530 > > In that thread, the user fails to chmod via NFS: > > chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': Operation > not permitted > > The root cause of which was a setting of restricted for aclmode: > > vdev1/proxCT aclmode restricted > local > > Per the man page "An aclmode property of restricted will cause the chmod(2) > operation to return an error when used on any file or directory which has a > non-trivial ACL whose entries can not be represented by a mode." > > The user could have set the inherited ACL on the initial filesystem to a > trivial ACL, in which case chmod would've worked fine over NFS. > > In any case, I don't see anything in that thread that seems relevant to > NFSv4 id mapping, which unless I misunderstand is the problem the OP is > trying to resolve. > > On that subject, NFSv4 id mapping seems to be working fine for me between > an > omnios client and server. On the server, the file system is mounted as: > > /export/user/henson on export/user/henson > read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c > > And exported as: > > /export/user/henson - nfs nosuid,sec=krb5i,sec=krb5p > > with the domain set: > > $ sharectl get -p nfsmapid_domain nfs > nfsmapid_domain=csupomona.edu > > if I create a file on the server, it has the correct ownership: > > $ touch test_server > $ ls -l test_server > -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server > > on the client, the NFS export is mounted as: > > /mnt on files-www.csupomona.edu:/export/user/henson > remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec 4 > 12:50:01 2014 > > the client has the same domain: > > $ sharectl get -p nfsmapid_domain nfs > nfsmapid_domain=csupomona.edu > > The file created on the server shows up with the correct ownership: > > $ ls -l test_server > -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server > > A file created on the client has the correct ownership: > > $ touch test_client > $ ls -l test_client > -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client > > And viewed back on the server, still correct: > > $ ls -l test_client > -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ikaufman at eng.ucsd.edu Fri Dec 5 16:53:13 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Fri, 5 Dec 2014 08:53:13 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141204201107.43ad5928@sleipner.datanom.net> <4ced01d01005$44363af0$cca2b0d0$@acm.org> Message-ID: As far as I recall, AUTH_UNIX (aka AUTH_SYS) uses RPC, and RPC has not been augmented to support NFSv4 yet. >From an old thread in the OpenSolaris boards: ------------------------------------- Note also, that although NFSv4 uses strings for uid/gid/domain, the underlying RPC layer uses the same authentication credentials as in previous NFS versions and other RPC programs. Since it's the RPC authentication that is used to control access, etc, don't expect too much from the NFSv4 name/id mapping. It's useful for ls -l listings, etc, but not for authentication. ------------------------------------- On Fri, Dec 5, 2014 at 5:32 AM, John Klimek wrote: > Paul, thanks! You understand exactly what I'm asking and what my problem > is. > > Something seems to be going wrong (or incorrectly configured) with the NFS > id mapping services on my Linux client or OmniOS server. > > I've setup the nfsmapid_domain and I believe it's working because before > configuring it, everything was showing as nobody/nobody, but afterwards it > shows the correct names. (on the client) I can also see in the client log > and it's correctly mapping user at domain.com in the Linux rpc.idmapd domain. > > The problem is that when I create a file on the client, the server (OmniOS) > is showing the UID of the user that created the file instead of mapping it > using the name to the local UID. > > It looks like I can avoid the problem if I synchronize UIDs and GIDs across > all servers (it's a small home network so it's not a big problem and I'm not > using LDAP or NIS), but NFSv4 was supposed to avoid this problem by using > name mapping so you didn't have to synchronize UIDs/GIDs. > > > > On Thu, Dec 4, 2014 at 3:59 PM, Paul B. Henson wrote: >> >> > From: Michael Rasmussen >> > Sent: Thursday, December 04, 2014 11:11 AM >> > >> > Yes, because you want to avoid Omnios presents ACL which is incompatible >> > with Linux ACL. >> >> I don't believe the ACL has anything to do with NFSv4 id mapping? And a >> ZFS >> ACL presented over NFSv4 is perfectly compatible with Linux. It's not a >> Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need >> to >> use the nfs4 ACL tools, but it works fine. >> >> > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS- >> > Share?p=81530#post81530 >> >> In that thread, the user fails to chmod via NFS: >> >> chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': >> Operation >> not permitted >> >> The root cause of which was a setting of restricted for aclmode: >> >> vdev1/proxCT aclmode restricted >> local >> >> Per the man page "An aclmode property of restricted will cause the >> chmod(2) >> operation to return an error when used on any file or directory which has >> a >> non-trivial ACL whose entries can not be represented by a mode." >> >> The user could have set the inherited ACL on the initial filesystem to a >> trivial ACL, in which case chmod would've worked fine over NFS. >> >> In any case, I don't see anything in that thread that seems relevant to >> NFSv4 id mapping, which unless I misunderstand is the problem the OP is >> trying to resolve. >> >> On that subject, NFSv4 id mapping seems to be working fine for me between >> an >> omnios client and server. On the server, the file system is mounted as: >> >> /export/user/henson on export/user/henson >> read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c >> >> And exported as: >> >> /export/user/henson - nfs nosuid,sec=krb5i,sec=krb5p >> >> with the domain set: >> >> $ sharectl get -p nfsmapid_domain nfs >> nfsmapid_domain=csupomona.edu >> >> if I create a file on the server, it has the correct ownership: >> >> $ touch test_server >> $ ls -l test_server >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server >> >> on the client, the NFS export is mounted as: >> >> /mnt on files-www.csupomona.edu:/export/user/henson >> remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec 4 >> 12:50:01 2014 >> >> the client has the same domain: >> >> $ sharectl get -p nfsmapid_domain nfs >> nfsmapid_domain=csupomona.edu >> >> The file created on the server shows up with the correct ownership: >> >> $ ls -l test_server >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:50 test_server >> >> A file created on the client has the correct ownership: >> >> $ touch test_client >> $ ls -l test_client >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client >> >> And viewed back on the server, still correct: >> >> $ ls -l test_client >> -rw-r--r--+ 1 henson csupomona 0 Dec 4 12:52 test_client >> >> > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From henson at acm.org Fri Dec 5 19:31:26 2014 From: henson at acm.org (Paul B. Henson) Date: Fri, 5 Dec 2014 11:31:26 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141204201107.43ad5928@sleipner.datanom.net> <4ced01d01005$44363af0$cca2b0d0$@acm.org> Message-ID: <57b501d010c2$11d3d6e0$357b84a0$@acm.org> > From: Ian Kaufman > Sent: Friday, December 05, 2014 8:53 AM > > As far as I recall, AUTH_UNIX (aka AUTH_SYS) uses RPC, and RPC has not > been augmented to support NFSv4 yet. Ah, right; I forgot about that annoyance :(. My systems are using kerberized NFS, which doesn't rely on uid/gid being passed over the wire. I remember now when we needed to integrate a system that wouldn't do kerberos, and we had the exact same uid mismatch issues. I don't know why they didn't introduce a new mechanism such as AUTH_SYSNAME with NFSv4 that would be identical to AUTH_SYS but use string identifiers rather than hardcoded id's over the wire, that's quite a deficiency. For a small scale, it's a bit of a pain/overkill to have to set up a kerberos infrastructure. I've never tried secure RPC using Diffie Hellman, that might work if you're purely Solaris/illumos, but I don't believe linux supports that. And I'm not even sure whether or not that uses string identifiers or numeric IDs on the wire? From rt at steait.net Fri Dec 5 19:56:11 2014 From: rt at steait.net (Rune Tipsmark) Date: Fri, 5 Dec 2014 19:56:11 +0000 Subject: [OmniOS-discuss] PCIe dedicated device for ZIL In-Reply-To: References: Message-ID: <1417809372716.19446@steait.net> its a good idea, I use Fusion IO SLC drives but I still see a limit of about 750 MB/sec which is too little, also I need to run multiple streams to achieve this, if I only use a single data stream I only get around 350 MB/sec. I makes no difference if I use 1 or 4 SLC drives, speed remains pretty much the same +/- 5%. After testing some mSata drives in one of Intels NUCs it became interesting to investigate if it would be possible to get a motherboard with say 10 or 20 mSata slots, I even asked SuperMicros engineers and they said no coz its not an enterprise product. However there are some PCIe cards available where you can attach mSata drives and even raid them and while I havent tried this yet, I think it could deliver some nice cheap SLOG perforformance check http://www.addonics.com/products/ad4mspx2.php Maybe there are better products out there, this was just what I came across initially. Maybe even completely skip a dedicated log device and set the pool to sync=always and base it on for example 8 of the above PCIe cards each with 4 1TB drives and you could have 32TB raw SSD ultrafast storage in a 3unit and with compression and possibly dedup you could easily store a lot of data on even when mirroring. This is probably my next project anyway so interested to see what you find out in terms of upping the speed and lowering the latency. br, Rune ________________________________ From: OmniOS-discuss on behalf of Angel Angelov Sent: Tuesday, December 2, 2014 3:57 PM To: omnios-discuss at lists.omniti.com Subject: [OmniOS-discuss] PCIe dedicated device for ZIL Hi guys, does anyone played with some of these PCIe devices on the market as a cheaper ZIL dedicated device instead of this one for example that would work with OmniOS out of the box? Here's one of the products I have found in the Internet googling around but it's way too big for the purpose: http://www.kingspec.com/en/product_xx187.html I am trying to find a good price/performance balance when full SSD storage based pool is in use. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fwp at deepthought.com Sun Dec 7 22:43:53 2014 From: fwp at deepthought.com (Frank Pittel) Date: Sun, 7 Dec 2014 16:43:53 -0600 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> Message-ID: <20141207224353.GA13916@warlock.deepthought.com> I've tried doing this with NFS4 a couple of years ago and what I found was that while the UID mapping worked for things like "ls" anytime you tried to actually do anything with the file RPC calls were made and since that's outside of NFS they failed. It was a couple of years ago now since I went through it but the upshot is that at least then uid mapping didn't work in any meaningful way! The Other Frank On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote: > Thanks, I will give this a try. > > However, when I create a file if I check the acl on Omni, it does have > extended permissions and looks correct. It's only the uid and gid that I'm > trying to get mapped correctly. > > Do you still think that noacl will solve that problem? > > (I'm migrating some virtual machines so I can't try out the setting you > mentioned until a few more minutes...) > > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen wrote: > > > On Thu, 4 Dec 2014 13:20:04 -0500 > > John Klimek wrote: > > > > > > > > What can I have setup wrong? > > > > > > I can't find any debugging or logging options for nfsmapid... > > > > > > Also, does the uidmap and gidmap share.nfs options only work for NFSv3? > > Solaris and derivatives implementation of NFS ACL is not compliant to the > > Linux NFS ACL. > > More directly it is the ACL in Linux which is not POSIX conformant so to > > avoid problems you > > should add the mount option noacl in your Linux fstab file. Noacl will > > instruct Omnios NFS > > to revert to plain old uid/gid. > > > > -- > > Hilsen/Regards > > Michael Rasmussen > > > > Get my public GnuPG keys: > > michael rasmussen cc > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > > mir datanom net > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > > mir miras org > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > > -------------------------------------------------------------- > > /usr/games/fortune -es says: > > Don't just echo the code with comments - make every comment count. > > - The Elements of Programming Style (Kernighan & Plaugher) > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From jimklimov at cos.ru Mon Dec 8 08:50:33 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Mon, 08 Dec 2014 09:50:33 +0100 Subject: [OmniOS-discuss] Timezones Message-ID: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru> Hello all, The /usr/share/lib/zoneinfo/src/README mentions that the Solaris zoneinfo is based on Olson tzdata 2010k. Is that true or was the data since then updated and just not the readme? Is it safe to remove this directory and replace by either vanilla rebuild of a recent tzdata (i.e. 2014j) or some transformation thereof to make it more solaris-compatible? Or should it be overlaid by whatever the new version provides? The set of filenames is noticeably different between the two variants (installed in OS and manually compiled with zic)... Also, do the distributions like OmniOS or OI, or the illumos-gate itself, track the IANA (exOlson) tzdata to provide some recent version in the os/net builds? Thanks, Jim Klimov -- Typos courtesy of K-9 Mail on my Samsung Android From jklimek at gmail.com Mon Dec 8 13:25:54 2014 From: jklimek at gmail.com (John Klimek) Date: Mon, 8 Dec 2014 08:25:54 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <20141207224353.GA13916@warlock.deepthought.com> References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> Message-ID: Thanks everybody. Are you guys saying that Solaris and Linux are incompatible for sharing NFSv4 ACLs (because of RPC differences, etc) ? On Sun, Dec 7, 2014 at 5:43 PM, Frank Pittel wrote: > I've tried doing this with NFS4 a couple of years ago and what I found was > that > while the UID mapping worked for things like "ls" anytime you tried to > actually > do anything with the file RPC calls were made and since that's outside of > NFS they > failed. > > It was a couple of years ago now since I went through it but the upshot is > that > at least then uid mapping didn't work in any meaningful way! > > The Other Frank > > > > On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote: > > Thanks, I will give this a try. > > > > However, when I create a file if I check the acl on Omni, it does have > > extended permissions and looks correct. It's only the uid and gid that > I'm > > trying to get mapped correctly. > > > > Do you still think that noacl will solve that problem? > > > > (I'm migrating some virtual machines so I can't try out the setting you > > mentioned until a few more minutes...) > > > > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen wrote: > > > > > On Thu, 4 Dec 2014 13:20:04 -0500 > > > John Klimek wrote: > > > > > > > > > > > What can I have setup wrong? > > > > > > > > I can't find any debugging or logging options for nfsmapid... > > > > > > > > Also, does the uidmap and gidmap share.nfs options only work for > NFSv3? > > > Solaris and derivatives implementation of NFS ACL is not compliant to > the > > > Linux NFS ACL. > > > More directly it is the ACL in Linux which is not POSIX conformant so > to > > > avoid problems you > > > should add the mount option noacl in your Linux fstab file. Noacl will > > > instruct Omnios NFS > > > to revert to plain old uid/gid. > > > > > > -- > > > Hilsen/Regards > > > Michael Rasmussen > > > > > > Get my public GnuPG keys: > > > michael rasmussen cc > > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E > > > mir datanom net > > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C > > > mir miras org > > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 > > > -------------------------------------------------------------- > > > /usr/games/fortune -es says: > > > Don't just echo the code with comments - make every comment count. > > > - The Elements of Programming Style (Kernighan & Plaugher) > > > > > > _______________________________________________ > > > OmniOS-discuss mailing list > > > OmniOS-discuss at lists.omniti.com > > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at cos.ru Mon Dec 8 12:57:34 2014 From: jim at cos.ru (Jim Klimov) Date: Mon, 08 Dec 2014 13:57:34 +0100 Subject: [OmniOS-discuss] FYI: Split-root installs are also applicable to OmniOS Message-ID: Hello all, Now that I'm playing with OmniOS as well, I am verifying my old Solaris/OpenSolaris/OI recipes with it. In particular, the "split-root installation" procedure worked flawlessly via copy-paste from my old article (now updated to smooth some non-critical rough edges): http://wiki.openindiana.org/oi/Advanced+-+Split-root+installation#Advanced-Split-rootinstallation-Upgrades Same caveats as before are applicable, i.e. that "pkg upgrade" which invokes "beadm clone" un-does much of the effort regarding dataset fine-tuning (compression/quota/... on components of the rootfs), so the script provided with the article is recommended as the way to clone the global zone BE. My other procedures (ZFS pools as SMF services, and Zones as SMF services) installed without apparent errors, but currently this box is a single-disk rpool-only machine without zones, so there was no practical testing of that part yet. On a side note, adding pkgsrc was a breeze, except that it does refer to some libraries like "/lib/64/libz.so.1" or "/lib/64/libxml2.so.2" - but creating corresponding symlinks to objects in /usr/lib/{amd64,} helped the installation. According to http://www.kdump.cn/forums/viewtopic.php?id=1325 just touching these FS objects would've helped ("These are pkgsrc deps, not linker deps") and anyhow ldd'ing the binaries refers to /usr/lib/... paths. HTH, Jim Klimov -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Dec 8 14:22:55 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 8 Dec 2014 09:22:55 -0500 Subject: [OmniOS-discuss] Timezones In-Reply-To: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru> References: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru> Message-ID: <94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com> Updating the zoneinfo is highly annoying, and the README is wrong - if you look in the Illumos source (I upstreamed this too), it's up to one of the 2014 updates. Thanks for the README catch. When 014 gets closer to release, I'll be updating the zoneinfo again. Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 8, 2014, at 3:50 AM, Jim Klimov wrote: > > Hello all, > > The /usr/share/lib/zoneinfo/src/README mentions that the Solaris zoneinfo is based on Olson tzdata 2010k. > > Is that true or was the data since then updated and just not the readme? > > Is it safe to remove this directory and replace by either vanilla rebuild of a recent tzdata (i.e. 2014j) or some transformation thereof to make it more solaris-compatible? > > Or should it be overlaid by whatever the new version provides? > > The set of filenames is noticeably different between the two variants (installed in OS and manually compiled with zic)... > > Also, do the distributions like OmniOS or OI, or the illumos-gate itself, track the IANA (exOlson) tzdata to provide some recent version in the os/net builds? > > Thanks, > Jim Klimov > -- > Typos courtesy of K-9 Mail on my Samsung Android > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From heinrich.vanriel at outlook.com Tue Dec 2 18:35:06 2014 From: heinrich.vanriel at outlook.com (Machine Man) Date: Tue, 2 Dec 2014 13:35:06 -0500 Subject: [OmniOS-discuss] Release 12 KVM poor network performance Message-ID: I sent this as one of my aliases and it was not allowed to be delivered since I am not register with it. I sent this message days ago and then resent it with the alias that is registered. Apologies. ________________________________ From: Dan McDonald Sent: ?12/?2/?2014 10:45 To: Machine Man Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Release 12 KVM poor network performance > On Nov 29, 2014, at 3:42 PM, Machine Man wrote: > > Experience slow network performance after upgrading. > Copying from another windows system (physical machine) it drops form the 85MB/s on R10 to between 9 and 14MB/s. Two systems experience the exact same symptom. Both make use of igb driver. > Copy tests are done between windows 8.1 and Windows 2012R2 VM. > Changing to virtio for nic has no impact other than causing kvm to crash after copying ~400MB. > > Any ideas? We are behind the illumos-kvm-cmd distribution because it requires changes in illumos that Joyent hasn't pushed upstream just yet (the "vnd" project). I'm hoping to address this for '014, but I have other 014-related things to do first. Sorry, Dan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkateley at kateley.com Thu Dec 4 15:33:24 2014 From: lkateley at kateley.com (Linda Kateley) Date: Thu, 04 Dec 2014 09:33:24 -0600 Subject: [OmniOS-discuss] anyone doing DirectPath I/O? In-Reply-To: References: Message-ID: <54807EC4.6020201@kateley.com> Just curious, how fast is the nfs in this config? linda On 12/4/14, 1:11 AM, Tim Rice wrote: > On Wed, 3 Dec 2014, Schweiss, Chip wrote: > > | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years > | to an OpenIndiana server. It's been very stable: > > My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for > about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since > updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS > storage to the ESXi host for (currently) 15 running VMS as well > as well as NFS home directories for various other computers around > the office. Other than drives failing, it's been working reasonably > well for what it is. > > | > | root at zfs01:~# uptime > | 21:01pm up 649 days 6:58, 2 users, load average: 0.46, 0.30, 0.25 > | > | > | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren wrote: > | > | > Greetings, > | > > | > I was wondering if anyone was using DirectPath, specifically for exclusive > | > use of a drive controller and attached drives to a specific VM. I have a > | > use case that would seem to be a good fit for this, so I played around with > | > a couple of RAID controllers I had, and was able to get one (3ware 9650se) > | > configured for directpath, but none of the drives attached would show to > | > OmniOS, regardless of how I configured them in the raid bois (JBOD, > | > individual disks, etc). I know that controller is poorly supported and I > | > was curious if anyone was using DirectPath this way in production and what > | > kind of drive controller/HBA/whatever was working. Also any "for the love > | > of god don't do it this way" scenarios? I seem to be really adept at > | > finding and trying those out first.... > | > > | > Thanks a ton and best regards, > | > > | > -jb- > | > *Joseph Boren* > | > > | > IT Specialist > | > *DRAKE COOPER* > | > + c: (208) 891-2128 + o: (208) 342-0925 > | > + 416 S. 8th St., Boise, ID 83702 > | > + w: drakecooper.com + f: /drakecooper + > | > t: @drakecooper > | > > | > > | > _______________________________________________ > | > OmniOS-discuss mailing list > | > OmniOS-discuss at lists.omniti.com > | > http://lists.omniti.com/mailman/listinfo/omnios-discuss > | > > | > > | > From orilali at evogene.com Thu Dec 4 17:51:46 2014 From: orilali at evogene.com (Koby Schwartz) Date: Thu, 04 Dec 2014 19:51:46 +0200 Subject: [OmniOS-discuss] Kernel stability. Message-ID: <54809F32.2080703@evogene.com> Hi, My name is Kobi and I work with a company called Evogene. We have a storage server based on a Supermicro X9DRD-7LN4F (Intel(R) Xeon(R) CPU E5-2609, 64GB RAM) equipped with Intel X520-DA2 10Gb Ethernet card. Two weeks ago we encountered a problem which regards Omnios (r15012) and OpenIndiana (151_a8) kernel stability. The system is a clean install of r151012, only configuration is networking and zpool (raidz3 1 vdev of 8 disks of data and 3 parity). The storage has a single NFS share mounted (nfsvers=3) on an 8 grid nodes (running CentOS6.6) with the following mount options : async,tcp,vers=3,noacl,timeo=20,rw,bg,hard,intr,rsize=65536,wsize=65536 0 0. Under normal conditions (random reads and writes) we get great performance and stability but if we increase nfs usage the Kernel memory usage increases until the machine eventually halts. To test it we used on each node iozone ( Version $Revision: 3.394 ) which simultaneously write and reads 12 files with a size of 500MB. (iozone -r 64k -i 0 -i 1 -s 500m -t 12 -F /evolon2/iozone/`hostname`.{1..12}). While the iozone's are running the memory of Kernel (as seen by echo '::memstat'| mdb -k) is steadily increasing until the machine is dead. For clarification, when the machine is dead/halts: There is no crash files. There is no new messages in /var/adm/messages. It doesn't respond to pings (and obviously other network services). Waiting doesn't help. It can only be brought to life by power cycling it. I attached 3 files: memstat, top and arc-kstat which I hope will assist to understand why the storage died/halted Thank you in advance, Kobi. -------------- next part -------------- Thu Dec 4 16:14:02 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 5667640 22139 34% ZFS File Data 1171645 4576 7% Anon 26071 101 0% Exec and libs 1181 4 0% Page cache 4609 18 0% Free (cachelist) 7723 30 0% Free (freelist) 9887541 38623 59% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:06 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 6380424 24923 38% ZFS File Data 1171645 4576 7% Anon 26096 101 0% Exec and libs 1175 4 0% Page cache 4610 18 0% Free (cachelist) 7729 30 0% Free (freelist) 9174731 35838 55% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:09 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 6524733 25487 39% ZFS File Data 1184093 4625 7% Anon 26096 101 0% Exec and libs 1181 4 0% Page cache 4612 18 0% Free (cachelist) 7722 30 0% Free (freelist) 9017973 35226 54% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:12 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7290171 28477 43% ZFS File Data 1184093 4625 7% Anon 26122 102 0% Exec and libs 1181 4 0% Page cache 4612 18 0% Free (cachelist) 7723 30 0% Free (freelist) 8252508 32236 49% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:16 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7456599 29127 44% ZFS File Data 1190237 4649 7% Anon 26122 102 0% Exec and libs 1181 4 0% Page cache 4613 18 0% Free (cachelist) 7723 30 0% Free (freelist) 8079935 31562 48% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:19 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7480950 29222 45% ZFS File Data 1513949 5913 9% Anon 26178 102 0% Exec and libs 1180 4 0% Page cache 4615 18 0% Free (cachelist) 7724 30 0% Free (freelist) 7731814 30202 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:25 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7481258 29223 45% ZFS File Data 1513949 5913 9% Anon 26174 102 0% Exec and libs 1181 4 0% Page cache 4616 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7731509 30201 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:28 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482405 29228 45% ZFS File Data 1577181 6160 9% Anon 26200 102 0% Exec and libs 1181 4 0% Page cache 4617 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7667103 29949 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:31 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482519 29228 45% ZFS File Data 1577181 6160 9% Anon 26226 102 0% Exec and libs 1181 4 0% Page cache 4618 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666962 29949 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:35 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482590 29228 45% ZFS File Data 1577181 6160 9% Anon 26226 102 0% Exec and libs 1180 4 0% Page cache 4619 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666891 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:38 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482751 29229 45% ZFS File Data 1577181 6160 9% Anon 26252 102 0% Exec and libs 1181 4 0% Page cache 4619 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666703 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:41 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7482755 29229 45% ZFS File Data 1577181 6160 9% Anon 26252 102 0% Exec and libs 1181 4 0% Page cache 4622 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666696 29948 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:44 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7483021 29230 45% ZFS File Data 1577181 6160 9% Anon 26278 102 0% Exec and libs 1181 4 0% Page cache 4623 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7666403 29946 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:48 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7483581 29232 45% ZFS File Data 1577181 6160 9% Anon 26303 102 0% Exec and libs 1181 4 0% Page cache 4622 18 0% Free (cachelist) 7724 30 0% Free (freelist) 7665818 29944 46% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:51 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7645816 29866 46% ZFS File Data 1577181 6160 9% Anon 26328 102 0% Exec and libs 1181 4 0% Page cache 4624 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7503557 29310 45% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:55 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7645904 29866 46% ZFS File Data 1577181 6160 9% Anon 26328 102 0% Exec and libs 1181 4 0% Page cache 4625 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7503468 29310 45% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:14:58 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 7753278 30286 46% ZFS File Data 1577181 6160 9% Anon 26355 102 0% Exec and libs 1181 4 0% Page cache 4626 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7396066 28890 44% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:01 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8085909 31585 48% ZFS File Data 1577181 6160 9% Anon 26380 103 0% Exec and libs 1179 4 0% Page cache 4627 18 0% Free (cachelist) 7725 30 0% Free (freelist) 7063409 27591 42% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:04 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8085951 31585 48% ZFS File Data 1577181 6160 9% Anon 26380 103 0% Exec and libs 1181 4 0% Page cache 4628 18 0% Free (cachelist) 7723 30 0% Free (freelist) 7063366 27591 42% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:08 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8209025 32066 49% ZFS File Data 1577181 6160 9% Anon 26407 103 0% Exec and libs 1181 4 0% Page cache 4630 18 0% Free (cachelist) 7723 30 0% Free (freelist) 6940263 27110 41% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:11 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 8763032 34230 52% ZFS File Data 1577181 6160 9% Anon 26432 103 0% Exec and libs 1181 4 0% Page cache 4630 18 0% Free (cachelist) 7723 30 0% Free (freelist) 6386231 24946 38% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:14 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9388716 36674 56% ZFS File Data 1577181 6160 9% Anon 26432 103 0% Exec and libs 1181 4 0% Page cache 4631 18 0% Free (cachelist) 7724 30 0% Free (freelist) 5760545 22502 34% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:17 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9586189 37446 57% ZFS File Data 1577181 6160 9% Anon 26459 103 0% Exec and libs 1181 4 0% Page cache 4632 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5563045 21730 33% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:20 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 9967338 38934 59% ZFS File Data 1577181 6160 9% Anon 26459 103 0% Exec and libs 1181 4 0% Page cache 4633 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5181895 20241 31% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:24 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10087818 39405 60% ZFS File Data 1577181 6160 9% Anon 26484 103 0% Exec and libs 1181 4 0% Page cache 4635 18 0% Free (cachelist) 7723 30 0% Free (freelist) 5061388 19771 30% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:27 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10620046 41484 63% ZFS File Data 1577181 6160 9% Anon 26510 103 0% Exec and libs 1157 4 0% Page cache 4633 18 0% Free (cachelist) 7750 30 0% Free (freelist) 4529133 17691 27% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:30 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 10728161 41906 64% ZFS File Data 1577181 6160 9% Anon 26510 103 0% Exec and libs 1181 4 0% Page cache 4637 18 0% Free (cachelist) 7723 30 0% Free (freelist) 4421017 17269 26% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:34 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11230924 43870 67% ZFS File Data 1577181 6160 9% Anon 26536 103 0% Exec and libs 1181 4 0% Page cache 4637 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3918228 15305 23% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:38 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11277074 44051 67% ZFS File Data 1577981 6163 9% Anon 26562 103 0% Exec and libs 1181 4 0% Page cache 4638 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3871251 15122 23% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:41 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11779320 46012 70% ZFS File Data 1577981 6163 9% Anon 26562 103 0% Exec and libs 1180 4 0% Page cache 4639 18 0% Free (cachelist) 7724 30 0% Free (freelist) 3369004 13160 20% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:44 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 11968305 46751 71% ZFS File Data 1577981 6163 9% Anon 26588 103 0% Exec and libs 1181 4 0% Page cache 4641 18 0% Free (cachelist) 7723 30 0% Free (freelist) 3179991 12421 19% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:47 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12424008 48531 74% ZFS File Data 1577981 6163 9% Anon 26614 103 0% Exec and libs 1181 4 0% Page cache 4642 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2724261 10641 16% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:51 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12708257 49641 76% ZFS File Data 1577981 6163 9% Anon 26614 103 0% Exec and libs 1181 4 0% Page cache 4643 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2440011 9531 15% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:54 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 12960558 50627 77% ZFS File Data 1577981 6163 9% Anon 26640 104 0% Exec and libs 1181 4 0% Page cache 4644 18 0% Free (cachelist) 7723 30 0% Free (freelist) 2187683 8545 13% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:15:58 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 13441434 52505 80% ZFS File Data 1577981 6163 9% Anon 26664 104 0% Exec and libs 1181 4 0% Page cache 4645 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1706782 6667 10% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:01 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 13724075 53609 82% ZFS File Data 1577981 6163 9% Anon 26691 104 0% Exec and libs 1181 4 0% Page cache 4646 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1424113 5562 8% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:05 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14075430 54982 84% ZFS File Data 1577981 6163 9% Anon 26691 104 0% Exec and libs 1181 4 0% Page cache 4647 18 0% Free (cachelist) 7723 30 0% Free (freelist) 1072757 4190 6% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:08 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14302316 55868 85% ZFS File Data 1577981 6163 9% Anon 26723 104 0% Exec and libs 1181 4 0% Page cache 4649 18 0% Free (cachelist) 7723 30 0% Free (freelist) 845837 3304 5% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:12 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14585839 56975 87% ZFS File Data 1541308 6020 9% Anon 26743 104 0% Exec and libs 1181 4 0% Page cache 4649 18 0% Free (cachelist) 7724 30 0% Free (freelist) 598966 2339 4% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:16 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 14682403 57353 88% ZFS File Data 1408152 5500 8% Anon 26768 104 0% Exec and libs 1181 4 0% Page cache 4651 18 0% Free (cachelist) 7723 30 0% Free (freelist) 635532 2482 4% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:19 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15056031 58812 90% ZFS File Data 1163626 4545 7% Anon 26778 104 0% Exec and libs 1181 4 0% Page cache 4652 18 0% Free (cachelist) 7723 30 0% Free (freelist) 506419 1978 3% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:23 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15285492 59708 91% ZFS File Data 1008029 3937 6% Anon 26795 104 0% Exec and libs 1181 4 0% Page cache 4653 18 0% Free (cachelist) 7429 29 0% Free (freelist) 432831 1690 3% Total 16766410 65493 Physical 16766409 65493 Thu Dec 4 16:16:27 IST 2014 Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel 15705417 61349 94% ZFS File Data 857418 3349 5% Anon 21468 83 0% Exec and libs 73 0 0% Page cache 721 2 0% Free (cachelist) 14622 57 0% Free (freelist) 166691 651 1% Total 16766410 65493 Physical 16766409 65493 -------------- next part -------------- last pid: 996; load avg: 5.95, 3.04, 1.31; up 0+00:19:43 16:16:27 59 processes: 58 sleeping, 1 on cpu CPU states: 30.6% idle, 1.6% user, 67.8% kernel, 0.0% iowait, 0.0% swap Kernel: 97556 ctxsw, 2440 trap, 448800 intr, 27430 syscall, 1 fork, 1407 flt Memory: 64G phys mem, 933M free mem, 4096M total swap, 4096M free swap PID USERNAME NLWP PRI NICE SIZE RES STATE TIME CPU COMMAND 675 daemon 10 60 -20 2904K 1968K sleep 2:28 4.35% nfsd 607 root 1 59 0 8308K 6668K sleep 0:00 0.01% perl 631 root 1 59 0 26M 25M cpu/5 0:00 0.01% top 711 root 1 59 0 5888K 4180K sleep 0:00 0.01% perl 705 root 1 59 0 4712K 2628K sleep 0:00 0.00% perl 633 root 1 59 0 7576K 4828K sleep 0:00 0.00% sshd 712 root 1 59 0 1644K 1040K sleep 0:00 0.00% tee 622 root 1 59 0 7576K 4828K sleep 0:00 0.00% sshd 645 root 1 59 0 7576K 4832K sleep 0:00 0.00% sshd 10 root 14 59 0 7684K 5664K sleep 0:01 0.00% svc.startd 580 root 27 59 0 23M 16M sleep 0:01 0.00% fmd 706 root 1 59 0 1644K 1040K sleep 0:00 0.00% tee 12 root 19 59 0 9704K 8372K sleep 0:04 0.00% svc.configd 241 root 7 59 0 4456K 3220K sleep 0:00 0.00% devfsadm 294 root 34 59 0 7480K 4936K sleep 0:00 0.00% nscd 367 root 1 59 0 1656K 940K sleep 0:00 0.00% utmpd 658 daemon 2 60 -20 2888K 1912K sleep 0:00 0.00% lockd 232 root 5 60 -20 2712K 1600K sleep 0:00 0.00% zonestatd 322 root 4 59 0 11M 8224K sleep 0:00 0.00% hald 301 root 5 59 0 5232K 4324K sleep 0:00 0.00% picld 487 root 3 59 0 5272K 3688K sleep 0:00 0.00% inetd 331 root 2 59 0 6660K 3368K sleep 0:00 0.00% hald-runner 225 root 17 59 0 6164K 3312K sleep 0:00 0.00% syseventd 632 root 1 59 0 5804K 3192K sleep 0:00 0.00% sshd 644 root 1 59 0 5804K 3192K sleep 0:00 0.00% sshd 612 root 1 59 0 6508K 3180K sleep 0:00 0.00% hald-addon-netw 614 root 1 59 0 6508K 3180K sleep 0:00 0.00% hald-addon-cpuf 621 root 1 59 0 5804K 3096K sleep 0:00 0.00% sshd 49 netadm 3 59 0 4084K 2836K sleep 0:00 0.00% ipmgmtd 673 root 13 59 0 4588K 2744K sleep 0:00 0.00% mountd -------------- next part -------------- Thu Dec 4 16:14:39 IST 2014 l2_size = 0 other_size = 16487240 c_max = 67601473536 hash_elements = 36191 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12438 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97172664 hash_elements_max = 41359 size = 5312841912 hits = 750510 mfu_hits = 507924 prefetch_metadata_hits = 10297 hash_chains = 54 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 519 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5283795456 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1052.750004426 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 28989 arc_meta_max = 97686696 c_min = 2146100480 mru_hits = 231752 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 710687 l2_free_on_write = 0 l2_feeds = 0 misses = 19974 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:42 IST 2014 l2_size = 0 other_size = 16557200 c_max = 67601473536 hash_elements = 35929 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12446 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97848832 hash_elements_max = 41359 size = 5242345984 hits = 765628 mfu_hits = 519125 prefetch_metadata_hits = 10297 hash_chains = 53 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 544 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5213229568 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1055.756679096 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 28999 arc_meta_max = 97815864 c_min = 2146100480 mru_hits = 235669 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 725795 l2_free_on_write = 0 l2_feeds = 0 misses = 19982 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:45 IST 2014 l2_size = 0 other_size = 16815440 c_max = 67601473536 hash_elements = 34070 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12446 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97619648 hash_elements_max = 41359 size = 5487483584 hits = 791669 mfu_hits = 537122 prefetch_metadata_hits = 10297 hash_chains = 50 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 581 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5458108928 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1058.767796546 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29003 arc_meta_max = 97968040 c_min = 2146100480 mru_hits = 243713 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 751832 l2_free_on_write = 0 l2_feeds = 0 misses = 19982 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:48 IST 2014 l2_size = 0 other_size = 16858360 c_max = 67601473536 hash_elements = 36794 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12454 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98313832 hash_elements_max = 41359 size = 5462880872 hits = 802717 mfu_hits = 544723 prefetch_metadata_hits = 10297 hash_chains = 75 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 626 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5433463296 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1061.787519516 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29022 arc_meta_max = 98312872 c_min = 2146100480 mru_hits = 247160 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 762861 l2_free_on_write = 0 l2_feeds = 0 misses = 19990 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:51 IST 2014 l2_size = 0 other_size = 16888240 c_max = 67601473536 hash_elements = 40590 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12458 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98425632 hash_elements_max = 42641 size = 5669168928 hits = 815255 mfu_hits = 552397 prefetch_metadata_hits = 10297 hash_chains = 86 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 664 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5639721472 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1064.791199302 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29029 arc_meta_max = 98388464 c_min = 2146100480 mru_hits = 252024 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 775392 l2_free_on_write = 0 l2_feeds = 0 misses = 19994 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:54 IST 2014 l2_size = 0 other_size = 17262640 c_max = 67601473536 hash_elements = 36737 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12462 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98304416 hash_elements_max = 42641 size = 6396890528 hits = 840144 mfu_hits = 570407 prefetch_metadata_hits = 10297 hash_chains = 80 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 690 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6367068672 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1067.797220708 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29038 arc_meta_max = 98730168 c_min = 2146100480 mru_hits = 258903 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 800272 l2_free_on_write = 0 l2_feeds = 0 misses = 19998 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:14:57 IST 2014 l2_size = 0 other_size = 17330840 c_max = 67601473536 hash_elements = 39702 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12466 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99036168 hash_elements_max = 42641 size = 6258292744 hits = 851791 mfu_hits = 578190 prefetch_metadata_hits = 10297 hash_chains = 63 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 710 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6228402688 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1070.806236777 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29044 arc_meta_max = 99036168 c_min = 2146100480 mru_hits = 262767 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 811913 l2_free_on_write = 0 l2_feeds = 0 misses = 20002 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:00 IST 2014 l2_size = 0 other_size = 17276800 c_max = 67601473536 hash_elements = 39938 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12470 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98994416 hash_elements_max = 42641 size = 5947348208 hits = 854591 mfu_hits = 580209 prefetch_metadata_hits = 10297 hash_chains = 64 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 713 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5917512192 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1073.811610852 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29051 arc_meta_max = 99051168 c_min = 2146100480 mru_hits = 263548 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 814706 l2_free_on_write = 0 l2_feeds = 0 misses = 20006 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:03 IST 2014 l2_size = 0 other_size = 17471000 c_max = 67601473536 hash_elements = 38061 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12470 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98656136 hash_elements_max = 42641 size = 6162492296 hits = 881460 mfu_hits = 598502 prefetch_metadata_hits = 10297 hash_chains = 77 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 756 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6132462080 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1076.82642884 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29052 arc_meta_max = 99051168 c_min = 2146100480 mru_hits = 272124 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 841574 l2_free_on_write = 0 l2_feeds = 0 misses = 20006 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:06 IST 2014 l2_size = 0 other_size = 17531840 c_max = 67601473536 hash_elements = 38754 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98766128 hash_elements_max = 42641 size = 6131800368 hits = 891251 mfu_hits = 605449 prefetch_metadata_hits = 10297 hash_chains = 83 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 774 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6101709312 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1079.87068534 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 274968 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 851351 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:09 IST 2014 l2_size = 0 other_size = 17593040 c_max = 67601473536 hash_elements = 40255 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98860096 hash_elements_max = 42641 size = 6205425728 hits = 902579 mfu_hits = 612627 prefetch_metadata_hits = 10297 hash_chains = 81 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 794 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6175273472 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1082.875826687 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 279118 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 862679 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:12 IST 2014 l2_size = 0 other_size = 17598240 c_max = 67601473536 hash_elements = 43117 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12478 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98865296 hash_elements_max = 43118 size = 6013541520 hits = 905362 mfu_hits = 613511 prefetch_metadata_hits = 10297 hash_chains = 111 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 831 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5983384064 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1085.879514131 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29066 arc_meta_max = 99360176 c_min = 2146100480 mru_hits = 281017 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 865462 l2_free_on_write = 0 l2_feeds = 0 misses = 20014 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:15 IST 2014 l2_size = 0 other_size = 17603440 c_max = 67601473536 hash_elements = 43174 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12482 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99480800 hash_elements_max = 43929 size = 5807325408 hits = 908319 mfu_hits = 615797 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 832 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5777162752 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1088.884358936 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29075 arc_meta_max = 99480800 c_min = 2146100480 mru_hits = 281688 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 868410 l2_free_on_write = 0 l2_feeds = 0 misses = 20018 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:18 IST 2014 l2_size = 0 other_size = 17668200 c_max = 67601473536 hash_elements = 41509 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12488 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98984408 hash_elements_max = 43929 size = 5884816856 hits = 927065 mfu_hits = 630052 prefetch_metadata_hits = 10297 hash_chains = 93 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 843 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5854589440 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1091.888514904 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29087 arc_meta_max = 99568328 c_min = 2146100480 mru_hits = 286179 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 887144 l2_free_on_write = 0 l2_feeds = 0 misses = 20024 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:21 IST 2014 l2_size = 0 other_size = 17646400 c_max = 67601473536 hash_elements = 39031 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12488 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 98978992 hash_elements_max = 43929 size = 5865412784 hits = 955131 mfu_hits = 649821 prefetch_metadata_hits = 10297 hash_chains = 76 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 857 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5835207168 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1094.945818674 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29087 arc_meta_max = 99568328 c_min = 2146100480 mru_hits = 294476 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 915210 l2_free_on_write = 0 l2_feeds = 0 misses = 20024 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:24 IST 2014 l2_size = 0 other_size = 17752040 c_max = 67601473536 hash_elements = 41863 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12492 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99137880 hash_elements_max = 43929 size = 6192858456 hits = 968809 mfu_hits = 659725 prefetch_metadata_hits = 10297 hash_chains = 87 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 880 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6162547200 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1097.957144074 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29094 arc_meta_max = 99660136 c_min = 2146100480 mru_hits = 298250 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 928881 l2_free_on_write = 0 l2_feeds = 0 misses = 20028 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:27 IST 2014 l2_size = 0 other_size = 17651840 c_max = 67601473536 hash_elements = 44052 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12496 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99623408 hash_elements_max = 44527 size = 5948239344 hits = 972940 mfu_hits = 661821 prefetch_metadata_hits = 10297 hash_chains = 92 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 891 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5918028288 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1100.964341965 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29100 arc_meta_max = 99660136 c_min = 2146100480 mru_hits = 300285 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 933006 l2_free_on_write = 0 l2_feeds = 0 misses = 20032 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:30 IST 2014 l2_size = 0 other_size = 17736840 c_max = 67601473536 hash_elements = 39286 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12500 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99134968 hash_elements_max = 44527 size = 5866748408 hits = 999296 mfu_hits = 681359 prefetch_metadata_hits = 10297 hash_chains = 89 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 922 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5836452352 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1103.989100171 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29108 arc_meta_max = 99718496 c_min = 2146100480 mru_hits = 307103 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 959354 l2_free_on_write = 0 l2_feeds = 0 misses = 20036 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:33 IST 2014 l2_size = 0 other_size = 17708240 c_max = 67601473536 hash_elements = 43427 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12500 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99106368 hash_elements_max = 44527 size = 5953882688 hits = 1009876 mfu_hits = 687985 prefetch_metadata_hits = 10297 hash_chains = 107 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 959 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5923615232 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1106.99462468 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29109 arc_meta_max = 99718496 c_min = 2146100480 mru_hits = 311057 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 969933 l2_free_on_write = 0 l2_feeds = 0 misses = 20036 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:36 IST 2014 l2_size = 0 other_size = 17820440 c_max = 67601473536 hash_elements = 38756 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12508 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99222664 hash_elements_max = 44527 size = 5931061384 hits = 1034158 mfu_hits = 705727 prefetch_metadata_hits = 10297 hash_chains = 79 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12559216 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 970 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5900681728 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1109.998397046 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29126 arc_meta_max = 99816784 c_min = 2146100480 mru_hits = 317597 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 994198 l2_free_on_write = 0 l2_feeds = 0 misses = 20044 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:39 IST 2014 l2_size = 0 other_size = 17808640 c_max = 67601473536 hash_elements = 44706 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12508 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99212592 hash_elements_max = 44706 size = 6196209968 hits = 1039940 mfu_hits = 709638 prefetch_metadata_hits = 10297 hash_chains = 103 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1014 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6165840384 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1113.002157194 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29126 arc_meta_max = 99816784 c_min = 2146100480 mru_hits = 319468 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 999980 l2_free_on_write = 0 l2_feeds = 0 misses = 20044 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:42 IST 2014 l2_size = 0 other_size = 17820240 c_max = 67601473536 hash_elements = 42808 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12516 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99240576 hash_elements_max = 46675 size = 6415390336 hits = 1059255 mfu_hits = 723283 prefetch_metadata_hits = 10297 hash_chains = 108 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1042 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6385009152 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1116.00772062 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29140 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 325138 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1019281 l2_free_on_write = 0 l2_feeds = 0 misses = 20052 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:45 IST 2014 l2_size = 0 other_size = 17674040 c_max = 67601473536 hash_elements = 44891 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12516 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99094376 hash_elements_max = 46675 size = 5900262248 hits = 1068199 mfu_hits = 728474 prefetch_metadata_hits = 10297 hash_chains = 130 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1081 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5870027264 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1119.021235208 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29140 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 328891 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1028225 l2_free_on_write = 0 l2_feeds = 0 misses = 20052 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:48 IST 2014 l2_size = 0 other_size = 17766200 c_max = 67601473536 hash_elements = 39994 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12524 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99751784 hash_elements_max = 46675 size = 5771420520 hits = 1088758 mfu_hits = 743653 prefetch_metadata_hits = 10297 hash_chains = 94 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1092 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5741093376 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1122.026835395 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29147 arc_meta_max = 99824704 c_min = 2146100480 mru_hits = 334271 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1048777 l2_free_on_write = 0 l2_feeds = 0 misses = 20060 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:51 IST 2014 l2_size = 0 other_size = 17829240 c_max = 67601473536 hash_elements = 43380 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12524 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99249576 hash_elements_max = 46675 size = 6035421608 hits = 1096297 mfu_hits = 749091 prefetch_metadata_hits = 10297 hash_chains = 87 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1102 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6005031424 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1125.031042354 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29154 arc_meta_max = 99829992 c_min = 2146100480 mru_hits = 336372 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1056309 l2_free_on_write = 0 l2_feeds = 0 misses = 20060 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:54 IST 2014 l2_size = 0 other_size = 17900600 c_max = 67601473536 hash_elements = 39997 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12528 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99390568 hash_elements_max = 46675 size = 5948399720 hits = 1120591 mfu_hits = 765902 prefetch_metadata_hits = 10297 hash_chains = 83 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1118 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5917938176 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1128.034852134 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29163 arc_meta_max = 99971728 c_min = 2146100480 mru_hits = 343855 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1080594 l2_free_on_write = 0 l2_feeds = 0 misses = 20064 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:15:57 IST 2014 l2_size = 0 other_size = 17931600 c_max = 67601473536 hash_elements = 44425 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12532 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100027776 hash_elements_max = 46675 size = 5889137024 hits = 1123794 mfu_hits = 768068 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1137 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5858644480 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1131.040711699 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29170 arc_meta_max = 99994208 c_min = 2146100480 mru_hits = 344892 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1083790 l2_free_on_write = 0 l2_feeds = 0 misses = 20068 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:00 IST 2014 l2_size = 0 other_size = 17803560 c_max = 67601473536 hash_elements = 42523 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12532 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99334488 hash_elements_max = 46675 size = 5995791704 hits = 1137123 mfu_hits = 776941 prefetch_metadata_hits = 10297 hash_chains = 84 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1145 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5965427200 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1134.044734783 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29171 arc_meta_max = 100042776 c_min = 2146100480 mru_hits = 349348 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1097118 l2_free_on_write = 0 l2_feeds = 0 misses = 20068 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:03 IST 2014 l2_size = 0 other_size = 18005000 c_max = 67601473536 hash_elements = 40668 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12540 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100146232 hash_elements_max = 46675 size = 5979957304 hits = 1157458 mfu_hits = 791193 prefetch_metadata_hits = 10297 hash_chains = 85 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1162 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 5949391360 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1137.04862393 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29187 arc_meta_max = 100145632 c_min = 2146100480 mru_hits = 355431 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1117437 l2_free_on_write = 0 l2_feeds = 0 misses = 20076 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:06 IST 2014 l2_size = 0 other_size = 18052040 c_max = 67601473536 hash_elements = 45931 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12540 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99615736 hash_elements_max = 46725 size = 6264246264 hits = 1168001 mfu_hits = 798095 prefetch_metadata_hits = 10297 hash_chains = 98 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1192 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6233633280 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1140.052771613 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29188 arc_meta_max = 100153872 c_min = 2146100480 mru_hits = 359072 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1127979 l2_free_on_write = 0 l2_feeds = 0 misses = 20076 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:09 IST 2014 l2_size = 0 other_size = 18144240 c_max = 67601473536 hash_elements = 44399 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12548 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99752992 hash_elements_max = 47118 size = 6250227744 hits = 1186886 mfu_hits = 811377 prefetch_metadata_hits = 10297 hash_chains = 155 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1302 l2_writes_hdr_miss = 0 evict_l2_ineligible = 4096 l2_write_bytes = 0 data_size = 6219522560 c = 67601473536 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1143.058190189 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29202 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 364675 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1146850 l2_free_on_write = 0 l2_feeds = 0 misses = 20084 evict_skip = 1214 evict_l2_eligible = 71168 p = 33800736768 l2_compress_successes = 0 recycle_miss = 0 l2_writes_done = 0 ---------- Thu Dec 4 16:16:12 IST 2014 l2_size = 0 other_size = 18053440 c_max = 67601473536 hash_elements = 46005 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12552 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 100254576 hash_elements_max = 47118 size = 6189538160 hits = 1194015 mfu_hits = 815523 prefetch_metadata_hits = 10297 hash_chains = 129 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12560944 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1311 l2_writes_hdr_miss = 0 evict_l2_ineligible = 397312 l2_write_bytes = 0 data_size = 6158923776 c = 6190066904 l2_asize = 0 class = misc mutex_miss = 0 snaptime = 1146.06243032 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29214 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 367658 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1153967 l2_free_on_write = 0 l2_feeds = 0 misses = 20088 evict_skip = 1214 evict_l2_eligible = 1011712 p = 3096114796 l2_compress_successes = 0 recycle_miss = 1 l2_writes_done = 0 ---------- Thu Dec 4 16:16:15 IST 2014 l2_size = 0 other_size = 17695320 c_max = 67601473536 hash_elements = 44156 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12556 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 99370392 hash_elements_max = 47845 size = 6062509464 hits = 1214370 mfu_hits = 831061 prefetch_metadata_hits = 10297 hash_chains = 121 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 12583744 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1369 l2_writes_hdr_miss = 0 evict_l2_ineligible = 132255744 l2_write_bytes = 0 data_size = 6032230400 c = 6190066904 l2_asize = 0 class = misc mutex_miss = 1 snaptime = 1149.080481297 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29222 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 372475 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1174314 l2_free_on_write = 0 l2_feeds = 0 misses = 20092 evict_skip = 1214 evict_l2_eligible = 285229056 p = 3401023084 l2_compress_successes = 0 recycle_miss = 219 l2_writes_done = 0 ---------- Thu Dec 4 16:16:18 IST 2014 l2_size = 0 other_size = 17003120 c_max = 67601473536 hash_elements = 49196 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12556 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 97914576 hash_elements_max = 49196 size = 5597583056 hits = 1214720 mfu_hits = 831312 prefetch_metadata_hits = 10297 hash_chains = 158 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 15 hdr_size = 11820128 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1414 l2_writes_hdr_miss = 0 evict_l2_ineligible = 132255744 l2_write_bytes = 0 data_size = 5568759808 c = 5859249032 l2_asize = 0 class = misc mutex_miss = 1 snaptime = 1152.093743051 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 0 demand_metadata_hits = 29222 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 372574 demand_data_misses = 2615 duplicate_buffers = 0 demand_data_hits = 1174664 l2_free_on_write = 0 l2_feeds = 0 misses = 20092 evict_skip = 1214 evict_l2_eligible = 285491200 p = 3295003257 l2_compress_successes = 0 recycle_miss = 219 l2_writes_done = 0 ---------- Thu Dec 4 16:16:21 IST 2014 l2_size = 0 other_size = 15860920 c_max = 67601473536 hash_elements = 44130 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12564 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 91144424 hash_elements_max = 49670 size = 4804212968 hits = 1244167 mfu_hits = 851345 prefetch_metadata_hits = 10297 hash_chains = 102 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 1063 hdr_size = 11332144 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3381 l2_evict_lock_retry = 0 hash_collisions = 1433 l2_writes_hdr_miss = 0 evict_l2_ineligible = 137154560 l2_write_bytes = 0 data_size = 4777019904 c = 4879960441 l2_asize = 0 class = misc mutex_miss = 9 snaptime = 1155.099409293 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 24 demand_metadata_hits = 29238 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 381988 demand_data_misses = 2639 duplicate_buffers = 0 demand_data_hits = 1204095 l2_free_on_write = 0 l2_feeds = 0 misses = 20124 evict_skip = 1214 evict_l2_eligible = 1330413056 p = 3274886866 l2_compress_successes = 0 recycle_miss = 519 l2_writes_done = 0 ---------- Thu Dec 4 16:16:24 IST 2014 l2_size = 0 other_size = 15703800 c_max = 67601473536 hash_elements = 41125 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12572 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 85702920 hash_elements_max = 49670 size = 4396642568 hits = 1257727 mfu_hits = 860930 prefetch_metadata_hits = 10297 hash_chains = 92 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 3379 hdr_size = 10493456 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3383 l2_evict_lock_retry = 0 hash_collisions = 1442 l2_writes_hdr_miss = 0 evict_l2_ineligible = 142102528 l2_write_bytes = 0 data_size = 4370445312 c = 4396690358 l2_asize = 0 class = misc mutex_miss = 9 snaptime = 1158.10348092 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 56 demand_metadata_hits = 29252 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 385963 demand_data_misses = 2669 duplicate_buffers = 0 demand_data_hits = 1217641 l2_free_on_write = 0 l2_feeds = 0 misses = 20164 evict_skip = 1214 evict_l2_eligible = 1491629056 p = 2997814168 l2_compress_successes = 0 recycle_miss = 821 l2_writes_done = 0 ---------- Thu Dec 4 16:16:27 IST 2014 l2_size = 0 other_size = 14938800 c_max = 67601473536 hash_elements = 39052 duplicate_reads = 0 prefetch_data_hits = 537 l2_read_bytes = 0 demand_metadata_misses = 12576 prefetch_data_misses = 1540 hash_chain_max = 2 l2_hits = 0 l2_compress_failures = 0 mfu_ghost_hits = 0 l2_rw_clash = 0 memory_throttle_count = 0 crtime = 4.537754192 arc_meta_used = 80821952 hash_elements_max = 49670 size = 3828283072 hits = 1264949 mfu_hits = 864148 prefetch_metadata_hits = 10297 hash_chains = 75 evict_l2_cached = 0 l2_writes_sent = 0 l2_writes_error = 0 deleted = 10138 hdr_size = 8953872 l2_compress_zeros = 0 l2_abort_lowmem = 0 l2_io_error = 0 l2_hdr_size = 0 l2_cksum_bad = 0 l2_evict_reading = 0 arc_meta_limit = 16900368384 prefetch_metadata_misses = 3383 l2_evict_lock_retry = 0 hash_collisions = 1454 l2_writes_hdr_miss = 0 evict_l2_ineligible = 144691200 l2_write_bytes = 0 data_size = 3804390400 c = 3830309700 l2_asize = 0 class = misc mutex_miss = 12 snaptime = 1161.109480472 duplicate_buffers_size = 0 l2_misses = 0 mru_ghost_hits = 77 demand_metadata_hits = 29256 arc_meta_max = 100285688 c_min = 2146100480 mru_hits = 389967 demand_data_misses = 2692 duplicate_buffers = 0 demand_data_hits = 1224859 l2_free_on_write = 0 l2_feeds = 0 misses = 20191 evict_skip = 1214 evict_l2_eligible = 1951429632 p = 2764099257 l2_compress_successes = 0 recycle_miss = 972 l2_writes_done = 0 ---------- From jklimek at gmail.com Thu Dec 4 18:12:54 2014 From: jklimek at gmail.com (John Klimek) Date: Thu, 4 Dec 2014 13:12:54 -0500 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? Message-ID: I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client. I'm only using sys (?) authentication and not Kerberos. I've correctly set my NFS domain name but it's only partially working. Here's an example: Server: user 'media', uid 1000. Client: user 'media', uid 2000. If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user. However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid. What can I have setup wrong? I can't find any debugging or logging options for nfsmapid... Also, does the uidmap and gidmap share.nfs options only work for NFSv3? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at cos.ru Mon Dec 8 13:29:18 2014 From: jim at cos.ru (Jim Klimov) Date: Mon, 08 Dec 2014 14:29:18 +0100 Subject: [OmniOS-discuss] Bug in omnios packaging Message-ID: Hi all, Found a small issue in some of the omnios (ms.omniti.com) packages: they include symlinks that point to the build-host's absolute paths, i.e.: # find /opt/omni -ls | grep tmp 12295 0 lrwxrwxrwx 1 root root 66 Dec 8 13:39 /opt/omni/bin/pbzcat -> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2 12296 0 lrwxrwxrwx 1 root root 66 Dec 8 13:39 /opt/omni/bin/pbunzip2 -> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2 12302 0 lrwxrwxrwx 1 root root 68 Dec 8 13:39 /opt/omni/bin/amd64/unpigz -> /tmp/build_esproul/omniti_compress_pigz_pkg//opt/omni/bin/amd64/pigz Cheers, //Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.sproul at circonus.com Mon Dec 8 15:24:00 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Mon, 8 Dec 2014 10:24:00 -0500 Subject: [OmniOS-discuss] Bug in omnios packaging In-Reply-To: References: Message-ID: On Mon, Dec 8, 2014 at 8:29 AM, Jim Klimov wrote: > Hi all, > > > Found a small issue in some of the omnios (ms.omniti.com) packages: they > include symlinks that point to the build-host's absolute paths, i.e.: > > > # find /opt/omni -ls | grep tmp > 12295 0 lrwxrwxrwx 1 root root 66 Dec 8 13:39 > /opt/omni/bin/pbzcat -> > /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2 > 12296 0 lrwxrwxrwx 1 root root 66 Dec 8 13:39 > /opt/omni/bin/pbunzip2 -> > /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2 > 12302 0 lrwxrwxrwx 1 root root 68 Dec 8 13:39 > /opt/omni/bin/amd64/unpigz -> > /tmp/build_esproul/omniti_compress_pigz_pkg//opt/omni/bin/amd64/pigz Haha... busted! Thanks for the bug report. For whatever reason, these compression tools all have very non-standard build procedures, and make all sorts of silly assumptions. We'll get it fixed. Eric From danmcd at omniti.com Mon Dec 8 15:52:48 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 8 Dec 2014 10:52:48 -0500 Subject: [OmniOS-discuss] Timezones In-Reply-To: <94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com> References: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru> <94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com> Message-ID: <88FF83DA-E619-490C-94E2-C37173BEAC8B@omniti.com> > On Dec 8, 2014, at 9:22 AM, Dan McDonald wrote: > > Updating the zoneinfo is highly annoying, and the README is wrong - if you look in the Illumos source (I upstreamed this too), it's up to one of the 2014 updates. > > Thanks for the README catch. When 014 gets closer to release, I'll be updating the zoneinfo again. The annoying parts of all of this are documented in $ILLUMOS_GATE/usr/src/cmd/zic/README.illumos Couldn't remember that until I had my workstation in front of me. Dan From chip at innovates.com Mon Dec 8 16:09:02 2014 From: chip at innovates.com (Schweiss, Chip) Date: Mon, 8 Dec 2014 10:09:02 -0600 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> Message-ID: I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18. I'm getting errors when trying to downgrade: Attempting to flash firmware to LSI SAS SAS2308_2(D1) : Executing Operation: Flash Firmware Image Firmware Image has a Valid Checksum. Firmware Version 18.00.00.00 Firmware Image compatible with Controller. Valid NVDATA Image found. NVDATA Version 11.00.00.00 Checking for a compatible NVData image... NVDATA Device ID and Chip Revision match verified. ERROR: Cannot downgrade NVDATA version 14.00.00.06 to 11.00.110000.00. ERROR: Failed to get valid NVDATA image from File! Firmware Image Validation Failed! Tried downgrading bios: Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : Validating BIOS Image... BIOS Header Signature is Valid BIOS Image has a Valid Checksum. BIOS PCI Structure Signature Valid. BIOS Image Compatible with the SAS Controller. Attempting to Flash BIOS Image... BIOS Version in flash : 07.39.00.00 BIOS Version from File : 07.35.00.00 Skipping flash since file version is not greater than existing. Flash BIOS Image Failed! I've tried sas2flash from P20 and P18. Can someone fill me in on the trick to downgrade? -Chip On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan wrote: > Thank you for your help Aaron! I downgraded firmware to P18 and there are > no more errors today. > > It seems, that there is something bad with P20 firmware! > > > > You saved me a lot of time J > > > > Filip > > > > > > *From:* Aaron Curry [mailto:asc1111 at gmail.com] > *Sent:* Tuesday, November 25, 2014 5:46 PM > *To:* Filip Marvan > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > > I had this exact same problem recently when setting up a new home > server... same controller, same firmware (P20). The errors were on all > disks attached to the controller, but only on high read activity. Writes > did not generate the errors. I downgraded the firmware to P18 (which is > what we are using at work) and the errors went away. Has anyone had success > with this controller and the P20 firmware? > > > > Aaron > > > > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan > wrote: > > Hi Dan, > > thanks for reply. > Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other > disks in different pool but on the same SAS cable. > So maybe I only had bad luck with that disks. > > Filip > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.lagoe at subrigo.net Mon Dec 8 16:15:08 2014 From: matthew.lagoe at subrigo.net (Matthew Lagoe) Date: Mon, 8 Dec 2014 08:15:08 -0800 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> Message-ID: <004501d01302$257c9b90$7075d2b0$@subrigo.net> I think you have to use a special flag to force it From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Schweiss, Chip Sent: Monday, December 08, 2014 08:09 AM To: Filip Marvan Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18. I'm getting errors when trying to downgrade: Attempting to flash firmware to LSI SAS SAS2308_2(D1) : Executing Operation: Flash Firmware Image Firmware Image has a Valid Checksum. Firmware Version 18.00.00.00 Firmware Image compatible with Controller. Valid NVDATA Image found. NVDATA Version 11.00.00.00 Checking for a compatible NVData image... NVDATA Device ID and Chip Revision match verified. ERROR: Cannot downgrade NVDATA version 14.00.00.06 to 11.00.110000.00. ERROR: Failed to get valid NVDATA image from File! Firmware Image Validation Failed! Tried downgrading bios: Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : Validating BIOS Image... BIOS Header Signature is Valid BIOS Image has a Valid Checksum. BIOS PCI Structure Signature Valid. BIOS Image Compatible with the SAS Controller. Attempting to Flash BIOS Image... BIOS Version in flash : 07.39.00.00 BIOS Version from File : 07.35.00.00 Skipping flash since file version is not greater than existing. Flash BIOS Image Failed! I've tried sas2flash from P20 and P18. Can someone fill me in on the trick to downgrade? -Chip On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan wrote: Thank you for your help Aaron! I downgraded firmware to P18 and there are no more errors today. It seems, that there is something bad with P20 firmware! You saved me a lot of time J Filip From: Aaron Curry [mailto:asc1111 at gmail.com] Sent: Tuesday, November 25, 2014 5:46 PM To: Filip Marvan Cc: Dan McDonald; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I had this exact same problem recently when setting up a new home server... same controller, same firmware (P20). The errors were on all disks attached to the controller, but only on high read activity. Writes did not generate the errors. I downgraded the firmware to P18 (which is what we are using at work) and the errors went away. Has anyone had success with this controller and the P20 firmware? Aaron On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan wrote: Hi Dan, thanks for reply. Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other disks in different pool but on the same SAS cable. So maybe I only had bad luck with that disks. Filip _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From ikaufman at eng.ucsd.edu Mon Dec 8 16:54:10 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Mon, 8 Dec 2014 08:54:10 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> Message-ID: No, what we are saying is NFSv4 and RPC are not compatible right now, and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number over RPC. If you are not going to use Kerberos or LDAP, then you need to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux are incompatible, it's that RPC does not support NFSv4. Ian On Mon, Dec 8, 2014 at 5:25 AM, John Klimek wrote: > Thanks everybody. > > Are you guys saying that Solaris and Linux are incompatible for sharing > NFSv4 ACLs (because of RPC differences, etc) ? > > On Sun, Dec 7, 2014 at 5:43 PM, Frank Pittel wrote: >> >> I've tried doing this with NFS4 a couple of years ago and what I found was >> that >> while the UID mapping worked for things like "ls" anytime you tried to >> actually >> do anything with the file RPC calls were made and since that's outside of >> NFS they >> failed. >> >> It was a couple of years ago now since I went through it but the upshot is >> that >> at least then uid mapping didn't work in any meaningful way! >> >> The Other Frank >> >> >> >> On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote: >> > Thanks, I will give this a try. >> > >> > However, when I create a file if I check the acl on Omni, it does have >> > extended permissions and looks correct. It's only the uid and gid that >> > I'm >> > trying to get mapped correctly. >> > >> > Do you still think that noacl will solve that problem? >> > >> > (I'm migrating some virtual machines so I can't try out the setting you >> > mentioned until a few more minutes...) >> > >> > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen wrote: >> > >> > > On Thu, 4 Dec 2014 13:20:04 -0500 >> > > John Klimek wrote: >> > > >> > > > >> > > > What can I have setup wrong? >> > > > >> > > > I can't find any debugging or logging options for nfsmapid... >> > > > >> > > > Also, does the uidmap and gidmap share.nfs options only work for >> > > > NFSv3? >> > > Solaris and derivatives implementation of NFS ACL is not compliant to >> > > the >> > > Linux NFS ACL. >> > > More directly it is the ACL in Linux which is not POSIX conformant so >> > > to >> > > avoid problems you >> > > should add the mount option noacl in your Linux fstab file. Noacl will >> > > instruct Omnios NFS >> > > to revert to plain old uid/gid. >> > > >> > > -- >> > > Hilsen/Regards >> > > Michael Rasmussen >> > > >> > > Get my public GnuPG keys: >> > > michael rasmussen cc >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E >> > > mir datanom net >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C >> > > mir miras org >> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 >> > > -------------------------------------------------------------- >> > > /usr/games/fortune -es says: >> > > Don't just echo the code with comments - make every comment count. >> > > - The Elements of Programming Style (Kernighan & Plaugher) >> > > >> > > _______________________________________________ >> > > OmniOS-discuss mailing list >> > > OmniOS-discuss at lists.omniti.com >> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > >> > > >> >> > _______________________________________________ >> > OmniOS-discuss mailing list >> > OmniOS-discuss at lists.omniti.com >> > http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From tim at multitalents.net Mon Dec 8 18:05:34 2014 From: tim at multitalents.net (Tim Rice) Date: Mon, 8 Dec 2014 10:05:34 -0800 (PST) Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> Message-ID: On Mon, 8 Dec 2014, Schweiss, Chip wrote: | I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 | to 18. | | I'm getting errors when trying to downgrade: Try erasing first. On the 6G cards it is "sas2flsh -o -e 6". Once erased, be sure to flash before rebooting or the card will need to be sent back. | Attempting to flash firmware to LSI SAS SAS2308_2(D1) : | | Executing Operation: Flash Firmware Image | | Firmware Image has a Valid Checksum. | Firmware Version 18.00.00.00 | Firmware Image compatible with Controller. | | Valid NVDATA Image found. | NVDATA Version 11.00.00.00 | Checking for a compatible NVData image... | | NVDATA Device ID and Chip Revision match verified. | ERROR: Cannot downgrade NVDATA version 14.00.00.06 | to 11.00.110000.00. | | ERROR: Failed to get valid NVDATA image from File! | | Firmware Image Validation Failed! | | Tried downgrading bios: | | Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : | | Validating BIOS Image... | | BIOS Header Signature is Valid | | BIOS Image has a Valid Checksum. | | BIOS PCI Structure Signature Valid. | | BIOS Image Compatible with the SAS Controller. | | Attempting to Flash BIOS Image... | | BIOS Version in flash : 07.39.00.00 | BIOS Version from File : 07.35.00.00 | Skipping flash since file version is not greater than | existing. | | Flash BIOS Image Failed! | | | I've tried sas2flash from P20 and P18. | | Can someone fill me in on the trick to downgrade? | | -Chip | | | On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan wrote: | | > Thank you for your help Aaron! I downgraded firmware to P18 and there are | > no more errors today. | > | > It seems, that there is something bad with P20 firmware! | > | > | > | > You saved me a lot of time J | > | > | > | > Filip | > | > | > | > | > | > *From:* Aaron Curry [mailto:asc1111 at gmail.com] | > *Sent:* Tuesday, November 25, 2014 5:46 PM | > *To:* Filip Marvan | > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com | > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2 | > | > | > | > I had this exact same problem recently when setting up a new home | > server... same controller, same firmware (P20). The errors were on all | > disks attached to the controller, but only on high read activity. Writes | > did not generate the errors. I downgraded the firmware to P18 (which is | > what we are using at work) and the errors went away. Has anyone had success | > with this controller and the P20 firmware? | > | > | > | > Aaron | > | > | > | > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan | > wrote: | > | > Hi Dan, | > | > thanks for reply. | > Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other | > disks in different pool but on the same SAS cable. | > So maybe I only had bad luck with that disks. | > | > Filip | > -- Tim Rice Multitalents tim at multitalents.net From eric.sproul at circonus.com Mon Dec 8 18:17:13 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Mon, 8 Dec 2014 13:17:13 -0500 Subject: [OmniOS-discuss] Bug in omnios packaging In-Reply-To: References: Message-ID: On Mon, Dec 8, 2014 at 10:24 AM, Eric Sproul wrote: > Haha... busted! Thanks for the bug report. For whatever reason, > these compression tools all have very non-standard build procedures, > and make all sorts of silly assumptions. We'll get it fixed. The fix was to make the symlink target relative instead of absolute. pkg://ms.omniti.com/omniti/compress/pbzip2 at 1.1.8,5.11-0.151006:20141208T180943Z is the updated package. Thanks again for pointing that out! Eric From henson at acm.org Mon Dec 8 20:23:20 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 8 Dec 2014 12:23:20 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> Message-ID: <674b01d01324$d0e28180$72a78480$@acm.org> > From: Ian Kaufman > Sent: Monday, December 08, 2014 8:54 AM > > No, what we are saying is NFSv4 and RPC are not compatible right now, > and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number > over RPC. If you are not going to use Kerberos or LDAP, then you need > to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux > are incompatible, it's that RPC does not support NFSv4. Well, I don't think it's technically accurate to say NFSv4 and RPC are not compatible, nor that RPC does not support NFSv4. It's really more of an issue that the NFSv4 protocol failed to address ID mapping completely for nonsecure RPC methods. For secure RPC (such as Kerberos), identifiers are passed over the wire symbolically as strings, and the ID mapper on each side translates them to numeric identifiers. However, for insecure RPC using AUTH_SYS, NFSv4 continues to pass raw uid/gid identifiers over the wire. This limitation of the protocol renders ID mapping somewhat useless for NFSv4 over insecure RPC :(. Ideally they could address it in v4.1 or v4.2 by creating a new insecure AUTH_SYS_NAMES protocol, which would simply use symbolic string identifiers over the wire like secure NFS does. I guess there is not much interest or concern about this in the communities that deal with NFS protocol specifications 8-/. Perhaps the majority of deployments for people in decision-making capacity use secure NFS and thus are not impacted by this deficiency... From ikaufman at eng.ucsd.edu Mon Dec 8 20:37:25 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Mon, 8 Dec 2014 12:37:25 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <674b01d01324$d0e28180$72a78480$@acm.org> References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> <674b01d01324$d0e28180$72a78480$@acm.org> Message-ID: Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX does not support NFSv4. AUTH_SYS uses the UID, not the name, so the mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL. http://thread.gmane.org/gmane.linux.nfsv4/7103/focus=7105 Supposedly. at least on the client side, this has been fixed somewhere upstream. However, the server side is not. https://bugzilla.linux-nfs.org/show_bug.cgi?id=226 Regardless, my point is that this is not a Solaris/Linux issue, as a Linux server and Linux clients would be in the same boat. On Mon, Dec 8, 2014 at 12:23 PM, Paul B. Henson wrote: >> From: Ian Kaufman >> Sent: Monday, December 08, 2014 8:54 AM >> >> No, what we are saying is NFSv4 and RPC are not compatible right now, >> and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number >> over RPC. If you are not going to use Kerberos or LDAP, then you need >> to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux >> are incompatible, it's that RPC does not support NFSv4. > > Well, I don't think it's technically accurate to say NFSv4 and RPC are not > compatible, nor that RPC does not support NFSv4. > > It's really more of an issue that the NFSv4 protocol failed to address ID > mapping completely for nonsecure RPC methods. For secure RPC (such as > Kerberos), identifiers are passed over the wire symbolically as strings, and > the ID mapper on each side translates them to numeric identifiers. However, > for insecure RPC using AUTH_SYS, NFSv4 continues to pass raw uid/gid > identifiers over the wire. > > This limitation of the protocol renders ID mapping somewhat useless for > NFSv4 over insecure RPC :(. Ideally they could address it in v4.1 or v4.2 by > creating a new insecure AUTH_SYS_NAMES protocol, which would simply use > symbolic string identifiers over the wire like secure NFS does. I guess > there is not much interest or concern about this in the communities that > deal with NFS protocol specifications 8-/. Perhaps the majority of > deployments for people in decision-making capacity use secure NFS and thus > are not impacted by this deficiency... > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From henson at acm.org Mon Dec 8 21:42:32 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 8 Dec 2014 13:42:32 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> <674b01d01324$d0e28180$72a78480$@acm.org> Message-ID: <6e8301d0132f$e1702c40$a45084c0$@acm.org> > From: Ian Kaufman > Sent: Monday, December 08, 2014 12:37 PM > > Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX > does not support NFSv4. AUTH_SYS uses the UID, not the name, so the > mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL. I apologize for being a pedant; it's a character flaw :). Your wording implies that you cannot use AUTH_SYS with NFSv4, which is not true. NFSv4 works perfectly fine with AUTH_SYS as long as you maintain synchronization of uid/gid's on both sides. I would pick NFSv4 with AUTH_SYS over NFSv3 with AUTH_SYS. Both require that you maintain uid synchronization, so it's not like you're gaining something by falling back, and why miss out on the other features of NFSv4? > Supposedly. at least on the client side, this has been fixed somewhere > upstream. However, the server side is not. > > https://bugzilla.linux-nfs.org/show_bug.cgi?id=226 I don't know if I would call this "fixed" 8-/. They are basically just disabling the idmapper and passing raw uid/gid info at the NFS level to match the raw info at the RPC level. I guess it makes it less confusing in such a scenario, because you're always broken if they don't match on each side rather than only broken in some cases. An actual fix would be introducing an RPC mechanism using names such as AUTH_SYS_NAME, one of these days maybe I'll find the time to go hassle some NFS developers and see why they don't just do that and make everything simpler. > Regardless, my point is that this is not a Solaris/Linux issue, as a > Linux server and Linux clients would be in the same boat. Agreed. It is a deficiency in the NFSv4 protocol :(... From info at houseofancients.nl Mon Dec 8 23:14:58 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Mon, 8 Dec 2014 23:14:58 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 Message-ID: <356582D1FC91784992ABB4265A16ED487EBAE58F@vEX01.mindstorm-internet.local> Guys, been having a odd issue with my Omnios box and c3750 the omnios box has 6 interfaces ( 2 e1000g and 4 igb) somehow the LACP bonding between the igb's and my cisco's will not stay in sync, while the same lacp config runs fine over the e1000's : channel 5 = e1000g's channel 6 and 7 are the igb's : Channel group 5 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi1/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x2 0x3F Gi2/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x1 0x3F Channel group 6 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi1/0/10 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x6 0x47 Gi1/0/12 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x5 0x47 Channel group 7 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi2/0/10 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x4 0x47 Gi2/0/11 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x3 0x47 LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS aggr0 L4 auto active short ----- aggr1 L4 auto active short ----- aggr2 L4 auto active short ----- aggr0 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 e1000g0 attached up 1000Mb full - - - x:x:x:x:x:x 1500 e1000g1 attached up 1000Mb full - - - x:x:x:x:x:x 1500 aggr1 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 igb0 attached up 1000Mb full - - - x:x:x:x:x:x 1500 igb1 attached up 1000Mb full - - - x:x:x:x:x:x 1500 aggr2 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 igb2 attached up 1000Mb full - - - x:x:x:x:x:x 1500 igb3 attached up 1000Mb full - - - x:x:x:x:x:x 1500 058673: Dec 8 22:15:51.593: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 1(selected) 058674: Dec 8 22:15:51.593: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING 058675: Dec 8 22:15:51.593: LACP: Gi2/0/10 lacp_action_mx_waiting entered 058676: Dec 8 22:15:51.593: LACP: timer lacp_w(Gi2/0/10) started with interval 2000. 058677: Dec 8 22:15:51.895: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/10 058678: Dec 8 22:15:51.895: LACP : packet size: 124 058679: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1 058680: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058681: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA, s-pri:0x0, s-mac:0000.0000.0000 058682: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA 058683: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0 058684: Dec 8 22:15:51.895: LACP: Gi2/0/10 LACP packet received, processing 058685: Dec 8 22:15:51.895: lacp_rx Gi2/0/10 - rx: during state CURRENT, got event 5(recv_lacpdu) 058686: Dec 8 22:15:51.895: @@@ lacp_rx Gi2/0/10 - rx: CURRENT -> CURRENT 058687: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_rx_current entered 058688: Dec 8 22:15:51.895: LACP: Gi2/0/10 set to UNSELECTED 058689: Dec 8 22:15:51.895: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 3(unselected) 058690: Dec 8 22:15:51.895: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> DETACHED 058691: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_mx_detached entered 058692: Dec 8 22:15:51.895: LACP: Gi2/0/10 Resetting hw_sw constraints 058693: Dec 8 22:15:51.895: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag. 058694: Dec 8 22:15:51.895: LACP: lacp_send_lacpdu: (Gi2/0/10) About to send the 110 LACPDU 058695: Dec 8 22:15:51.895: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/10 058696: Dec 8 22:15:51.895: LACP : packet size: 124 058697: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1 058698: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20B, p-state:0x5, s-pri:0x8000, s-mac:0019.aa89.b580 058699: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058700: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000 058701: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0 058702: Dec 8 22:15:51.895: LACP: lacp_write: LACP 124 bytes out Gi2/0/10 058703: Dec 8 22:15:51.895: lacp_handle_standby_port_internal called, depth = 1 058704: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 LACP PDU Rcvd. Partners oper state is hex 47 058705: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 Partner out of sync 058706: Dec 8 22:15:51.895: LACP: Gi2/0/10 Partners oper state is hex 47 058707: Dec 8 22:15:51.895: LACP: timer lacp_c_l(Gi2/0/10) started with interval 90000. 058708: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG_PARTNER_UP. 058709: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG unchanged 058710: Dec 8 22:15:51.903: LACP: Gi2/0/10 STANDBY aggregator hex address is 5FB71A4 058711: Dec 8 22:15:51.903: LACP: Gi2/0/10 set to STANDBY 058712: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state DETACHED, got event 2(standby) 058713: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: DETACHED -> WAITING 058714: Dec 8 22:15:51.903: LACP: Gi2/0/10 lacp_action_mx_waiting entered 058715: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING 058716: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 6(outof_sync) (ignored) 058717: Dec 8 22:15:51.903: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/11 058718: Dec 8 22:15:51.903: LACP : packet size: 124 058719: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1 058720: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058721: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA, s-pri:0x0, s-mac:0000.0000.0000 058722: Dec 8 22:15:51.903: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA 058723: Dec 8 22:15:51.903: LACP: term-tlv:0 termr-tlv-len:0 058724: Dec 8 22:15:51.903: LACP: Gi2/0/11 LACP packet received, processing 058725: Dec 8 22:15:51.903: lacp_rx Gi2/0/11 - rx: during state CURRENT, got event 5(recv_lacpdu) 058726: Dec 8 22:15:51.903: @@@ lacp_rx Gi2/0/11 - rx: CURRENT -> CURRENT 058727: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_rx_current entered 058728: Dec 8 22:15:51.903: LACP: Gi2/0/11 set to UNSELECTED 058729: Dec 8 22:15:51.903: lacp_mux Gi2/0/11 - mux: during state WAITING, got event 3(unselected) 058730: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/11 - mux: WAITING -> DETACHED 058731: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_mx_detached entered 058732: Dec 8 22:15:51.903: LACP: Gi2/0/11 Resetting hw_sw constraints 058733: Dec 8 22:15:51.903: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag. 058734: Dec 8 22:15:51.903: LACP: lacp_send_lacpdu: (Gi2/0/11) About to send the 110 LACPDU 058735: Dec 8 22:15:51.903: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/11 058736: Dec 8 22:15:51.903: LACP : packet size: 124 058737: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1 058738: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20C, p-state:0x5, s-pri:0x8000, s-mac:0019.aa89.b580 058739: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 so in short, i see it coming up, shows out of sync, and down it goes... have tried short times, long times active , passive... nothing will work Met vriendelijke groet / With kind regards, Floris van Essen / Claudia Dufornee House of Ancients Amstaffs info at houseofancients.nl www.houseofancients.nl ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl -------------- next part -------------- An HTML attachment was scrubbed... URL: From filip.marvan at aira.cz Tue Dec 9 09:06:01 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Tue, 9 Dec 2014 10:06:01 +0100 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <004501d01302$257c9b90$7075d2b0$@subrigo.net> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> Hi, first you have to erase P20 firmware with: sas2flsh -o -e 6 now do not reboot(!) and flash P18 with sas2flsh -f XXXXXX.bin -b mptsas2.rom Filip From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Schweiss, Chip Sent: Monday, December 08, 2014 08:09 AM To: Filip Marvan Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18. I'm getting errors when trying to downgrade: Attempting to flash firmware to LSI SAS SAS2308_2(D1) : Executing Operation: Flash Firmware Image Firmware Image has a Valid Checksum. Firmware Version 18.00.00.00 Firmware Image compatible with Controller. Valid NVDATA Image found. NVDATA Version 11.00.00.00 Checking for a compatible NVData image... NVDATA Device ID and Chip Revision match verified. ERROR: Cannot downgrade NVDATA version 14.00.00.06 to 11.00.110000.00. ERROR: Failed to get valid NVDATA image from File! Firmware Image Validation Failed! Tried downgrading bios: Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : Validating BIOS Image... BIOS Header Signature is Valid BIOS Image has a Valid Checksum. BIOS PCI Structure Signature Valid. BIOS Image Compatible with the SAS Controller. Attempting to Flash BIOS Image... BIOS Version in flash : 07.39.00.00 BIOS Version from File : 07.35.00.00 Skipping flash since file version is not greater than existing. Flash BIOS Image Failed! I've tried sas2flash from P20 and P18. Can someone fill me in on the trick to downgrade? -Chip On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan > wrote: Thank you for your help Aaron! I downgraded firmware to P18 and there are no more errors today. It seems, that there is something bad with P20 firmware! You saved me a lot of time :) Filip From: Aaron Curry [mailto:asc1111 at gmail.com] Sent: Tuesday, November 25, 2014 5:46 PM To: Filip Marvan Cc: Dan McDonald; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I had this exact same problem recently when setting up a new home server... same controller, same firmware (P20). The errors were on all disks attached to the controller, but only on high read activity. Writes did not generate the errors. I downgraded the firmware to P18 (which is what we are using at work) and the errors went away. Has anyone had success with this controller and the P20 firmware? Aaron On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan > wrote: Hi Dan, thanks for reply. Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other disks in different pool but on the same SAS cable. So maybe I only had bad luck with that disks. Filip _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From chip at innovates.com Tue Dec 9 14:49:30 2014 From: chip at innovates.com (Schweiss, Chip) Date: Tue, 9 Dec 2014 08:49:30 -0600 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> Message-ID: I'm still fighting this. Under OmniOS the erase function is disabled in sas2flash for solaris. I couldn't seem to find an iso bootable DOS that I could stage files in. FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode. Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation Failed!" Tried both P18 sas2flash and P20 sas2flash. What OS environment are you running this in? On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan wrote: > Hi, > > > > first you have to erase P20 firmware with: > > > > sas2flsh -o -e 6 > > > > now do not reboot(!) and flash P18 with > > > > sas2flsh -f XXXXXX.bin -b mptsas2.rom > > > > Filip > > > > > > *From:* OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] *On > Behalf Of *Schweiss, Chip > *Sent:* Monday, December 08, 2014 08:09 AM > *To:* Filip Marvan > *Cc:* omnios-discuss at lists.omniti.com > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > > I've got some new LSI HBAs I'm trying to downgrade from firmware version > 20 to 18. > > I'm getting errors when trying to downgrade: > > Attempting to flash firmware to LSI SAS SAS2308_2(D1) : > > Executing Operation: Flash Firmware Image > > Firmware Image has a Valid Checksum. > Firmware Version 18.00.00.00 > Firmware Image compatible with Controller. > > Valid NVDATA Image found. > NVDATA Version 11.00.00.00 > Checking for a compatible NVData image... > > NVDATA Device ID and Chip Revision match verified. > ERROR: Cannot downgrade NVDATA version 14.00.00.06 > to 11.00.110000.00. > > ERROR: Failed to get valid NVDATA image from File! > > Firmware Image Validation Failed! > > Tried downgrading bios: > > Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : > > Validating BIOS Image... > > BIOS Header Signature is Valid > > BIOS Image has a Valid Checksum. > > BIOS PCI Structure Signature Valid. > > BIOS Image Compatible with the SAS Controller. > > Attempting to Flash BIOS Image... > > BIOS Version in flash : 07.39.00.00 > BIOS Version from File : 07.35.00.00 > Skipping flash since file version is not greater than > existing. > > Flash BIOS Image Failed! > > I've tried sas2flash from P20 and P18. > > Can someone fill me in on the trick to downgrade? > > -Chip > > > > > > On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan > wrote: > > Thank you for your help Aaron! I downgraded firmware to P18 and there are > no more errors today. > > It seems, that there is something bad with P20 firmware! > > > > You saved me a lot of time J > > > > Filip > > > > > > *From:* Aaron Curry [mailto:asc1111 at gmail.com] > *Sent:* Tuesday, November 25, 2014 5:46 PM > *To:* Filip Marvan > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > > I had this exact same problem recently when setting up a new home > server... same controller, same firmware (P20). The errors were on all > disks attached to the controller, but only on high read activity. Writes > did not generate the errors. I downgraded the firmware to P18 (which is > what we are using at work) and the errors went away. Has anyone had success > with this controller and the P20 firmware? > > > > Aaron > > > > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan > wrote: > > Hi Dan, > > thanks for reply. > Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other > disks in different pool but on the same SAS cable. > So maybe I only had bad luck with that disks. > > Filip > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From filip.marvan at aira.cz Tue Dec 9 14:58:05 2014 From: filip.marvan at aira.cz (Filip Marvan) Date: Tue, 9 Dec 2014 15:58:05 +0100 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local> Yes, you have to use bootable DOS and DOS version of sas2flash. There is an awesome tool for creating bootable USB stick with DOS called Rufus http://rufus.akeo.ie/ Filip From: Schweiss, Chip [mailto:chip at innovates.com] Sent: Tuesday, December 09, 2014 3:50 PM To: Filip Marvan Cc: Matthew Lagoe; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I'm still fighting this. Under OmniOS the erase function is disabled in sas2flash for solaris. I couldn't seem to find an iso bootable DOS that I could stage files in. FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode. Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation Failed!" Tried both P18 sas2flash and P20 sas2flash. What OS environment are you running this in? On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan wrote: Hi, first you have to erase P20 firmware with: sas2flsh -o -e 6 now do not reboot(!) and flash P18 with sas2flsh -f XXXXXX.bin -b mptsas2.rom Filip From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Schweiss, Chip Sent: Monday, December 08, 2014 08:09 AM To: Filip Marvan Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18. I'm getting errors when trying to downgrade: Attempting to flash firmware to LSI SAS SAS2308_2(D1) : Executing Operation: Flash Firmware Image Firmware Image has a Valid Checksum. Firmware Version 18.00.00.00 Firmware Image compatible with Controller. Valid NVDATA Image found. NVDATA Version 11.00.00.00 Checking for a compatible NVData image... NVDATA Device ID and Chip Revision match verified. ERROR: Cannot downgrade NVDATA version 14.00.00.06 to 11.00.110000.00. ERROR: Failed to get valid NVDATA image from File! Firmware Image Validation Failed! Tried downgrading bios: Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : Validating BIOS Image... BIOS Header Signature is Valid BIOS Image has a Valid Checksum. BIOS PCI Structure Signature Valid. BIOS Image Compatible with the SAS Controller. Attempting to Flash BIOS Image... BIOS Version in flash : 07.39.00.00 BIOS Version from File : 07.35.00.00 Skipping flash since file version is not greater than existing. Flash BIOS Image Failed! I've tried sas2flash from P20 and P18. Can someone fill me in on the trick to downgrade? -Chip On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan wrote: Thank you for your help Aaron! I downgraded firmware to P18 and there are no more errors today. It seems, that there is something bad with P20 firmware! You saved me a lot of time J Filip From: Aaron Curry [mailto:asc1111 at gmail.com] Sent: Tuesday, November 25, 2014 5:46 PM To: Filip Marvan Cc: Dan McDonald; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 I had this exact same problem recently when setting up a new home server... same controller, same firmware (P20). The errors were on all disks attached to the controller, but only on high read activity. Writes did not generate the errors. I downgraded the firmware to P18 (which is what we are using at work) and the errors went away. Has anyone had success with this controller and the P20 firmware? Aaron On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan wrote: Hi Dan, thanks for reply. Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other disks in different pool but on the same SAS cable. So maybe I only had bad luck with that disks. Filip _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6247 bytes Desc: not available URL: From hasslerd at gmx.li Tue Dec 9 15:51:03 2014 From: hasslerd at gmx.li (Dominik Hassler) Date: Tue, 09 Dec 2014 16:51:03 +0100 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> Message-ID: <54871A67.3040805@gmx.li> Chip, when I had to reflash an onboard LSI2308 I used the UEFI shell. No fiddling around with boot disks etc. Not sure, if it is applicable to your problem. On 12/09/2014 03:49 PM, Schweiss, Chip wrote: > I'm still fighting this. > > Under OmniOS the erase function is disabled in sas2flash for solaris. > > I couldn't seem to find an iso bootable DOS that I could stage files > in. FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode. > Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation > Failed!" Tried both P18 sas2flash and P20 sas2flash. > > What OS environment are you running this in? > > > > On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan > wrote: > > Hi,____ > > __ __ > > first you have to erase P20 firmware with:____ > > __ __ > > sas2flsh -o -e 6____ > > __ __ > > now do not reboot(!) and flash P18 with____ > > __ __ > > sas2flsh -f XXXXXX.bin -b mptsas2.rom____ > > __ __ > > Filip____ > > __ __ > > __ __ > > *From:*OmniOS-discuss > [mailto:omnios-discuss-bounces at lists.omniti.com > ] *On Behalf Of > *Schweiss, Chip > *Sent:* Monday, December 08, 2014 08:09 AM > *To:* Filip Marvan > *Cc:* omnios-discuss at lists.omniti.com > > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____ > > __ __ > > I've got some new LSI HBAs I'm trying to downgrade from firmware > version 20 to 18.____ > > I'm getting errors when trying to downgrade: > > Attempting to flash firmware to LSI SAS SAS2308_2(D1) : > > Executing Operation: Flash Firmware Image > > Firmware Image has a Valid Checksum. > Firmware Version 18.00.00.00 > Firmware Image compatible with Controller. > > Valid NVDATA Image found. > NVDATA Version 11.00.00.00 > Checking for a compatible NVData image... > > NVDATA Device ID and Chip Revision match verified. > ERROR: Cannot downgrade NVDATA version 14.00.00.06 > to 11.00.110000.00. > > ERROR: Failed to get valid NVDATA image from File! > > Firmware Image Validation Failed!____ > > Tried downgrading bios: > > Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : > > Validating BIOS Image... > > BIOS Header Signature is Valid > > BIOS Image has a Valid Checksum. > > BIOS PCI Structure Signature Valid. > > BIOS Image Compatible with the SAS Controller. > > Attempting to Flash BIOS Image... > > BIOS Version in flash : 07.39.00.00 > BIOS Version from File : 07.35.00.00 > Skipping flash since file version is not greater > than existing. > > Flash BIOS Image Failed!____ > > I've tried sas2flash from P20 and P18. > > Can someone fill me in on the trick to downgrade? ____ > > -Chip____ > > __ __ > > __ __ > > On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan > wrote:____ > > Thank you for your help Aaron! I downgraded firmware to P18 and > there are no more errors today.____ > > It seems, that there is something bad with P20 firmware!____ > > ____ > > You saved me a lot of time J____ > > ____ > > Filip____ > > ____ > > ____ > > *From:*Aaron Curry [mailto:asc1111 at gmail.com > ] > *Sent:* Tuesday, November 25, 2014 5:46 PM > *To:* Filip Marvan > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com > > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____ > > ____ > > I had this exact same problem recently when setting up a new home > server... same controller, same firmware (P20). The errors were on > all disks attached to the controller, but only on high read > activity. Writes did not generate the errors. I downgraded the > firmware to P18 (which is what we are using at work) and the errors > went away. Has anyone had success with this controller and the P20 > firmware?____ > > ____ > > Aaron____ > > ____ > > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan > wrote:____ > > Hi Dan, > > thanks for reply. > Yes, errors are on all 5 disks in the same RAIDZ pool. No > problem on other > disks in different pool but on the same SAS cable. > So maybe I only had bad luck with that disks. > > Filip > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss____ > > ____ > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss____ > > __ __ > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0xF9ECC6A5.asc Type: application/pgp-keys Size: 2686 bytes Desc: not available URL: From danmcd at omniti.com Tue Dec 9 17:10:50 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 9 Dec 2014 12:10:50 -0500 Subject: [OmniOS-discuss] Please UPDATE now Message-ID: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com> Hello OmniOS users! Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well. This bug had allowed an ordinary user in the global zone to kernel-panic the machine. That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions: - bloody - r151012 (aka. Stable) - r151010 (aka. previous Stable) - r151006 (aka. Long-Term Support) If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot. I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet. Thank you! Dan McDonald -- OmniOS Engineering From henson at acm.org Tue Dec 9 17:47:24 2014 From: henson at acm.org (Paul B. Henson) Date: Tue, 9 Dec 2014 09:47:24 -0800 Subject: [OmniOS-discuss] Please UPDATE now In-Reply-To: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com> References: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com> Message-ID: Is this only an issue if a malicious user intentionally crashes the system, or could it also potentially occur under regular use? IE, if you have a system with no local users only providing network services, would this still be a critical patch or could it wait for a more convenient installation schedule? The bug report isn't particularly detailed, it's not clear when/why devzvol_readdir() would call strchr or what would cause that call to return NULL. Thanks... > On Dec 9, 2014, at 9:10 AM, Dan McDonald wrote: > > Hello OmniOS users! > > Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well. This bug had allowed an ordinary user in the global zone to kernel-panic the machine. That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions: > > - bloody > - r151012 (aka. Stable) > - r151010 (aka. previous Stable) > - r151006 (aka. Long-Term Support) > > If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot. I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet. > > Thank you! > Dan McDonald -- OmniOS Engineering > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Dec 9 18:37:21 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 9 Dec 2014 13:37:21 -0500 Subject: [OmniOS-discuss] Please UPDATE now In-Reply-To: References: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com> Message-ID: Only specifically buggy code or malicious use will cause the panic. The deployment as you describe (esp. No local users) has reduced the risk enough where you can likely wait for your window. Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 9, 2014, at 12:47 PM, Paul B. Henson wrote: > > Is this only an issue if a malicious user intentionally crashes the system, or could it also potentially occur under regular use? IE, if you have a system with no local users only providing network services, would this still be a critical patch or could it wait for a more convenient installation schedule? The bug report isn't particularly detailed, it's not clear when/why devzvol_readdir() would call strchr or what would cause that call to return NULL. > > Thanks... > >> On Dec 9, 2014, at 9:10 AM, Dan McDonald wrote: >> >> Hello OmniOS users! >> >> Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well. This bug had allowed an ordinary user in the global zone to kernel-panic the machine. That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions: >> >> - bloody >> - r151012 (aka. Stable) >> - r151010 (aka. previous Stable) >> - r151006 (aka. Long-Term Support) >> >> If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot. I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet. >> >> Thank you! >> Dan McDonald -- OmniOS Engineering >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From brogyi at gmail.com Tue Dec 9 23:03:44 2014 From: brogyi at gmail.com (=?windows-1252?Q?Brogy=E1nyi_J=F3zsef?=) Date: Wed, 10 Dec 2014 00:03:44 +0100 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <54871A67.3040805@gmx.li> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> <54871A67.3040805@gmx.li> Message-ID: <54877FD0.7060605@gmail.com> Here is my short description about UEFI flashing. Please modify your version. download 9211_8i_Package_P18_IR_IT_Firmware_BIOS_for_MSDOS_Windows 9211_8i_Package_P18_IR_IT_Firmware_BIOS_for_MSDOS_Windows\Firmware\HBA_9211_8i_IT\2118it.bin 9211_8i_Package_P20_IR_IT_Firmware_BIOS_for_MSDOS_Windows\sasbios_rel\mptsas2.rom download Installer_P18_for_UEFI Installer_P18_for_UEFI\sas2flash_efi_ebc_rel\sas2flash.efi Shell.efi/shellx64.efi ### Both name are working. https://svn.code.sf.net/p/edk2/code/trunk/edk2/EdkShellBinPkg/FullShell/X64/Shell_Full.efi ### https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/Shell.efi ### The first link what I worked for me. The second is working with some Asus mb. Format stick MBR and FAT, and there?s no need to make a bootable disk since we?ll be using EFI. map ### Not need if you know it. mount fs2 ### My stick name. fs2: ls cd 9211 ### It was my directory name. sa2flash.efi -o -e 6 ### Do not restart or switch it off after this step!!! sa2flash.efi -o -f 2118it.bin -b mptsas2.rom sa2flash.efi -listall ### For check. sa2flash.efi -list ### For check. From tobi at oetiker.ch Wed Dec 10 00:41:54 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 10 Dec 2014 01:41:54 +0100 (CET) Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade Message-ID: So tonight, we finally took the plunge and upgraded our zfs/kvm server to r151012 ... the results were terrible. The kvm booted very slowly and all networking felt really slow ... so I did a little test: ubutu-14.04-guest$ dd if=/dev/zero bs=1M count=20 | ssh omnios-r151012-host dd of=/dev/null 20971520 bytes (21 MB) copied, 6.27333 s, 3.3 MB/s ubutu-14.04-guest$ ssh omnios-r151012-host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null 20971520 bytes transferred in 8.010208 secs (2618099 bytes/sec) These numbers were obtained using virtio net drivers but switching to e1000 did not significantly change things. So we booted back into r151010 again ... the difference was immediately apparent ... but there are also number to back this up. ubutu-14.04-guest$ dd if=/dev/zero bs=1M count=20 | ssh omnios-r151010-host dd of=/dev/null 20971520 bytes (21 MB) copied, 0.812479 s, 25.8 MB/s ubutu-14.04-guest$ ssh omnios-r151010-host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null 20971520 bytes (21 MB) copied, 0.545423 s, 38.5 MB/s as you can see the difference in guest network performance is roughly one magnitude ... I have not tested disk performance explicitly, but even booting a windows host took ages ... so I suspect whatever is causing this influences all kvm guest IO. cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From danmcd at omniti.com Wed Dec 10 00:59:47 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 9 Dec 2014 19:59:47 -0500 Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade In-Reply-To: References: Message-ID: > > I have not tested disk performance explicitly, but even booting a > windows host took ages ... so I suspect whatever is causing this > influences all kvm guest IO. What's really REALLY weird about this is that we did not alter anything about how we built KVM between these releases. Tell me, can you run "lockstat sleep " in the global zone while you run your KVM tests? They will produce a lot of output, but they may be very informative about what's going on. Also, I'd be curious if you might (BEs and rpool space being available) upgrade a BE to bloody and repeat your tests? We don't have the facilities to stress out VMs like this, which is why we didn't notice this before 012 went out the door. Clearly something's messing up KVM performance (you're not the first to report this, but you seem to have a decent environment for comparisons). Before the next stable (and incidentally long-term-support as well) release, I hope to have these problems cleared up. One thing that should happen soon is that Joyent is upstreaming the VND changes into illumos-gate, which will allow us to be fully caught up to their illumos-kvm-cmd source, which we've frozen at revision 1c6181be55d1cadc4426069960688307a6083131 since r151010. Thanks, and I wish I could be of more immediate assistance! Dan From tobi at oetiker.ch Wed Dec 10 05:14:56 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 10 Dec 2014 06:14:56 +0100 (CET) Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade In-Reply-To: References: Message-ID: Yesterday Dan McDonald wrote: > > > > I have not tested disk performance explicitly, but even booting a > > windows host took ages ... so I suspect whatever is causing this > > influences all kvm guest IO. > > What's really REALLY weird about this is that we did not alter > anything about how we built KVM between these releases. > > Tell me, can you run "lockstat sleep " in the > global zone while you run your KVM tests? They will produce a > lot of output, but they may be very informative about what's > going on. > Unfortunately I don't have a spare 'big' iron box to play with, but we should be able to do some downtime tonight to run that lockstat sleep experiment and also to do some simple disk io test (with dd). > Also, I'd be curious if you might (BEs and rpool space being > available) upgrade a BE to bloody and repeat your tests? > We don't have the facilities to stress out VMs like this, which > is why we didn't notice this before 012 went out the door. > Clearly something's messing up KVM performance (you're not the > first to report this, but you seem to have a decent environment > for comparisons). Before the next stable (and incidentally > long-term-support as well) release, I hope to have these problems > cleared up. One thing that should happen soon is that Joyent is > upstreaming the VND changes into illumos-gate, which will allow > us to be fully caught up to their illumos-kvm-cmd source, which > we've frozen at revision 1c6181be55d1cadc4426069960688307a6083131 > since r151010. I know that there was no kvm change ... so this must be some side effect of another modificiation ... What seems odd, is that only 2 (or 3) few people reported this problem on the list. After all, it's not something that was difficult to notice. After the upgrade the kvm guests really are almost un-usable for interactive work involing network or disk IO, especially when compared to before. This leads me to suspect, that either only very few people are using omnios as a kvm server OR it is also a hardware dependent problem. I was also wondering if we should try to boot the current smaros on the box just to see what it does to kvm perf. But as I said, it is a production machine, so it is all a bit tricky. cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From stephan.budach at JVM.DE Wed Dec 10 09:16:31 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 10:16:31 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation Message-ID: <54880F6F.2010904@jvm.de> Hi all, I have run into a situation, where ipadm hangs upon trying to create a new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't return and either got killed eventually by the system or by kill -9, which I issued against that process. I just tried to create another if on the on-board NIC igb1, which still works without issues. What could cause ipadm to hang this way and is there a way to get ipadm working again on the ixgbe adaptors, preferreably without having to boot the OmniOS server, since it is serving a NFS export from the remaining functioning ixgbe3 port to my Oracle VM servers. Thanks, budy From gate03 at landcroft.co.uk Wed Dec 10 10:15:40 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 10 Dec 2014 20:15:40 +1000 Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade In-Reply-To: References: Message-ID: <20141210201540.6fe5cfee@emeritus> On Wed, 10 Dec 2014 06:14:56 +0100 (CET) Tobias Oetiker wrote: > This leads me to suspect, that either only very few people are > using omnios as a kvm server OR it is also a hardware dependent > problem. I think it must be. I'm running KVM (Gentoo Linux guests) and have just gone from 151010 to 151012. I haven't carried-out any quantitative assessment, but didn't notice any slowdown. For the record, my KVM invocation is: /usr/bin/qemu-system-x86_64 \ -name "Gentoo "$WHAT \ -cpu host \ -boot order=d \ -enable-kvm \ -vnc cortex:$VNC \ -smp 1,maxcpus=1,cores=2 \ -m 1024 \ -no-hpet \ -localtime \ -kernel /gentoo/kernel-source/linux-3.17.4-gentoo-vbox/arch/x86/boot/bzImage \ -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \ -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-${WHAT},cache=none,if=virtio,index=0 \ -drive file=/dev/zvol/dsk/rpool/vol/Linuswap-${WHAT},cache=none,if=virtio,index=1 \ -net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \ -net vnic,vlan=0,name=net0,ifname=$VNIC,macaddr=$mac \ -monitor telnet:127.0.0.1:${monitor},server,nowait \ -vga std \ -daemonize where ${WHAT} is either KDE or XFCE. Machine is a Supermicro 5017C-LF with 1 x Intel Xeon E3-1240V2 3.40 GHz.4 Cores , 4 Threads,8Mb cache and 8 GiB RAM. If you are interested in any performance figures, let me know any tar or dd etc. commands you'd like me to run. Michael. From tobi at oetiker.ch Wed Dec 10 10:58:27 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 10 Dec 2014 11:58:27 +0100 (CET) Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade In-Reply-To: <20141210201540.6fe5cfee@emeritus> References: <20141210201540.6fe5cfee@emeritus> Message-ID: Hi Michael, Today Michael Mounteney wrote: > On Wed, 10 Dec 2014 06:14:56 +0100 (CET) > Tobias Oetiker wrote: > > > This leads me to suspect, that either only very few people are > > using omnios as a kvm server OR it is also a hardware dependent > > problem. > > I think it must be. I'm running KVM (Gentoo Linux guests) and have > just gone from 151010 to 151012. I haven't carried-out any > quantitative assessment, but didn't notice any slowdown. For the > record, my KVM invocation is: > > /usr/bin/qemu-system-x86_64 \ > -name "Gentoo "$WHAT \ > -cpu host \ > -boot order=d \ > -enable-kvm \ > -vnc cortex:$VNC \ > -smp 1,maxcpus=1,cores=2 \ > -m 1024 \ > -no-hpet \ > -localtime \ > -kernel /gentoo/kernel-source/linux-3.17.4-gentoo-vbox/arch/x86/boot/bzImage \ > -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \ > -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-${WHAT},cache=none,if=virtio,index=0 \ > -drive file=/dev/zvol/dsk/rpool/vol/Linuswap-${WHAT},cache=none,if=virtio,index=1 \ > -net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \ > -net vnic,vlan=0,name=net0,ifname=$VNIC,macaddr=$mac \ > -monitor telnet:127.0.0.1:${monitor},server,nowait \ > -vga std \ > -daemonize > > where ${WHAT} is either KDE or XFCE. Machine is a Supermicro 5017C-LF with 1 x Intel Xeon E3-1240V2 3.40 GHz.4 Cores , 4 Threads,8Mb cache and 8 GiB RAM. > > If you are interested in any performance figures, let me know any tar or dd etc. commands you'd like me to run. yes, a very simple test: guest$ dd if=/dev/zero bs=1M count=20 | ssh host dd of=/dev/null guest$ ssh host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null and since I suspect that the disk io suffers too but due to caching, maybe reading just a bit off the disk device might be an interesting test: dd if=/dev/vda of=/dev/zero bs=1M count=1000 cheers tobi > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From chip at innovates.com Wed Dec 10 15:37:28 2014 From: chip at innovates.com (Schweiss, Chip) Date: Wed, 10 Dec 2014 09:37:28 -0600 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <54871A67.3040805@gmx.li> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> <54871A67.3040805@gmx.li> Message-ID: On Tue, Dec 9, 2014 at 9:51 AM, Dominik Hassler wrote: > Chip, > > when I had to reflash an onboard LSI2308 I used the UEFI shell. No > fiddling around with boot disks etc. > > Not sure, if it is applicable to your problem. > That was the trick. Under DOS I kept getting "ERROR: Failed to initialize PAL." Turns out this is a problem on X9 and newer Supermicro boards. With the efi sas2flash version on a USB stick it was quite easy. The system BIOS has a boot override to drop into the UEFI shell from there I could see the USB stick and run the erase and flash. Thanks for everyone's input on this. Hopefully I will only be upgrading in the future and can stick to the Solaris version of sas2flash. -Chip > On 12/09/2014 03:49 PM, Schweiss, Chip wrote: > > I'm still fighting this. > > > > Under OmniOS the erase function is disabled in sas2flash for solaris. > > > > I couldn't seem to find an iso bootable DOS that I could stage files > > in. FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode. > > Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation > > Failed!" Tried both P18 sas2flash and P20 sas2flash. > > > > What OS environment are you running this in? > > > > > > > > On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan > > wrote: > > > > Hi,____ > > > > __ __ > > > > first you have to erase P20 firmware with:____ > > > > __ __ > > > > sas2flsh -o -e 6____ > > > > __ __ > > > > now do not reboot(!) and flash P18 with____ > > > > __ __ > > > > sas2flsh -f XXXXXX.bin -b mptsas2.rom____ > > > > __ __ > > > > Filip____ > > > > __ __ > > > > __ __ > > > > *From:*OmniOS-discuss > > [mailto:omnios-discuss-bounces at lists.omniti.com > > ] *On Behalf Of > > *Schweiss, Chip > > *Sent:* Monday, December 08, 2014 08:09 AM > > *To:* Filip Marvan > > *Cc:* omnios-discuss at lists.omniti.com > > > > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____ > > > > __ __ > > > > I've got some new LSI HBAs I'm trying to downgrade from firmware > > version 20 to 18.____ > > > > I'm getting errors when trying to downgrade: > > > > Attempting to flash firmware to LSI SAS SAS2308_2(D1) : > > > > Executing Operation: Flash Firmware Image > > > > Firmware Image has a Valid Checksum. > > Firmware Version 18.00.00.00 > > Firmware Image compatible with Controller. > > > > Valid NVDATA Image found. > > NVDATA Version 11.00.00.00 > > Checking for a compatible NVData image... > > > > NVDATA Device ID and Chip Revision match verified. > > ERROR: Cannot downgrade NVDATA version 14.00.00.06 > > to 11.00.110000.00. > > > > ERROR: Failed to get valid NVDATA image from File! > > > > Firmware Image Validation Failed!____ > > > > Tried downgrading bios: > > > > Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : > > > > Validating BIOS Image... > > > > BIOS Header Signature is Valid > > > > BIOS Image has a Valid Checksum. > > > > BIOS PCI Structure Signature Valid. > > > > BIOS Image Compatible with the SAS Controller. > > > > Attempting to Flash BIOS Image... > > > > BIOS Version in flash : 07.39.00.00 > > BIOS Version from File : 07.35.00.00 > > Skipping flash since file version is not greater > > than existing. > > > > Flash BIOS Image Failed!____ > > > > I've tried sas2flash from P20 and P18. > > > > Can someone fill me in on the trick to downgrade? ____ > > > > -Chip____ > > > > __ __ > > > > __ __ > > > > On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan > > wrote:____ > > > > Thank you for your help Aaron! I downgraded firmware to P18 and > > there are no more errors today.____ > > > > It seems, that there is something bad with P20 firmware!____ > > > > ____ > > > > You saved me a lot of time J____ > > > > ____ > > > > Filip____ > > > > ____ > > > > ____ > > > > *From:*Aaron Curry [mailto:asc1111 at gmail.com > > ] > > *Sent:* Tuesday, November 25, 2014 5:46 PM > > *To:* Filip Marvan > > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com > > > > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____ > > > > ____ > > > > I had this exact same problem recently when setting up a new home > > server... same controller, same firmware (P20). The errors were on > > all disks attached to the controller, but only on high read > > activity. Writes did not generate the errors. I downgraded the > > firmware to P18 (which is what we are using at work) and the errors > > went away. Has anyone had success with this controller and the P20 > > firmware?____ > > > > ____ > > > > Aaron____ > > > > ____ > > > > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan > > wrote:____ > > > > Hi Dan, > > > > thanks for reply. > > Yes, errors are on all 5 disks in the same RAIDZ pool. No > > problem on other > > disks in different pool but on the same SAS cable. > > So maybe I only had bad luck with that disks. > > > > Filip > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > > > http://lists.omniti.com/mailman/listinfo/omnios-discuss____ > > > > ____ > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com OmniOS-discuss at lists.omniti.com> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss____ > > > > __ __ > > > > > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Dec 10 15:53:21 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 10:53:21 -0500 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <54880F6F.2010904@jvm.de> References: <54880F6F.2010904@jvm.de> Message-ID: <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> > On Dec 10, 2014, at 4:16 AM, Stephan Budach wrote: > > Hi all, > > I have run into a situation, where ipadm hangs upon trying to create a new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't return and either got killed eventually by the system or by kill -9, which I issued against that process. I just tried to create another if on the on-board NIC igb1, which still works without issues. What does "pstack `pgrep ipadm`" say when they hang? And what's your exact incantation? And "killed by the system", what exactly do you mean? Did ipadm(1M) time out? > What could cause ipadm to hang this way and is there a way to get ipadm working again on the ixgbe adaptors, preferreably without having to boot the OmniOS server, since it is serving a NFS export from the remaining functioning ixgbe3 port to my Oracle VM servers. I'll need more information before I can help you. Dan From stephan.budach at JVM.DE Wed Dec 10 16:12:10 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 17:12:10 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> Message-ID: <548870DA.6080308@jvm.de> Hi Dan, I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running? If not, please correct me on that. What is really funny, is that running pstack against the ipadm process, somehow brought things back in line: root at nfsvmpool01:~# pkg info entire Name: entire Summary: Incorporation to constrain core system packages to same build Description: This package constrains core system package versions to the same build and provides a minimal set of additional packages. State: Installed Publisher: omnios Version: 11 Build Release: 5.11 Branch: 0.151006 Packaging Date: Mon Oct 27 19:15:00 2014 Size: 0.00 B FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested: root at nfsvmpool01:~# pstack `pgrep ipadm` 12397: ipadm create-if ixgbe3 fef07723 open (8047190, 2, fec9404c) feef24f8 open (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105 fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149 fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3 0805553e do_create_if (2) + 8f 08052e72 main (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df 080525c7 _start (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83 ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors. Cheers, budy Am 10.12.14 um 16:53 schrieb Dan McDonald: >> On Dec 10, 2014, at 4:16 AM, Stephan Budach wrote: >> >> Hi all, >> >> I have run into a situation, where ipadm hangs upon trying to create a new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't return and either got killed eventually by the system or by kill -9, which I issued against that process. I just tried to create another if on the on-board NIC igb1, which still works without issues. > What does "pstack `pgrep ipadm`" say when they hang? And what's your exact incantation? And "killed by the system", what exactly do you mean? Did ipadm(1M) time out? > >> What could cause ipadm to hang this way and is there a way to get ipadm working again on the ixgbe adaptors, preferreably without having to boot the OmniOS server, since it is serving a NFS export from the remaining functioning ixgbe3 port to my Oracle VM servers. > I'll need more information before I can help you. > > Dan > -- Stephan Budach Deputy Managing Director Jung von Matt/it-services GmbH Glash?ttenstra?e 79 20357 Hamburg Tel: +49 40-4321-1353 Fax: +49 40-4321-1114 E-Mail: stephan.budach at jvm.de Internet: http://www.jvm.com Gesch?ftsf?hrer: Frank Wilhelm, Stephan Budach (stellv.) AG HH HRB 98380 From danmcd at omniti.com Wed Dec 10 16:20:46 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 11:20:46 -0500 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <548870DA.6080308@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> Message-ID: > On Dec 10, 2014, at 11:12 AM, Stephan Budach wrote: > > Hi Dan, > > I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running? I meant what exact command-line arguments you typed for ipadm(1M). But that's okay, because it's good to know the other things you're about to show me. > What is really funny, is that running pstack against the ipadm process, somehow brought things back in line: > > root at nfsvmpool01:~# pkg info entire > Name: entire > Summary: Incorporation to constrain core system packages to same build > Description: This package constrains core system package versions to the same > build and provides a minimal set of additional packages. > State: Installed > Publisher: omnios > Version: 11 > Build Release: 5.11 > Branch: 0.151006 > Packaging Date: Mon Oct 27 19:15:00 2014 > Size: 0.00 B > FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z > > I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested: > > root at nfsvmpool01:~# pstack `pgrep ipadm` > 12397: ipadm create-if ixgbe3 > fef07723 open (8047190, 2, fec9404c) > feef24f8 open (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105 > fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed > fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a > fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149 > fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb > fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a > fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3 > 0805553e do_create_if (2) + 8f > 08052e72 main (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df > 080525c7 _start (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83 > > ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors. That's VERY odd, albeit with pleasant results. I wouldn't think pstack would unblock something like what you showed me. You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now. You mentioned a desire to not reboot your system. Did you plug in these ixgbes while booted (i.e. hotplugging)? Or were they always there, and you just hadn't configured them until now? It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free. I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem. The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur. Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug. Dan From stephan.budach at JVM.DE Wed Dec 10 16:38:20 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 17:38:20 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> Message-ID: <548876FC.9040108@jvm.de> Am 10.12.14 um 17:20 schrieb Dan McDonald: >> On Dec 10, 2014, at 11:12 AM, Stephan Budach wrote: >> >> Hi Dan, >> >> I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running? > I meant what exact command-line arguments you typed for ipadm(1M). But that's okay, because it's good to know the other things you're about to show me. > >> What is really funny, is that running pstack against the ipadm process, somehow brought things back in line: >> >> root at nfsvmpool01:~# pkg info entire >> Name: entire >> Summary: Incorporation to constrain core system packages to same build >> Description: This package constrains core system package versions to the same >> build and provides a minimal set of additional packages. >> State: Installed >> Publisher: omnios >> Version: 11 >> Build Release: 5.11 >> Branch: 0.151006 >> Packaging Date: Mon Oct 27 19:15:00 2014 >> Size: 0.00 B >> FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z >> >> I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested: >> >> root at nfsvmpool01:~# pstack `pgrep ipadm` >> 12397: ipadm create-if ixgbe3 >> fef07723 open (8047190, 2, fec9404c) >> feef24f8 open (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105 >> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed >> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a >> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149 >> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb >> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a >> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3 >> 0805553e do_create_if (2) + 8f >> 08052e72 main (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df >> 080525c7 _start (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83 >> >> ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors. > That's VERY odd, albeit with pleasant results. I wouldn't think pstack would unblock something like what you showed me. You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now. Yes, it is? ;) > > You mentioned a desire to not reboot your system. Did you plug in these ixgbes while booted (i.e. hotplugging)? Or were they always there, and you just hadn't configured them until now? The two cards had been in the system right from the start and I already configured one port (ixgbe0) to connect to a 10GbE port on our 6509, which went without issues. I wanted to resolve this issue without rebooting, since rebooting would have killed 53 VMs that are residing on my NFS export that is running over ixgbe0. > It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free. I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem. The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur. I can maybe help with that as well? I tried to set the MTU to the maximum, that the driver would accept and I ended up with a mtu of 15500 (what a odd number anyway) which I set using dladm on ixgbe3. When i tried to create the interface using ipadm, that was the moment, when things went quirky. Although I trimmed the mtu down to a reasonable 9216 ipadm wouldn't create any longer and would hang at creating interfaces on any of the remaining 3 ports of my T540-X2's. > > Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug. > > Dan > Ha ha, no prob - you already provided a great deal of help! Thanks a lot, budy From ikaufman at eng.ucsd.edu Wed Dec 10 17:01:43 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Wed, 10 Dec 2014 09:01:43 -0800 Subject: [OmniOS-discuss] weird library issue Message-ID: Hi all, I have a situation where a bunch of libs are complaining about missing the following: libc.so.1 (ILLUMOS_0.7) or libc.so.1 (ILLUMOS_0.6) This is happening on one system, but I have others that are fine. A standard libc.so.1 exists, but various libs are looking for the tagged one. Any ideas? Thanks, Ian -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From danmcd at omniti.com Wed Dec 10 18:15:54 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 13:15:54 -0500 Subject: [OmniOS-discuss] weird library issue In-Reply-To: References: Message-ID: > On Dec 10, 2014, at 12:01 PM, Ian Kaufman wrote: > > Hi all, > > I have a situation where a bunch of libs are complaining about missing > the following: > > libc.so.1 (ILLUMOS_0.7) > > or > > libc.so.1 (ILLUMOS_0.6) > > This is happening on one system, but I have others that are fine. A > standard libc.so.1 exists, but various libs are looking for the tagged > one. > > Any ideas? You'll need to provide more data. It's as if something you compiled was compiled against a later version of libc, and then moved to an earlier version of libc. Dan From ikaufman at eng.ucsd.edu Wed Dec 10 18:42:35 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Wed, 10 Dec 2014 10:42:35 -0800 Subject: [OmniOS-discuss] weird library issue In-Reply-To: References: Message-ID: It looks like this system is in some weird state where there are packages from 151006 and 151008. For example, /etc/release shows: root at massive02-b:~# cat /etc/release OmniOS v11 r151008 Copyright 2013 OmniTI Computer Consulting, Inc. All rights reserved. Use is subject to license terms. but, when I try to do a pkg update, I see this (multiple repeating lines): No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151008:20131204T231613Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151008:20131204T231613Z Reason: A version for 'incorporate' dependency on pkg:/library/security/openssl at 1.0.1,5.11-0.151008 cannot be found No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151004:20121024T180535Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151004:20121024T180535Z Reason: A version for 'incorporate' dependency on pkg:/archiver/gnu-tar at 1.26,5.11-0.151004 cannot be found No suitable version of required package pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151006:20130506T214442Z found: Reject: pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151006:20130506T214442Z Reason: A version for 'incorporate' dependency on pkg:/archiver/gnu-tar at 1.26,5.11-0.151006 cannot be found and for openssl, I see this: root at massive02-b:~# pkg info library/security/openssl Name: library/security/openssl Summary: openssl - A toolkit for Secure Sockets Layer (SSL v2/v3) and Transport Layer (TLS v1) protocols and general purpose cryptographic library State: Installed Publisher: omnios Version: 1.0.1.10 (1.0.1j) Build Release: 5.11 Branch: 0.151006 Packaging Date: October 15, 2014 03:40:22 PM Size: 18.08 MB FMRI: pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z root at massive02-b:~# pkg install pkg:/library/security/openssl at 1.0.1,5.11-0.151008 Creating Plan / pkg install: No matching version of library/security/openssl can be installed: Reject: pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z Reason: Newer version pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z is already installed Ian On Wed, Dec 10, 2014 at 10:15 AM, Dan McDonald wrote: > >> On Dec 10, 2014, at 12:01 PM, Ian Kaufman wrote: >> >> Hi all, >> >> I have a situation where a bunch of libs are complaining about missing >> the following: >> >> libc.so.1 (ILLUMOS_0.7) >> >> or >> >> libc.so.1 (ILLUMOS_0.6) >> >> This is happening on one system, but I have others that are fine. A >> standard libc.so.1 exists, but various libs are looking for the tagged >> one. >> >> Any ideas? > > You'll need to provide more data. It's as if something you compiled was compiled against a later version of libc, and then moved to an earlier version of libc. > > Dan > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From danmcd at omniti.com Wed Dec 10 18:55:00 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 13:55:00 -0500 Subject: [OmniOS-discuss] weird library issue In-Reply-To: References: Message-ID: > On Dec 10, 2014, at 1:42 PM, Ian Kaufman wrote: > > It looks like this system is in some weird state where there are > packages from 151006 and 151008. Oh my. Yes, you've been bitten by the pkg upgrade problems between 006 and 008. This was before my time here, so I'm not sure how best to extricate yourself from things. If you want, you can jump forward to the current stable release (r151012). If you want to try that, and you don't have zones, do this: 1.) Create a new BE, call it 012 for this example: beadm create 012 2.) Mount the BE on /mnt: beadm mount 012 /mnt 3.) Change the publisher on the new BE to point to r151012's repo: pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/release/ -g http://pkg.omniti.com/omnios/r151012/ omnios 4.) Apply the update to the new BE: pkg -R /mnt update 5.) Unmount the BE: beadm umount 012 6.) Either reboot and use the GRUB menu to try out the new 012 BE, or "beadm activate 012" to make it go there on the next reboot. Dan From stephan.budach at JVM.DE Wed Dec 10 19:07:47 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 20:07:47 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <548876FC.9040108@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> Message-ID: <54889A03.7080307@jvm.de> Am 10.12.14 um 17:38 schrieb Stephan Budach: > Am 10.12.14 um 17:20 schrieb Dan McDonald: >>> On Dec 10, 2014, at 11:12 AM, Stephan Budach >>> wrote: >>> >>> Hi Dan, >>> >>> I actually don't know the term "incantation" yet, but I assume, that >>> you wanted to know the release of OmniOS I am running? >> I meant what exact command-line arguments you typed for ipadm(1M). >> But that's okay, because it's good to know the other things you're >> about to show me. >> >>> What is really funny, is that running pstack against the ipadm >>> process, somehow brought things back in line: >>> >>> root at nfsvmpool01:~# pkg info entire >>> Name: entire >>> Summary: Incorporation to constrain core system packages to >>> same build >>> Description: This package constrains core system package versions >>> to the same >>> build and provides a minimal set of additional >>> packages. >>> State: Installed >>> Publisher: omnios >>> Version: 11 >>> Build Release: 5.11 >>> Branch: 0.151006 >>> Packaging Date: Mon Oct 27 19:15:00 2014 >>> Size: 0.00 B >>> FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z >>> >>> I was running ipadm create-if ixgbe3 in another session when I ran >>> pstack as suggested: >>> >>> root at nfsvmpool01:~# pstack `pgrep ipadm` >>> 12397: ipadm create-if ixgbe3 >>> fef07723 open (8047190, 2, fec9404c) >>> feef24f8 open (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105 >>> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed >>> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a >>> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149 >>> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb >>> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a >>> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3 >>> 0805553e do_create_if (2) + 8f >>> 08052e72 main (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) >>> + df >>> 080525c7 _start (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83 >>> >>> ipadm returned and created the interface, just as if nothing ever >>> happened. Afterwards I was able to create and delete interfaces >>> without any issue on the ixgbe adaptors. >> That's VERY odd, albeit with pleasant results. I wouldn't think >> pstack would unblock something like what you showed me. You're >> running 006 (long-term support), but I'm not seeing any major changes >> in the plumbing of interfaces between then and now. > Yes, it is? ;) >> >> You mentioned a desire to not reboot your system. Did you plug in >> these ixgbes while booted (i.e. hotplugging)? Or were they always >> there, and you just hadn't configured them until now? > The two cards had been in the system right from the start and I > already configured one port (ixgbe0) to connect to a 10GbE port on our > 6509, which went without issues. I wanted to resolve this issue > without rebooting, since rebooting would have killed 53 VMs that are > residing on my NFS export that is running over ixgbe0. >> It seems like you tickled a race condition or some other odd >> combination of circumstances that inspecting the ipadm(1M) process >> jostled free. I don't have any ixgbe handy at the moment, but had I, >> I'd like to know if this is a reproducible problem. The only theory >> I can think of is that something weird happened while ipadm was >> communicating with ipmgmtd, and inspecting the process with pstack >> allowed a context switch to occur. > I can maybe help with that as well? I tried to set the MTU to the > maximum, that the driver would accept and I ended up with a mtu of > 15500 (what a odd number anyway) which I set using dladm on ixgbe3. > When i tried to create the interface using ipadm, that was the moment, > when things went quirky. Although I trimmed the mtu down to a > reasonable 9216 ipadm wouldn't create any longer and would hang at > creating interfaces on any of the remaining 3 ports of my T540-X2's. >> >> Thank you, and while it's nice to see things working, I'm sorry I >> don't have a better explanation about what happened, or the ability >> to immediately reproduce this bug. >> >> Dan >> > Ha ha, no prob - you already provided a great deal of help! > > Thanks a lot, > budy Ahh? here we go again? root at nfsvmpool01:~# dladm show-linkprop -p mtu LINK PROPERTY PERM VALUE DEFAULT POSSIBLE igb0 mtu rw 1500 1500 60-9000 igb1 mtu rw 1500 1500 60-9000 igb2 mtu rw 1500 1500 60-9000 igb3 mtu rw 1500 1500 60-9000 ixgbe2 mtu rw 1500 1500 1500-15500 ixgbe0 mtu rw 1500 1500 1500-15500 root at nfsvmpool01:~# pstack `pgrep dladm` 23948: dladm show-linkprop -p mtu What I did was to remove the ixgbe3 interface and try to set the mtu to 9216 like this: dladm set-linkprop -p mtu=9216 ixgbe3 I seem only to be able to kill dladm by killing it using kill -9 Cheers, budy From danmcd at omniti.com Wed Dec 10 19:12:15 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 14:12:15 -0500 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <54889A03.7080307@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> Message-ID: <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> You said it was ipadm earlier, not dladm. > On Dec 10, 2014, at 2:07 PM, Stephan Budach wrote: > Ahh? here we go again? > > root at nfsvmpool01:~# dladm show-linkprop -p mtu > LINK PROPERTY PERM VALUE DEFAULT POSSIBLE > igb0 mtu rw 1500 1500 60-9000 > igb1 mtu rw 1500 1500 60-9000 > igb2 mtu rw 1500 1500 60-9000 > igb3 mtu rw 1500 1500 60-9000 > ixgbe2 mtu rw 1500 1500 1500-15500 > ixgbe0 mtu rw 1500 1500 1500-15500 > > > root at nfsvmpool01:~# pstack `pgrep dladm` > 23948: dladm show-linkprop -p mtu > > What I did was to remove the ixgbe3 interface and try to set the mtu to 9216 like this: How did you remove the ixgbe3 interface? With delete-phys? > dladm set-linkprop -p mtu=9216 ixgbe3 > > I seem only to be able to kill dladm by killing it using kill -9 It may be hanging because you removed it. Generally speaking, you shouldn't remove hardware entities (vs. created entities like vnics, aggrs, etc.) with dladm. They are harmless lying in the background, if you don't have them configured up with ifconfig or ipadm, you won't be using them. Dan From stephan.budach at JVM.DE Wed Dec 10 19:23:32 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 20:23:32 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> Message-ID: <54889DB4.40803@jvm.de> Am 10.12.14 um 20:12 schrieb Dan McDonald: > You said it was ipadm earlier, not dladm. Well, yes it was. I am sure that ipadm will now hang as well. >> On Dec 10, 2014, at 2:07 PM, Stephan Budach wrote: >> Ahh? here we go again? >> >> root at nfsvmpool01:~# dladm show-linkprop -p mtu >> LINK PROPERTY PERM VALUE DEFAULT POSSIBLE >> igb0 mtu rw 1500 1500 60-9000 >> igb1 mtu rw 1500 1500 60-9000 >> igb2 mtu rw 1500 1500 60-9000 >> igb3 mtu rw 1500 1500 60-9000 >> ixgbe2 mtu rw 1500 1500 1500-15500 >> ixgbe0 mtu rw 1500 1500 1500-15500 >> >> >> root at nfsvmpool01:~# pstack `pgrep dladm` >> 23948: dladm show-linkprop -p mtu >> >> What I did was to remove the ixgbe3 interface and try to set the mtu to 9216 like this: > How did you remove the ixgbe3 interface? With delete-phys? > >> dladm set-linkprop -p mtu=9216 ixgbe3 >> >> I seem only to be able to kill dladm by killing it using kill -9 > It may be hanging because you removed it. > > Generally speaking, you shouldn't remove hardware entities (vs. created entities like vnics, aggrs, etc.) with dladm. They are harmless lying in the background, if you don't have them configured up with ifconfig or ipadm, you won't be using them. > > Dan > Maybe I have told things not clearly enough. I have created the interface like this: ipadm create-if ixgbe3 ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static This worked, but I wanted to change the MTU, so I deciced to remove this config again: ipadm delete-addr ixgb3/v4static ipadm delete-if ixgbe3 Then I tried to set the MTU of ixgbe3 like this: dladm set-linkprop -p mtu=9216 ixgbe3 I can't remember, if that already stalled, however, now dladm show-linkprop hangs and it does need to get killed -9. Cheers, budy From danmcd at omniti.com Wed Dec 10 19:28:37 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 14:28:37 -0500 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <54889DB4.40803@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> <54889DB4.40803@jvm.de> Message-ID: <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> > On Dec 10, 2014, at 2:23 PM, Stephan Budach wrote: > > This worked, but I wanted to change the MTU, so I deciced to remove this config again: > > ipadm delete-addr ixgb3/v4static > ipadm delete-if ixgbe3 Interesting. Your "dladm show-linkprop" output didn't have anything for ixgbe3. I wonder if that happened because of the delete-if for ipadm? > Then I tried to set the MTU of ixgbe3 like this: > dladm set-linkprop -p mtu=9216 ixgbe3 > > I can't remember, if that already stalled, however, now dladm show-linkprop hangs and it does need to get killed -9. It's probably not able to find ixgbe3, and hanging. The question is how did ixgbe3 disappear from the list of links (vs. ip addrs)? Dan From stephan.budach at JVM.DE Wed Dec 10 19:56:08 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 20:56:08 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> <54889DB4.40803@jvm.de> <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> Message-ID: <5488A558.5010401@jvm.de> Am 10.12.14 um 20:28 schrieb Dan McDonald: >> On Dec 10, 2014, at 2:23 PM, Stephan Budach wrote: >> >> This worked, but I wanted to change the MTU, so I deciced to remove this config again: >> >> ipadm delete-addr ixgb3/v4static >> ipadm delete-if ixgbe3 > Interesting. Your "dladm show-linkprop" output didn't have anything for ixgbe3. I wonder if that happened because of the delete-if for ipadm? > >> Then I tried to set the MTU of ixgbe3 like this: >> dladm set-linkprop -p mtu=9216 ixgbe3 >> >> I can't remember, if that already stalled, however, now dladm show-linkprop hangs and it does need to get killed -9. > It's probably not able to find ixgbe3, and hanging. The question is how did ixgbe3 disappear from the list of links (vs. ip addrs)? > I don't know? it seems to have serious issues with the MTU specifically? root at nfsvmpool01:~# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE igb0 Ethernet up 1000 full igb0 igb1 Ethernet unknown 1000 full igb1 igb2 Ethernet unknown 0 half igb2 igb3 Ethernet unknown 0 half igb3 ixgbe2 Ethernet down 0 unknown ixgbe2 ixgbe0 Ethernet up 10000 full ixgbe0 ixgbe3 Ethernet up 10000 full ixgbe3 ixgbe1 Ethernet down 0 unknown ixgbe1 root at nfsvmpool01:~# dladm show-linkprop -p mtu LINK PROPERTY PERM VALUE DEFAULT POSSIBLE igb0 mtu rw 1500 1500 60-9000 igb1 mtu rw 1500 1500 60-9000 igb2 mtu rw 1500 1500 60-9000 igb3 mtu rw 1500 1500 60-9000 ixgbe2 mtu rw 1500 1500 1500-15500 ixgbe0 mtu rw 1500 1500 1500-15500 ^C So, dladm shows the phys status, but it isn't able to tell me about the MTU of ixgbe3?? Needless to mention that ipadm also doesn't work. I will try to reset the MTU back to 1500 again and seem if I can get the interface back. Cheers, budy From stephan.budach at JVM.DE Wed Dec 10 20:14:59 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 21:14:59 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <5488A558.5010401@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> <54889DB4.40803@jvm.de> <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> <5488A558.5010401@jvm.de> Message-ID: <5488A9C3.3080903@jvm.de> Am 10.12.14 um 20:56 schrieb Stephan Budach: > I don't know? it seems to have serious issues with the MTU specifically? > > root at nfsvmpool01:~# dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > igb0 Ethernet up 1000 full igb0 > igb1 Ethernet unknown 1000 full igb1 > igb2 Ethernet unknown 0 half igb2 > igb3 Ethernet unknown 0 half igb3 > ixgbe2 Ethernet down 0 unknown ixgbe2 > ixgbe0 Ethernet up 10000 full ixgbe0 > ixgbe3 Ethernet up 10000 full ixgbe3 > ixgbe1 Ethernet down 0 unknown ixgbe1 > > root at nfsvmpool01:~# dladm show-linkprop -p mtu > LINK PROPERTY PERM VALUE DEFAULT POSSIBLE > igb0 mtu rw 1500 1500 60-9000 > igb1 mtu rw 1500 1500 60-9000 > igb2 mtu rw 1500 1500 60-9000 > igb3 mtu rw 1500 1500 60-9000 > ixgbe2 mtu rw 1500 1500 1500-15500 > ixgbe0 mtu rw 1500 1500 1500-15500 > ^C > > So, dladm shows the phys status, but it isn't able to tell me about > the MTU of ixgbe3?? > > Needless to mention that ipadm also doesn't work. I will try to reset > the MTU back to 1500 again and seem if I can get the interface back. Now, this one? I don't get it. ;) root at nfsvmpool01:~# ipadm create-if ixgbe3 root at nfsvmpool01:~# ipadm show-if IFNAME STATE CURRENT PERSISTENT lo0 ok -m-v------46 --- igb0 ok bm--------46 -46 ixgbe0 ok bm--------46 -46 ixgbe3 down bm--------46 -46 root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static root at nfsvmpool01:~# ping 192.168.1.2 192.168.1.2 is alive root at nfsvmpool01:~# dladm show-linkprop -p mtu LINK PROPERTY PERM VALUE DEFAULT POSSIBLE igb0 mtu rw 1500 1500 60-9000 igb1 mtu rw 1500 1500 60-9000 igb2 mtu rw 1500 1500 60-9000 igb3 mtu rw 1500 1500 60-9000 ixgbe2 mtu rw 1500 1500 1500-15500 ixgbe0 mtu rw 1500 1500 1500-15500 ixgbe3 mtu rw 9216 1500 1500-15500 ixgbe1 mtu rw 1500 1500 1500-15500 I do have a OmniOS 012 system, where all this works without these issues? ;) Now, I won#t change the mtu that soon again, I guess? Cheers, budy From danmcd at omniti.com Wed Dec 10 20:17:27 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 15:17:27 -0500 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <5488A9C3.3080903@jvm.de> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> <54889DB4.40803@jvm.de> <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> <5488A558.5010401@jvm.de> <5488A9C3.3080903@jvm.de> Message-ID: <360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com> > On Dec 10, 2014, at 3:14 PM, Stephan Budach wrote: > >> > Now, this one? I don't get it. ;) > > root at nfsvmpool01:~# ipadm create-if ixgbe3 > root at nfsvmpool01:~# ipadm show-if > IFNAME STATE CURRENT PERSISTENT > lo0 ok -m-v------46 --- > igb0 ok bm--------46 -46 > ixgbe0 ok bm--------46 -46 > ixgbe3 down bm--------46 -46 > root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static > root at nfsvmpool01:~# ping 192.168.1.2 > 192.168.1.2 is alive > root at nfsvmpool01:~# dladm show-linkprop -p mtu > LINK PROPERTY PERM VALUE DEFAULT POSSIBLE > igb0 mtu rw 1500 1500 60-9000 > igb1 mtu rw 1500 1500 60-9000 > igb2 mtu rw 1500 1500 60-9000 > igb3 mtu rw 1500 1500 60-9000 > ixgbe2 mtu rw 1500 1500 1500-15500 > ixgbe0 mtu rw 1500 1500 1500-15500 > ixgbe3 mtu rw 9216 1500 1500-15500 > ixgbe1 mtu rw 1500 1500 1500-15500 Call me old-fashioned, but I'm used to doing this all with ifconfig(1M) and editing /etc/hostname.X files. I know I should learn this new stuff, but I never have had the wherewithal until now. > I do have a OmniOS 012 system, where all this works without these issues? ;) REALLY? So this fails on 006, but works on 012?! I quickly looked at illumos-gate to see what all got fixed if anything, and I didn't see anything that would've necessarily improved things. > Now, I won#t change the mtu that soon again, I guess? I guess. Sorry I couldn't provide an authoritative answer, Dan From stephan.budach at JVM.DE Wed Dec 10 20:27:11 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Wed, 10 Dec 2014 21:27:11 +0100 Subject: [OmniOS-discuss] ipadm hangs on interface creation In-Reply-To: <360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com> References: <54880F6F.2010904@jvm.de> <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com> <548870DA.6080308@jvm.de> <548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de> <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com> <54889DB4.40803@jvm.de> <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com> <5488A558.5010401@jvm.de> <5488A9C3.3080903@jvm.de> <360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com> Message-ID: <5488AC9F.9000400@jvm.de> Hi Dan, Am 10.12.14 um 21:17 schrieb Dan McDonald: >> On Dec 10, 2014, at 3:14 PM, Stephan Budach wrote: >> >> Now, this one? I don't get it. ;) >> >> root at nfsvmpool01:~# ipadm create-if ixgbe3 >> root at nfsvmpool01:~# ipadm show-if >> IFNAME STATE CURRENT PERSISTENT >> lo0 ok -m-v------46 --- >> igb0 ok bm--------46 -46 >> ixgbe0 ok bm--------46 -46 >> ixgbe3 down bm--------46 -46 >> root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static >> root at nfsvmpool01:~# ping 192.168.1.2 >> 192.168.1.2 is alive >> root at nfsvmpool01:~# dladm show-linkprop -p mtu >> LINK PROPERTY PERM VALUE DEFAULT POSSIBLE >> igb0 mtu rw 1500 1500 60-9000 >> igb1 mtu rw 1500 1500 60-9000 >> igb2 mtu rw 1500 1500 60-9000 >> igb3 mtu rw 1500 1500 60-9000 >> ixgbe2 mtu rw 1500 1500 1500-15500 >> ixgbe0 mtu rw 1500 1500 1500-15500 >> ixgbe3 mtu rw 9216 1500 1500-15500 >> ixgbe1 mtu rw 1500 1500 1500-15500 > Call me old-fashioned, but I'm used to doing this all with ifconfig(1M) and editing /etc/hostname.X files. I know I should learn this new stuff, but I never have had the wherewithal until now. > >> I do have a OmniOS 012 system, where all this works without these issues? ;) > REALLY? So this fails on 006, but works on 012?! I quickly looked at illumos-gate to see what all got fixed if anything, and I didn't see anything that would've necessarily improved things. > >> Now, I won#t change the mtu that soon again, I guess? > I guess. > > Sorry I couldn't provide an authoritative answer, > Dan Ha ha - never mind! Sometimes, I find it enough to chat about such issues with someone. I do have to admit, that on the 012 system, none of the ixgbe ports were already in use. You provided mental care and that alone helped. ;) Besides, you' re able to make something of the illumos sources - one thing that really escapes me? Cheers, budy From danmcd at omniti.com Wed Dec 10 22:10:43 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 10 Dec 2014 17:10:43 -0500 Subject: [OmniOS-discuss] NEWS: r151008 packages retroactively separated into own repo References: Message-ID: <63929042-B3DF-4A99-ABBF-8EF9DC11DCEE@omniti.com> Thank you to Eric Sproul (now of Circonus)! Here's his good news: . . . Good news everyone! As many of you are aware, the current practice for core OmniOS packaging is to separate packages for a given release into their own repository. This was prompted by all the headaches caused when r151008 packages were added to the release repo where r151006 (and earlier) lived. As of about an hour ago, I have split out the r151008 packages from http://pkg.omniti.com/omnios/release/ into http://pkg.omniti.com/omnios/r151008/ . The good news is that LTS users now no longer need to take extraordinary measures to stay on r151006. We'll be updating the documentation shortly. If you've already done the pkg freeze stuff, you can either leave it in place and plan to remove it when you're ready to upgrade to the next LTS, or go ahead and remove it now. Either way, you will never again see post-r151006 packages in the /omnios/release/ location. Since r151008 is EOL this is essentially just an archival operation-- no new packages will ever be published for this release. Those still running r151008 (pro tip: you should upgrade!) will no longer be able to install any existing packages unless you switch publisher URLs: pkg set-publisher -G -g omnios . . . I will reiterate the protip mentioned earlier. You should not be running 008 anywhere anymore. Thanks again to Eric! Dan From ikaufman at eng.ucsd.edu Thu Dec 11 00:17:49 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Wed, 10 Dec 2014 16:17:49 -0800 Subject: [OmniOS-discuss] weird library issue In-Reply-To: References: Message-ID: Thanks Dan. Alas, I get stuck here: root at massive02-b:~# pkg -R /mnt update Creating Plan - pkg update: The certificate which issued this certificate:/C=US/ST=Maryland/O=OmniTI/OU=OmniOS/CN=OmniOS r151012 Release Signing Certificate/emailAddress=omnios-support at omniti.com could not be found. The issuer is:/C=US/ST=Maryland/L=Fulton/O=OmniTI/CN=OmniTI Certificate Authority The package involved is:pkg://omnios/locale/gu at 0.5.11,5.11-0.151012:20140913T033547Z Ian On Wed, Dec 10, 2014 at 10:55 AM, Dan McDonald wrote: > >> On Dec 10, 2014, at 1:42 PM, Ian Kaufman wrote: >> >> It looks like this system is in some weird state where there are >> packages from 151006 and 151008. > > Oh my. > > Yes, you've been bitten by the pkg upgrade problems between 006 and 008. This was before my time here, so I'm not sure how best to extricate yourself from things. > > If you want, you can jump forward to the current stable release (r151012). If you want to try that, and you don't have zones, do this: > > > 1.) Create a new BE, call it 012 for this example: beadm create 012 > > 2.) Mount the BE on /mnt: beadm mount 012 /mnt > > 3.) Change the publisher on the new BE to point to r151012's repo: > pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/release/ -g http://pkg.omniti.com/omnios/r151012/ omnios > > 4.) Apply the update to the new BE: pkg -R /mnt update > > 5.) Unmount the BE: beadm umount 012 > > 6.) Either reboot and use the GRUB menu to try out the new 012 BE, or "beadm activate 012" to make it go there on the next reboot. > > > Dan > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From alexmcwhirter at mojovapes.net Thu Dec 11 00:47:28 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Wed, 10 Dec 2014 19:47:28 -0500 Subject: [OmniOS-discuss] Change Zone Hostname Message-ID: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? I had a look at /etc/nodname and it is an empty file when used in a zone. From tim at multitalents.net Thu Dec 11 06:30:01 2014 From: tim at multitalents.net (Tim Rice) Date: Wed, 10 Dec 2014 22:30:01 -0800 (PST) Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> Message-ID: On Wed, 10 Dec 2014, Alex McWhirter wrote: > I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? > > I had a look at /etc/nodname and it is an empty file when used in a zone. What happens if you put the hostname you want in /etc/nodname ? -- Tim Rice Multitalents tim at multitalents.net From alexmcwhirter at mojovapes.net Thu Dec 11 06:48:34 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Thu, 11 Dec 2014 01:48:34 -0500 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> Message-ID: Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist. I don't think this was ever an intended function for a zone. Sent from my iPhone > On Dec 11, 2014, at 1:30 AM, Tim Rice wrote: > >> On Wed, 10 Dec 2014, Alex McWhirter wrote: >> >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? >> >> I had a look at /etc/nodname and it is an empty file when used in a zone. > > What happens if you put the hostname you want in /etc/nodname ? > > > -- > Tim Rice Multitalents > tim at multitalents.net From danmcd at omniti.com Thu Dec 11 07:16:05 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 02:16:05 -0500 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> Message-ID: <4C2DF45B-2E0B-4624-BB35-DA3EC09CD51C@omniti.com> Editing nodename and rebooting the zone will give it a different name. I do this on my own HDC. Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 11, 2014, at 1:30 AM, Tim Rice wrote: > >> On Wed, 10 Dec 2014, Alex McWhirter wrote: >> >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? >> >> I had a look at /etc/nodname and it is an empty file when used in a zone. > > What happens if you put the hostname you want in /etc/nodname ? > > > -- > Tim Rice Multitalents > tim at multitalents.net > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From tim at multitalents.net Thu Dec 11 07:18:45 2014 From: tim at multitalents.net (Tim Rice) Date: Wed, 10 Dec 2014 23:18:45 -0800 (PST) Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> Message-ID: On Thu, 11 Dec 2014, Alex McWhirter wrote: | Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist. | | I don't think this was ever an intended function for a zone. Well it does on solaris 10 so I thought it might be worth a try. ..... # uname -n ftp22 # zonename ftp # cat /etc/nodename ftp22 # ..... | | Sent from my iPhone | | > On Dec 11, 2014, at 1:30 AM, Tim Rice wrote: | > | >> On Wed, 10 Dec 2014, Alex McWhirter wrote: | >> | >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? | >> | >> I had a look at /etc/nodname and it is an empty file when used in a zone. | > | > What happens if you put the hostname you want in /etc/nodname ? | > | > | > -- | > Tim Rice Multitalents | > tim at multitalents.net | _______________________________________________ | OmniOS-discuss mailing list | OmniOS-discuss at lists.omniti.com | http://lists.omniti.com/mailman/listinfo/omnios-discuss | | | -- Tim Rice Multitalents (707) 456-1146 tim at multitalents.net From alexmcwhirter at mojovapes.net Thu Dec 11 07:26:40 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Thu, 11 Dec 2014 02:26:40 -0500 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> Message-ID: <40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net> I had been toying around with a lot of different settings, it's possible I just messed something up in the process. Let me try again with a fresh zone. Sent from my iPhone > On Dec 11, 2014, at 2:18 AM, Tim Rice wrote: > > On Thu, 11 Dec 2014, Alex McWhirter wrote: > > | Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist. > | > | I don't think this was ever an intended function for a zone. > > Well it does on solaris 10 so I thought it might be worth a try. > ..... > # uname -n > ftp22 > # zonename > ftp > # cat /etc/nodename > ftp22 > # > ..... > > | > | Sent from my iPhone > | > | > On Dec 11, 2014, at 1:30 AM, Tim Rice wrote: > | > > | >> On Wed, 10 Dec 2014, Alex McWhirter wrote: > | >> > | >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? > | >> > | >> I had a look at /etc/nodname and it is an empty file when used in a zone. > | > > | > What happens if you put the hostname you want in /etc/nodname ? > | > > | > > | > -- > | > Tim Rice Multitalents > | > tim at multitalents.net > | _______________________________________________ > | OmniOS-discuss mailing list > | OmniOS-discuss at lists.omniti.com > | http://lists.omniti.com/mailman/listinfo/omnios-discuss > | > | > | > > -- > Tim Rice Multitalents (707) 456-1146 > tim at multitalents.net From jimklimov at cos.ru Thu Dec 11 10:38:05 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Thu, 11 Dec 2014 11:38:05 +0100 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: <40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net> References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net> <40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net> Message-ID: <472E6EE3-F06B-48B5-97AC-E6EEC513A7A1@cos.ru> 11 ??????? 2014??. 8:26:40 CET, Alex McWhirter ?????: >I had been toying around with a lot of different settings, it's >possible I just messed something up in the process. Let me try again >with a fresh zone. > >Sent from my iPhone > >> On Dec 11, 2014, at 2:18 AM, Tim Rice wrote: >> >> On Thu, 11 Dec 2014, Alex McWhirter wrote: >> >> | Yea, some services picked it up but others did not. It seems the >identity/node service does not have the config variable set for >nodename. I attempted to set it but received and error saying that the >variable didn't exist. >> | >> | I don't think this was ever an intended function for a zone. >> >> Well it does on solaris 10 so I thought it might be worth a try. >> ..... >> # uname -n >> ftp22 >> # zonename >> ftp >> # cat /etc/nodename >> ftp22 >> # >> ..... >> >> | >> | Sent from my iPhone >> | >> | > On Dec 11, 2014, at 1:30 AM, Tim Rice >wrote: >> | > >> | >> On Wed, 10 Dec 2014, Alex McWhirter wrote: >> | >> >> | >> I have a small cluster of servers hosting zones over multiple >VLAN?s. I would like to name the zones so that they don?t run into >naming collisions. For example, instead of ?web0? i would like to name >them ?2049_web0? and ?2050_web0?. The problem is that this set?s the >hostname to ?XXXX_web0?. Is there a way to change the hostname to be >something different than the zone name? >> | >> >> | >> I had a look at /etc/nodname and it is an empty file when used >in a zone. >> | > >> | > What happens if you put the hostname you want in /etc/nodname ? >> | > >> | > >> | > -- >> | > Tim Rice Multitalents >> | > tim at multitalents.net >> | _______________________________________________ >> | OmniOS-discuss mailing list >> | OmniOS-discuss at lists.omniti.com >> | http://lists.omniti.com/mailman/listinfo/omnios-discuss >> | >> | >> | >> >> -- >> Tim Rice Multitalents (707) 456-1146 >> tim at multitalents.net >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Did you reboot it after changing the file? Generally running "hostname `cat /etc/nodename" should've done the job too. -- Typos courtesy of K-9 Mail on my Samsung Android From tobi at oetiker.ch Thu Dec 11 15:44:37 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Thu, 11 Dec 2014 16:44:37 +0100 (CET) Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19 Message-ID: I am looking at upgrading the firmware of our LSI HBAs to P19 since we suspect that our use of P17 is the cause for disk timeouts we are seeing every few weeks. I am wondering, is it save to flash the HBAs from a running omnios system, and if so, has anyone written some notes down on the process and tools to use? cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From mmabis at vmware.com Thu Dec 11 16:36:13 2014 From: mmabis at vmware.com (Matthew Mabis) Date: Thu, 11 Dec 2014 16:36:13 +0000 Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19 In-Reply-To: References: Message-ID: <50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com> Just an FYI I wanted to pass some information along that I became aware of earlier today. There is a fairly serious firmware issue with the entire LSI 92XX HBA product line. The currently shipping firmware version for the 92XX HBA line is P20 which was released on September 22. The P20 firmware contains a bug that will show up as IO timeouts, disk errors and corruption on nearly every storage drive on LSI?s compatibility list. I became aware of the issue after observing a catastrophic ZFS storage failure and tracking it back to the HBAs. After isolating the issue I contacted LSI support who informed me of the bug and the fact that they have not publicly documented the issue. LSI support also informed me that the only fix for this issue is to return any effected HBA to the P19 firmware. Matt Mabis ________________________________________ From: OmniOS-discuss on behalf of Tobias Oetiker Sent: Thursday, December 11, 2014 7:44 AM To: omnios-discuss at lists.omniti.com Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19 I am looking at upgrading the firmware of our LSI HBAs to P19 since we suspect that our use of P17 is the cause for disk timeouts we are seeing every few weeks. I am wondering, is it save to flash the HBAs from a running omnios system, and if so, has anyone written some notes down on the process and tools to use? cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.omniti.com_mailman_listinfo_omnios-2Ddiscuss&d=AAICAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=6lrHKBYs7FGYRAd987joFNP44jNhX5Ha5IypFnBFBJE&m=ffEtrdn5eO8TKjy27ecAvbkgdKvSgf_ODZ0kSPuqz50&s=HyYgGdm8SWTXlGepFA7xj-5PYCtJBlSmai-tLvol3aQ&e= From mir at miras.org Thu Dec 11 17:16:22 2014 From: mir at miras.org (Michael Rasmussen) Date: Thu, 11 Dec 2014 18:16:22 +0100 Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19 In-Reply-To: <50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com> References: <50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com> Message-ID: <20141211181622.4bf08199@sleipner.datanom.net> On Thu, 11 Dec 2014 16:36:13 +0000 Matthew Mabis wrote: have not publicly documented the issue. LSI support also informed me that the only fix for this issue is to return any effected HBA to the P19 firmware. > Someone on this list has also claimed that P18 should be a safe harbor. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: "On the Internet, no one knows you're using Windows NT" (Submitted by Ramiro Estrugo, restrugo at fateware.com) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From gearboxes at outlook.com Thu Dec 11 17:47:49 2014 From: gearboxes at outlook.com (Machine Man) Date: Thu, 11 Dec 2014 12:47:49 -0500 Subject: [OmniOS-discuss] KVM - Windows guest disks Message-ID: Hi, Are there any pointers for create the underlying ZFS volume for the windows disk. Currently I am using block volume created at 4k. The servers have LSI 2308 controllers in IT mode and I am seeing high disk queue lengths with the system not doing really anything. The system was just installed. It also boots slow and the pool is maxed out. Pool consists of 6x 7200K NL-SAS 2 drives raidz2. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjorge+ml at blackdot.be Thu Dec 11 17:55:57 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Thu, 11 Dec 2014 18:55:57 +0100 Subject: [OmniOS-discuss] KVM - Windows guest disks In-Reply-To: References: Message-ID: <593b44443c150ad3a8a1d7af9d6e2285@blackdot.be> Are you running r012? Some other people reported poor IO with KVM earlier this week. On 2014-12-11 18:47, Machine Man wrote: > Hi, > Are there any pointers for create the underlying ZFS volume for the > windows disk. > Currently I am using block volume created at 4k. > The servers have LSI 2308 controllers in IT mode and I am seeing high > disk queue lengths with the system not doing really anything. The > system was just installed. It also boots slow and the pool is maxed > out. > Pool consists of 6x 7200K NL-SAS 2 drives raidz2. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss [1] Links: ------ [1] http://lists.omniti.com/mailman/listinfo/omnios-discuss From chip at innovates.com Thu Dec 11 18:30:45 2014 From: chip at innovates.com (Schweiss, Chip) Date: Thu, 11 Dec 2014 12:30:45 -0600 Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19 In-Reply-To: References: Message-ID: On Thu, Dec 11, 2014 at 9:44 AM, Tobias Oetiker wrote: > I am looking at upgrading the firmware of our LSI HBAs to P19 since > we suspect that our use of P17 is the cause for disk timeouts we > are seeing every few weeks. > > I am wondering, is it save to flash the HBAs from a running omnios system, > and if so, has anyone written some notes down on the process and > tools to use? > > I don't believe that is possible on a live system. I have upgraded using the Solaris sas2flash utility on a system with the pool exported. The last step of the flash function is to reset the adapter which fails requiring the system to be rebooted. What I haven't tried is disconnecting the SAS cables then applying the update. I suspect this may work, but is only a useful method if you have redundant SAS paths and can do one HBA at a time. These issues are why I run HA systems and can migrate my pools from server to server and perform maintenance at a leisurely pace and confirm everything is the way I want it before putting the server back in service. -Chip > cheers > tobi > > > -- > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joeveliscos at gmail.com Thu Dec 11 21:00:53 2014 From: joeveliscos at gmail.com (Joe Veliscos) Date: Thu, 11 Dec 2014 22:00:53 +0100 Subject: [OmniOS-discuss] Python difference Message-ID: Hi, I have an application working on Omnios r10 which depends on certain python libraries. There are certain environment variables in place which point to the location of those libraries. I have updated the r10 machine to r12. The application now cannot be started with errors stating that the needed libraries cannot be found in the given paths. Maybe somebody can tell me what the difference is in python version between the two omnios releases. Python -V gives me 2.6.8. on both versions. I am looking for pointers I can use to make this application work. Rgds Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Dec 11 21:26:23 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 16:26:23 -0500 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: Message-ID: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos wrote: > > Hi, > > I have an application working on Omnios r10 which depends on certain python libraries. There are certain environment variables in place which point to the location of those libraries. > > I have updated the r10 machine to r12. The application now cannot be started with errors stating that the needed libraries cannot be found in the given paths. Share the errors please? I'll need more details. > Maybe somebody can tell me what the difference is in python version between the two omnios releases. Python -V gives me 2.6.8. on both versions. We updated supplemental python libraries, which may contribute to what you're seeing. Also, we had not updated the "entire" metapackage on 010 to show we were actually running 2.6.8. Dan From joeveliscos at gmail.com Thu Dec 11 22:23:49 2014 From: joeveliscos at gmail.com (Joe Veliscos) Date: Thu, 11 Dec 2014 23:23:49 +0100 Subject: [OmniOS-discuss] Python difference In-Reply-To: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: Hi The command that I execute : root#crm abort: couldn't find crm libraries in [/opt/ha/sbin /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip /usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] (check your install and PYTHONPATH) I have the following environment variables set: export PYTHONPATH=/opt/ha/lib/python2.6/site-packages export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH export OCF_ROOT=/opt/ha/lib/ocf export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat I have exactly the same in an r10 release (pre upgrade to rr12) where there is no problem I did a truss -d crm and it seems that many files it searches for are not found. Snippets of the output (very long file) hope this helps: Below (as I understand it ) some searches which it can resolve: 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023) = 18 0.0100 resolvepath("/usr/bin/amd64/python2.6", "/usr/bin/amd64/python2.6", 1023) = 24 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) = 0 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 0.0113 close(3) = 0 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 0.0127 close(3) = 0 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 0.0139 close(3) = 0 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 0.0150 close(3) = 0 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1", 1023) = 20 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 0.0162 close(3) = 0 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0168 resolvepath("/lib/64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Below searches which it cannot resolve: 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22 EINVAL 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) Err#2 ENOENT 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0 0.0351 brk(0x00485250) = 0 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 0.0459 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) = 0 0.0461 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) = 0 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY) Err#2 ENOENT 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", O_RDONLY) Err#2 ENOENT 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) Err#2 ENOENT 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) Err#2 ENOENT 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 ENOENT 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 ENOENT 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 0.0494 brk(0x004D5250) = 0 0.0496 brk(0x004D9250) = 0 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) Err#2 ENOENT 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", O_RDONLY) Err#2 ENOENT 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) Err#2 ENOENT 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) Err#2 ENOENT 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", O_RDONLY) Err#2 ENOENT 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0575 open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", O_RDONLY) Err#2 ENOENT 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", O_RDONLY) Err#2 ENOENT 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 And the list goes on. Hope there's a solution for this. Joe On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald wrote: > > > > On Dec 11, 2014, at 4:00 PM, Joe Veliscos wrote: > > > > Hi, > > > > I have an application working on Omnios r10 which depends on certain > python libraries. There are certain environment variables in place which > point to the location of those libraries. > > > > I have updated the r10 machine to r12. The application now cannot be > started with errors stating that the needed libraries cannot be found in > the given paths. > > Share the errors please? I'll need more details. > > > Maybe somebody can tell me what the difference is in python version > between the two omnios releases. Python -V gives me 2.6.8. on both versions. > > We updated supplemental python libraries, which may contribute to what > you're seeing. Also, we had not updated the "entire" metapackage on 010 to > show we were actually running 2.6.8. > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Thu Dec 11 22:25:48 2014 From: rt at steait.net (Rune Tipsmark) Date: Thu, 11 Dec 2014 22:25:48 +0000 Subject: [OmniOS-discuss] hangs on reboot Message-ID: <1418336750819.83182@steait.net> hi all, I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot. The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1 two nics igb0 and igb1 configured as aggregation (aggr0) besides this its 100% default install, not 10 minutes old... Any ideas? br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From henson at acm.org Thu Dec 11 22:32:26 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 11 Dec 2014 14:32:26 -0800 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <1418336750819.83182@steait.net> References: <1418336750819.83182@steait.net> Message-ID: <893101d01592$59168140$0b4383c0$@acm.org> > Rune Tipsmark > Sent: Thursday, December 11, 2014 2:26 PM > > I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 > have issues rebooting. They simply hang and never ever reboot. Disable fast reboot: svccfg -s "system/boot-config:default" setprop config/fastreboot_default=false svccfg -s "system/boot-config:default" setprop config/fastreboot_onpanic=false svcadm refresh svc:/system/boot-config:default Some systems (it seems particularly common on supermicro hardware) tend to wedge up with fast reboot enabled. If you want to verify this is the issue before changing the configuration, try 'reboot -p' which should do a one time regular reboot without changing the default. Dan, there was some talk of making fast reboot disabled the default, and having people who wanted it need to enable it, rather than the other way around? Any thoughts? From danmcd at omniti.com Thu Dec 11 22:37:18 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 17:37:18 -0500 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com> Did you look to see if there's anything in /opt/ha/lib/python2.6/site-packages ? The massive ENOENTs you're showing me are just library searching. See how several path and filename combos are tried? HA should be more clear about how it failed. Also, look toward the end of the truss, and just look for open calls. Once you see a string of failures that doesn't end with a success, you'll know what's missing. Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 11, 2014, at 5:23 PM, Joe Veliscos wrote: > > Hi > > The command that I execute : > root#crm > abort: couldn't find crm libraries in [/opt/ha/sbin /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip /usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] > (check your install and PYTHONPATH) > > I have the following environment variables set: > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH > export OCF_ROOT=/opt/ha/lib/ocf > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat > > I have exactly the same in an r10 release (pre upgrade to rr12) where there is no problem > > I did a truss -d crm and it seems that many files it searches for are not found. Snippets of the output (very long file) hope this helps: > > Below (as I understand it ) some searches which it can resolve: > > 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023) = 18 > 0.0100 resolvepath("/usr/bin/amd64/python2.6", "/usr/bin/amd64/python2.6", 1023) = 24 > 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 > 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT > 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) = 0 > 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 > 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 > 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0113 close(3) = 0 > 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 > 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 > 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0127 close(3) = 0 > 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 > 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22 > 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 > 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0139 close(3) = 0 > 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 > 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20 > 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 > 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0150 close(3) = 0 > 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 > 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1", 1023) = 20 > 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 > 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0162 close(3) = 0 > 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 > 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 > 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0168 resolvepath("/lib/64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22 > 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20 > 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 > 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Below searches which it cannot resolve: > > 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 > 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 > 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22 EINVAL > 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT > 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT > 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT > 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 > 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT > 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) Err#2 ENOENT > 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0 > 0.0351 brk(0x00485250) = 0 > > 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 > 0.0459 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) = 0 > 0.0461 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) = 0 > 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY) Err#2 ENOENT > 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", O_RDONLY) Err#2 ENOENT > 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) Err#2 ENOENT > 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) Err#2 ENOENT > 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT > 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 > 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT > 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 > 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 > 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 ENOENT > 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 ENOENT > 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 > 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 > 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 > 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 > 0.0494 brk(0x004D5250) = 0 > 0.0496 brk(0x004D9250) = 0 > 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 > 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY > 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 > > 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT > 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) Err#2 ENOENT > 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", O_RDONLY) Err#2 ENOENT > 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) Err#2 ENOENT > 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) Err#2 ENOENT > 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT > 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT > 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT > 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 > 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 > 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 > 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 > 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 > 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY > 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 > > 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT > 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", O_RDONLY) Err#2 ENOENT > 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0575 open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT > 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", O_RDONLY) Err#2 ENOENT > 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", O_RDONLY) Err#2 ENOENT > 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT > 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT > 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT > 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 > 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 > 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 > 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 > > > And the list goes on. > > Hope there's a solution for this. > > > Joe > > >> On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald wrote: >> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos wrote: >> > >> > Hi, >> > >> > I have an application working on Omnios r10 which depends on certain python libraries. There are certain environment variables in place which point to the location of those libraries. >> > >> > I have updated the r10 machine to r12. The application now cannot be started with errors stating that the needed libraries cannot be found in the given paths. >> >> Share the errors please? I'll need more details. >> >> > Maybe somebody can tell me what the difference is in python version between the two omnios releases. Python -V gives me 2.6.8. on both versions. >> >> We updated supplemental python libraries, which may contribute to what you're seeing. Also, we had not updated the "entire" metapackage on 010 to show we were actually running 2.6.8. >> >> Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Dec 11 22:39:37 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 17:39:37 -0500 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <1418336750819.83182@steait.net> References: <1418336750819.83182@steait.net> Message-ID: Nothing printed out on the console? And try eliminating the target mode first - just in case. Also, is one of your NICs a dual IPMI/host NIC? If so, disable the IPMI portion, we don't cope with shared NIC ports like that. Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 11, 2014, at 5:25 PM, Rune Tipsmark wrote: > > hi all, > > > > I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot. > > > > The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1 > > > > two nics igb0 and igb1 configured as aggregation (aggr0) > > > > besides this its 100% default install, not 10 minutes old... > > > > Any ideas? > > br, > > Rune > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Dec 11 22:40:49 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 17:40:49 -0500 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <893101d01592$59168140$0b4383c0$@acm.org> References: <1418336750819.83182@steait.net> <893101d01592$59168140$0b4383c0$@acm.org> Message-ID: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> Thanks for the fast reboot catch! Yes, were contemplating it for '014 AND for upstreaming. Not sure yet if it'll happen, but this thread makes a compelling case. Dan Sent from my iPhone (typos, autocorrect, and all) On Dec 11, 2014, at 5:32 PM, Paul B. Henson wrote: >> Rune Tipsmark >> Sent: Thursday, December 11, 2014 2:26 PM >> >> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 >> have issues rebooting. They simply hang and never ever reboot. > > Disable fast reboot: > > svccfg -s "system/boot-config:default" setprop > config/fastreboot_default=false > svccfg -s "system/boot-config:default" setprop > config/fastreboot_onpanic=false > svcadm refresh svc:/system/boot-config:default > > Some systems (it seems particularly common on supermicro hardware) tend to > wedge up with fast reboot enabled. If you want to verify this is the issue > before changing the configuration, try 'reboot -p' which should do a one > time regular reboot without changing the default. > > Dan, there was some talk of making fast reboot disabled the default, and > having people who wanted it need to enable it, rather than the other way > around? Any thoughts? > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From rt at steait.net Thu Dec 11 22:41:35 2014 From: rt at steait.net (Rune Tipsmark) Date: Thu, 11 Dec 2014 22:41:35 +0000 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: References: <1418336750819.83182@steait.net>, Message-ID: <1418337697672.85350@steait.net> NIC's are not shared with IPMI. I will try the target-mode first, however on the 3rd box I actually have Infiniband and it also caused the same issue without touching the target-mode for FC. I have a 4th system on an old HP server and it reboots perfect every time, no target-mode and has Infiniband as well... I am leaning towards something with the SuperMicro hardware but can't really pinpoint it. br, Rune ________________________________ From: Dan McDonald Sent: Thursday, December 11, 2014 11:39 PM To: Rune Tipsmark Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] hangs on reboot Nothing printed out on the console? And try eliminating the target mode first - just in case. Also, is one of your NICs a dual IPMI/host NIC? If so, disable the IPMI portion, we don't cope with shared NIC ports like that. Dan Sent from my iPhone (typos, autocorrect, and all) On Dec 11, 2014, at 5:25 PM, Rune Tipsmark > wrote: hi all, I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot. The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1 two nics igb0 and igb1 configured as aggregation (aggr0) besides this its 100% default install, not 10 minutes old... Any ideas? br, Rune _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Thu Dec 11 22:48:39 2014 From: rt at steait.net (Rune Tipsmark) Date: Thu, 11 Dec 2014 22:48:39 +0000 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: References: <1418336750819.83182@steait.net>, Message-ID: <1418338121673.68415@steait.net> still same... output can be seen here: http://i.imgur.com/BuwaGGn.png ________________________________ From: Dan McDonald Sent: Thursday, December 11, 2014 11:39 PM To: Rune Tipsmark Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] hangs on reboot Nothing printed out on the console? And try eliminating the target mode first - just in case. Also, is one of your NICs a dual IPMI/host NIC? If so, disable the IPMI portion, we don't cope with shared NIC ports like that. Dan Sent from my iPhone (typos, autocorrect, and all) On Dec 11, 2014, at 5:25 PM, Rune Tipsmark > wrote: hi all, I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot. The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1 two nics igb0 and igb1 configured as aggregation (aggr0) besides this its 100% default install, not 10 minutes old... Any ideas? br, Rune _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Thu Dec 11 23:04:53 2014 From: rt at steait.net (Rune Tipsmark) Date: Thu, 11 Dec 2014 23:04:53 +0000 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <893101d01592$59168140$0b4383c0$@acm.org> References: <1418336750819.83182@steait.net>, <893101d01592$59168140$0b4383c0$@acm.org> Message-ID: <1418339095532.45737@steait.net> didn't even see this one in the thread, reboot -p did it for me. I have a system where I had done the below but it still hangs, will try -p on it during the weekend. thx, br Rune ________________________________________ From: Paul Henson on behalf of Paul B. Henson Sent: Thursday, December 11, 2014 11:32 PM To: Rune Tipsmark; omnios-discuss at lists.omniti.com Subject: RE: [OmniOS-discuss] hangs on reboot > Rune Tipsmark > Sent: Thursday, December 11, 2014 2:26 PM > > I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 > have issues rebooting. They simply hang and never ever reboot. Disable fast reboot: svccfg -s "system/boot-config:default" setprop config/fastreboot_default=false svccfg -s "system/boot-config:default" setprop config/fastreboot_onpanic=false svcadm refresh svc:/system/boot-config:default Some systems (it seems particularly common on supermicro hardware) tend to wedge up with fast reboot enabled. If you want to verify this is the issue before changing the configuration, try 'reboot -p' which should do a one time regular reboot without changing the default. Dan, there was some talk of making fast reboot disabled the default, and having people who wanted it need to enable it, rather than the other way around? Any thoughts? From joeveliscos at gmail.com Thu Dec 11 23:23:25 2014 From: joeveliscos at gmail.com (Joe Veliscos) Date: Fri, 12 Dec 2014 00:23:25 +0100 Subject: [OmniOS-discuss] Python difference In-Reply-To: <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com> References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com> Message-ID: Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains maps with crm (HA) specific python modules . At the end of the truss there's a long list of unsuccesfull open calls for modules many of which are to a readline module e.g. (short list) /usr/lib/python2.6/lib-dynload/cStringIO /usr/lib/python2.6/lib-tk/cStringIO.pyc /usr/lib/python2.6/plat-sunos5/64/cStringIO.so /opt/ha/lib/python2.6/site-packages/readline /opt/ha/lib/python2.6/site-packages/64/readline.so /opt/ha/lib/python2.6/site-packages/readline.py These are nowhere to be found on the system! How would I go about getting such into the system? Compile python from source or did the update go wrong? Just trying to understand the situation. Thanks for your help. Joe On Thu, Dec 11, 2014 at 11:37 PM, Dan McDonald wrote: > > Did you look to see if there's anything in > /opt/ha/lib/python2.6/site-packages ? > > The massive ENOENTs you're showing me are just library searching. See how > several path and filename combos are tried? > > HA should be more clear about how it failed. Also, look toward the end of > the truss, and just look for open calls. Once you see a string of failures > that doesn't end with a success, you'll know what's missing. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Dec 11, 2014, at 5:23 PM, Joe Veliscos wrote: > > Hi > > The command that I execute : > root#crm > abort: couldn't find crm libraries in [/opt/ha/sbin > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip > /usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] > (check your install and PYTHONPATH) > > I have the following environment variables set: > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH > export OCF_ROOT=/opt/ha/lib/ocf > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat > > I have exactly the same in an r10 release (pre upgrade to rr12) where > there is no problem > > I did a truss -d crm and it seems that many files it searches for are not > found. Snippets of the output (very long file) hope this helps: > > Below (as I understand it ) some searches which it can resolve: > > 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", > 1023) = 18 > 0.0100 resolvepath("/usr/bin/amd64/python2.6", > "/usr/bin/amd64/python2.6", 1023) = 24 > 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 > 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT > 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", > 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) > = 0 > 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 > 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 > 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0113 close(3) = 0 > 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 > 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, > 0, 0) = 0 > 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) > Err#2 ENOENT > 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", > "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 > 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0127 close(3) = 0 > 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 > 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) > Err#2 ENOENT > 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", > "/lib/amd64/libnsl.so.1", 1023) = 22 > 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 > 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0139 close(3) = 0 > 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 > 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, > 0, 0) = 0 > 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 > ENOENT > 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2", > 1023) = 20 > 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 > 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0150 close(3) = 0 > 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 > 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 > ENOENT > 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1", > 1023) = 20 > 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 > 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0162 close(3) = 0 > 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 > 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, > 0, 0) = 0 > 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0168 resolvepath("/lib/64/libsocket.so.1", > "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", > 1023) = 22 > 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) > = 20 > 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) > Err#2 ENOENT > 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 > 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Below searches which it cannot resolve: > > 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 > 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 > 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22 > EINVAL > 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT > 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 > ENOENT > 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2 > ENOENT > 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 > 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT > 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) > Err#2 ENOENT > 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0 > 0.0351 brk(0x00485250) = 0 > > 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 > 0.0459 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) > = 0 > 0.0461 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) > = 0 > 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", > O_RDONLY) Err#2 ENOENT > 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", > O_RDONLY) Err#2 ENOENT > 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) > Err#2 ENOENT > 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) > Err#2 ENOENT > 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT > 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 > 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT > 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 > 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 > 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 ENOENT > 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 > ENOENT > 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 > 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 > 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 > 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 > 0.0494 brk(0x004D5250) = 0 > 0.0496 brk(0x004D9250) = 0 > 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 > 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY > 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 > > 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD320) Err#2 ENOENT > 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) > Err#2 ENOENT > 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", > O_RDONLY) Err#2 ENOENT > 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) > Err#2 ENOENT > 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) > Err#2 ENOENT > 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT > 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT > 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT > 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 > 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 > 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 > 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 > 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 > 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY > 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 > > 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC000) Err#2 ENOENT > 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", > O_RDONLY) Err#2 ENOENT > 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0575 > open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY) > Err#2 ENOENT > 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", > O_RDONLY) Err#2 ENOENT > 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", > O_RDONLY) Err#2 ENOENT > 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 > ENOENT > 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 > ENOENT > 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) > Err#2 ENOENT > 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 > 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 > 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 > 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 > > > And the list goes on. > > Hope there's a solution for this. > > > Joe > > > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald wrote: >> >> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos >> wrote: >> > >> > Hi, >> > >> > I have an application working on Omnios r10 which depends on certain >> python libraries. There are certain environment variables in place which >> point to the location of those libraries. >> > >> > I have updated the r10 machine to r12. The application now cannot be >> started with errors stating that the needed libraries cannot be found in >> the given paths. >> >> Share the errors please? I'll need more details. >> >> > Maybe somebody can tell me what the difference is in python version >> between the two omnios releases. Python -V gives me 2.6.8. on both versions. >> >> We updated supplemental python libraries, which may contribute to what >> you're seeing. Also, we had not updated the "entire" metapackage on 010 to >> show we were actually running 2.6.8. >> >> Dan >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexmcwhirter at mojovapes.net Thu Dec 11 23:35:07 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Thu, 11 Dec 2014 18:35:07 -0500 Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted, hangs on shutdown. Message-ID: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> I have a client that is using iSCSI mounts as zone storage. Upon system shutdown the host kills the network before unmounting the iSCSI zpools. This causes the system to hang as it attempts to unmount the zpool after the disks have gone offline. Whats the best way to prevent something like this? From danmcd at omniti.com Thu Dec 11 23:42:52 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 11 Dec 2014 18:42:52 -0500 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com> Message-ID: > On Dec 11, 2014, at 6:23 PM, Joe Veliscos wrote: > > Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains maps with crm (HA) specific python modules . > > At the end of the truss there's a long list of unsuccesfull open calls for modules many of which are to a readline module e.g. (short list) > > /usr/lib/python2.6/lib-dynload/cStringIO > /usr/lib/python2.6/lib-tk/cStringIO.pyc > /usr/lib/python2.6/plat-sunos5/64/cStringIO.so > > /opt/ha/lib/python2.6/site-packages/readline > /opt/ha/lib/python2.6/site-packages/64/readline.so > /opt/ha/lib/python2.6/site-packages/readline.py > > These are nowhere to be found on the system! > > How would I go about getting such into the system? Compile python from source or did the update go wrong? Just trying to understand the situation. But they are on your r151010 BE? If you have the old BE, use: beadm mount old-be /mnt and go poking around in there to see where they are. Also, how do you install the HA packages? Perhaps you need to reinstall them? BTW, I don't know much about HA, but maybe someone else on this list does, especially with how it interacts with r151012. Dan From joeveliscos at gmail.com Fri Dec 12 00:31:41 2014 From: joeveliscos at gmail.com (Joe Veliscos) Date: Fri, 12 Dec 2014 01:31:41 +0100 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com> Message-ID: Just checked, Most of them are in the /usr/lib/python2.6/ and sub directories. The HA package was created by sazo, usually a frequent member of this list. I seems there's a diff in the installed python packages between releases so I'll have to play around to get this working I think. Thanks for all your help and pointers again, High regards, Joe On Fri, Dec 12, 2014 at 12:42 AM, Dan McDonald wrote: > > > > On Dec 11, 2014, at 6:23 PM, Joe Veliscos wrote: > > > > Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains > maps with crm (HA) specific python modules . > > > > At the end of the truss there's a long list of unsuccesfull open calls > for modules many of which are to a readline module e.g. (short list) > > > > /usr/lib/python2.6/lib-dynload/cStringIO > > /usr/lib/python2.6/lib-tk/cStringIO.pyc > > /usr/lib/python2.6/plat-sunos5/64/cStringIO.so > > > > /opt/ha/lib/python2.6/site-packages/readline > > /opt/ha/lib/python2.6/site-packages/64/readline.so > > /opt/ha/lib/python2.6/site-packages/readline.py > > > > These are nowhere to be found on the system! > > > > How would I go about getting such into the system? Compile python from > source or did the update go wrong? Just trying to understand the situation. > > But they are on your r151010 BE? If you have the old BE, use: > > beadm mount old-be /mnt > > and go poking around in there to see where they are. Also, how do you > install the HA packages? Perhaps you need to reinstall them? > > BTW, I don't know much about HA, but maybe someone else on this list does, > especially with how it interacts with r151012. > > Dan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexmcwhirter at mojovapes.net Fri Dec 12 06:12:48 2014 From: alexmcwhirter at mojovapes.net (Alex McWhirter) Date: Fri, 12 Dec 2014 01:12:48 -0500 Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted, hangs on shutdown. In-Reply-To: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> Message-ID: This is an exact description of the error i?m having. Changing the timeout did not solve this problem. Nor did adding /network/iscsi/initiator as a dependecy of /system/filesystem/local http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/ Hopefully someone can chime in on this, i really need zones to work over shared storage. > On Dec 11, 2014, at 6:35 PM, Alex McWhirter wrote: > > I have a client that is using iSCSI mounts as zone storage. Upon system shutdown the host kills the network before unmounting the iSCSI zpools. This causes the system to hang as it attempts to unmount the zpool after the disks have gone offline. Whats the best way to prevent something like this? > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimklimov at cos.ru Fri Dec 12 06:36:56 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 07:36:56 +0100 Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted, hangs on shutdown. In-Reply-To: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> Message-ID: 12 ??????? 2014??. 0:35:07 CET, Alex McWhirter ?????: >I have a client that is using iSCSI mounts as zone storage. Upon system >shutdown the host kills the network before unmounting the iSCSI zpools. >This causes the system to hang as it attempts to unmount the zpool >after the disks have gone offline. Whats the best way to prevent >something like this? >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Hi, this seems exactly like a job for my half-baked procedures (manual work needed during setup) to wrap each pool, zone and iscsi service into SMF instances so you can set up dependencies for ordered startup-shutdown. The first two I use on a number of machines (Solaris 10, OpenSolaris Nevada, OpenIndiana and now starting with OmniOS). Don't have much iscsi beside initial experiments though... See here: http://wiki.openindiana.org/display/oi/Zones+as+SMF+services http://wiki.openindiana.org/oi/Advanced+-+ZFS+Pools+as+SMF+services+and+iSCSI+loopback+mounts Hope this helps, Jim Klimov -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Fri Dec 12 06:42:38 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 07:42:38 +0100 Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted, hangs on shutdown. In-Reply-To: References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net> Message-ID: <15349DD3-62FD-4FE7-AABE-3E76AE636DE7@cos.ru> 12 ??????? 2014??. 7:12:48 CET, Alex McWhirter ?????: >This is an exact description of the error i?m having. Changing the >timeout did not solve this problem. Nor did adding >/network/iscsi/initiator as a dependecy of /system/filesystem/local > >http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/ > > >Hopefully someone can chime in on this, i really need zones to work >over shared storage. > > > >> On Dec 11, 2014, at 6:35 PM, Alex McWhirter > wrote: >> >> I have a client that is using iSCSI mounts as zone storage. Upon >system shutdown the host kills the network before unmounting the iSCSI >zpools. This causes the system to hang as it attempts to unmount the >zpool after the disks have gone offline. Whats the best way to prevent >something like this? >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Just in case: you did do "svcadm refresh servicename" after svccfg to actually apply the new attributes, right? -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Fri Dec 12 07:52:23 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 08:52:23 +0100 Subject: [OmniOS-discuss] Java paths in OmniOS Message-ID: Hi all, While trying to install Java in OmniOS, I found it delivers one. However, it has 'simplified' the path structure by removing essentially the tree for different jvm versions under /usr/jdk, instead making it equivalent to /usr/java... In Sun packages, there were /usr/jdk/instances/$manyjavaversions/, a /usr/jdk/latest symlink pointing to ./instances/$somejavaversion and a /usr/java symlink to ./jdk/latest (typically JAVA_HOME=/usr/java) and some 20 binaries like /usr/bin/javac being symlinks to ../java/bin/javac. Symlinks like the 'latest' could be easily changed by an admin to choose a particular JVM as default. While the $manyjavaversions could be managed by packaging, this also allowed to pluck in zip-installs and/or customized jvm's (custom timezone updates, ca's, jce settings etc.) to jse certain jvm versions for some obscure software which did not like updates for example. The new structure all but forbids such customization; was there and particular benefit from changing to it? Does anything in OmniOS or omniti-ms packages depend on the new paths (can I locally change back to old structure without something strangely misbehaving)? Thanks, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Fri Dec 12 07:58:33 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 08:58:33 +0100 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: References: <1418336750819.83182@steait.net> <1418337697672.85350@steait.net> Message-ID: 12 ??????? 2014??. 8:35:18 CET, Jim Klimov ?????: >11 ??????? 2014??. 23:41:35 CET, Rune Tipsmark ?????: >>NIC's are not shared with IPMI. >> >> >> >>I will try the target-mode first, however on the 3rd box I actually >>have Infiniband and it also caused the same issue without touching the >>target-mode for FC. >> >> >> >>I have a 4th system on an old HP server and it reboots perfect every >>time, no target-mode and has Infiniband as well... I am leaning >towards >>something with the SuperMicro hardware but can't really pinpoint it. >> >>br, >> >>Rune >> >>________________________________ >>From: Dan McDonald >>Sent: Thursday, December 11, 2014 11:39 PM >>To: Rune Tipsmark >>Cc: omnios-discuss at lists.omniti.com >>Subject: Re: [OmniOS-discuss] hangs on reboot >> >>Nothing printed out on the console? >> >>And try eliminating the target mode first - just in case. >> >>Also, is one of your NICs a dual IPMI/host NIC? If so, disable the >>IPMI portion, we don't cope with shared NIC ports like that. >> >>Dan >> >>Sent from my iPhone (typos, autocorrect, and all) >> >>On Dec 11, 2014, at 5:25 PM, Rune Tipsmark >>> wrote: >> >> >>hi all, >> >> >> >>I got a bunch (3) installations of omnios on SuperMicro hardware and >>all 3 have issues rebooting. They simply hang and never ever reboot. >> >> >> >>The install is latest version and I only added the storage-server >>package and installed napp-it and changed the fibre channel setting in >>/kernel/drv/emlxs.conf target-mode=1 >> >> >> >>two nics igb0 and igb1 configured as aggregation (aggr0) >> >> >> >>besides this its 100% default install, not 10 minutes old... >> >> >> >>Any ideas? >> >>br, >> >>Rune >> >>_______________________________________________ >>OmniOS-discuss mailing list >>OmniOS-discuss at lists.omniti.com >>http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> >>------------------------------------------------------------------------ >> >>_______________________________________________ >>OmniOS-discuss mailing list >>OmniOS-discuss at lists.omniti.com >>http://lists.omniti.com/mailman/listinfo/omnios-discuss > >I've installed latest bloody on an HP Z400 workstation last week with >an lsi 9212 addon hba, and it too needed to disable fastboot. > >IIRC this is somehow tied to ability of all drivers to quiesce like for >standby - if some one refuses to, suspend or fastboot fail. >IMHO the fastboot handler should have a reasonable/configurable timeout >after 'stand by for reboot' to do slowboot if that expires. > >Alternately on servers you can set up bmc-watchdog and have the mobo >reboot itself. But then use long timeouts (10-15 min) and/or take care >that the watchdog driver is among the first pieces of software that >runs in your bootup or especially 'live' repair-boots. > >HTH, >Jim >-- >Typos courtesy of K-9 Mail on my Samsung Android Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also a signal for devs to improve at least nominal hw support for new usb3, video chips, maybe some sata HBAs? Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or usb1, or let radeon be a vesa-video, but let it work without errors) -- they should not impede system work either. Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jimklimov at cos.ru Fri Dec 12 07:35:18 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 08:35:18 +0100 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <1418337697672.85350@steait.net> References: <1418336750819.83182@steait.net> <1418337697672.85350@steait.net> Message-ID: 11 ??????? 2014??. 23:41:35 CET, Rune Tipsmark ?????: >NIC's are not shared with IPMI. > > > >I will try the target-mode first, however on the 3rd box I actually >have Infiniband and it also caused the same issue without touching the >target-mode for FC. > > > >I have a 4th system on an old HP server and it reboots perfect every >time, no target-mode and has Infiniband as well... I am leaning towards >something with the SuperMicro hardware but can't really pinpoint it. > >br, > >Rune > >________________________________ >From: Dan McDonald >Sent: Thursday, December 11, 2014 11:39 PM >To: Rune Tipsmark >Cc: omnios-discuss at lists.omniti.com >Subject: Re: [OmniOS-discuss] hangs on reboot > >Nothing printed out on the console? > >And try eliminating the target mode first - just in case. > >Also, is one of your NICs a dual IPMI/host NIC? If so, disable the >IPMI portion, we don't cope with shared NIC ports like that. > >Dan > >Sent from my iPhone (typos, autocorrect, and all) > >On Dec 11, 2014, at 5:25 PM, Rune Tipsmark >> wrote: > > >hi all, > > > >I got a bunch (3) installations of omnios on SuperMicro hardware and >all 3 have issues rebooting. They simply hang and never ever reboot. > > > >The install is latest version and I only added the storage-server >package and installed napp-it and changed the fibre channel setting in >/kernel/drv/emlxs.conf target-mode=1 > > > >two nics igb0 and igb1 configured as aggregation (aggr0) > > > >besides this its 100% default install, not 10 minutes old... > > > >Any ideas? > >br, > >Rune > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss I've installed latest bloody on an HP Z400 workstation last week with an lsi 9212 addon hba, and it too needed to disable fastboot. IIRC this is somehow tied to ability of all drivers to quiesce like for standby - if some one refuses to, suspend or fastboot fail. IMHO the fastboot handler should have a reasonable/configurable timeout after 'stand by for reboot' to do slowboot if that expires. Alternately on servers you can set up bmc-watchdog and have the mobo reboot itself. But then use long timeouts (10-15 min) and/or take care that the watchdog driver is among the first pieces of software that runs in your bootup or especially 'live' repair-boots. HTH, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From stephan.budach at JVM.DE Fri Dec 12 11:24:33 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Fri, 12 Dec 2014 12:24:33 +0100 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: References: <1418336750819.83182@steait.net> <1418337697672.85350@steait.net> Message-ID: <548AD071.5070306@jvm.de> Am 12.12.14 um 08:58 schrieb Jim Klimov: > 12 ??????? 2014 ?. 8:35:18 CET, Jim Klimov ?????: >> 11 ??????? 2014 ?. 23:41:35 CET, Rune Tipsmark ?????: >>> NIC's are not shared with IPMI. >>> >>> >>> >>> I will try the target-mode first, however on the 3rd box I actually >>> have Infiniband and it also caused the same issue without touching the >>> target-mode for FC. >>> >>> >>> >>> I have a 4th system on an old HP server and it reboots perfect every >>> time, no target-mode and has Infiniband as well... I am leaning >> towards >>> something with the SuperMicro hardware but can't really pinpoint it. >>> >>> br, >>> >>> Rune >>> >>> ________________________________ >>> From: Dan McDonald >>> Sent: Thursday, December 11, 2014 11:39 PM >>> To: Rune Tipsmark >>> Cc: omnios-discuss at lists.omniti.com >>> Subject: Re: [OmniOS-discuss] hangs on reboot >>> >>> Nothing printed out on the console? >>> >>> And try eliminating the target mode first - just in case. >>> >>> Also, is one of your NICs a dual IPMI/host NIC? If so, disable the >>> IPMI portion, we don't cope with shared NIC ports like that. >>> >>> Dan >>> >>> Sent from my iPhone (typos, autocorrect, and all) >>> >>> On Dec 11, 2014, at 5:25 PM, Rune Tipsmark >>> > wrote: >>> >>> >>> hi all, >>> >>> >>> >>> I got a bunch (3) installations of omnios on SuperMicro hardware and >>> all 3 have issues rebooting. They simply hang and never ever reboot. >>> >>> >>> >>> The install is latest version and I only added the storage-server >>> package and installed napp-it and changed the fibre channel setting in >>> /kernel/drv/emlxs.conf target-mode=1 >>> >>> >>> >>> two nics igb0 and igb1 configured as aggregation (aggr0) >>> >>> >>> >>> besides this its 100% default install, not 10 minutes old... >>> >>> >>> >>> Any ideas? >>> >>> br, >>> >>> Rune >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> I've installed latest bloody on an HP Z400 workstation last week with >> an lsi 9212 addon hba, and it too needed to disable fastboot. >> >> IIRC this is somehow tied to ability of all drivers to quiesce like for >> standby - if some one refuses to, suspend or fastboot fail. >> IMHO the fastboot handler should have a reasonable/configurable timeout >> after 'stand by for reboot' to do slowboot if that expires. >> >> Alternately on servers you can set up bmc-watchdog and have the mobo >> reboot itself. But then use long timeouts (10-15 min) and/or take care >> that the watchdog driver is among the first pieces of software that >> runs in your bootup or especially 'live' repair-boots. >> >> HTH, >> Jim >> -- >> Typos courtesy of K-9 Mail on my Samsung Android > Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also a signal for devs to improve at least nominal hw support for new usb3, video chips, maybe some sata HBAs? > Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or usb1, or let radeon be a vesa-video, but let it work without errors) -- they should not impede system work either. > > Jim > -- I have recently bought a new SuperMicro which showed exactly the same behaviour regarding fast reboots. Rebooting through the prom however, worked each time I can remember trying it. That system had been originally installed using the release 006 of OmniOS. After upgrading to 012 stable, these issues seem to have gone. The only thing that is different on the new host is, that it's equipped with a SSD DOM instead of a "traditional" HD,. This DOM is stuffed in SATA0 and somewhat cabled to the MB, so it's not just a dumb one. This cable seems to be a bit picky about its connection of the plug that goes into the DOM. As for 012? I just performed 4 fast reboots in a row without issues on my new SuperMicro box? Cheers, budy From jimklimov at cos.ru Fri Dec 12 12:57:37 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Fri, 12 Dec 2014 13:57:37 +0100 Subject: [OmniOS-discuss] Parity error on path mpt_sas2 In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local> References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local> <1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local> <004501d01302$257c9b90$7075d2b0$@subrigo.net> <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local> <3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local> Message-ID: 9 ??????? 2014??. 15:58:05 CET, Filip Marvan ?????: >Yes, you have to use bootable DOS and DOS version of sas2flash. > >There is an awesome tool for creating bootable USB stick with DOS >called Rufus > > > >http://rufus.akeo.ie/ > > > >Filip > > > > > >From: Schweiss, Chip [mailto:chip at innovates.com] >Sent: Tuesday, December 09, 2014 3:50 PM >To: Filip Marvan >Cc: Matthew Lagoe; omnios-discuss at lists.omniti.com >Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > >I'm still fighting this. > >Under OmniOS the erase function is disabled in sas2flash for solaris. > >I couldn't seem to find an iso bootable DOS that I could stage files >in. >FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode. Tried a >Linux >rescue CD and I get " ERROR: Erase Flash Operation Failed!" Tried >both P18 >sas2flash and P20 sas2flash. > >What OS environment are you running this in? > > > > > >On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan >wrote: > >Hi, > > > >first you have to erase P20 firmware with: > > > >sas2flsh -o -e 6 > > > >now do not reboot(!) and flash P18 with > > > >sas2flsh -f XXXXXX.bin -b mptsas2.rom > > > >Filip > > > > > >From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] >On >Behalf Of Schweiss, Chip >Sent: Monday, December 08, 2014 08:09 AM >To: Filip Marvan >Cc: omnios-discuss at lists.omniti.com >Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > >I've got some new LSI HBAs I'm trying to downgrade from firmware >version 20 to >18. > >I'm getting errors when trying to downgrade: > >Attempting to flash firmware to LSI SAS SAS2308_2(D1) : > > Executing Operation: Flash Firmware Image > > Firmware Image has a Valid Checksum. > Firmware Version 18.00.00.00 > Firmware Image compatible with Controller. > > Valid NVDATA Image found. > NVDATA Version 11.00.00.00 > Checking for a compatible NVData image... > > NVDATA Device ID and Chip Revision match verified. > ERROR: Cannot downgrade NVDATA version 14.00.00.06 > to 11.00.110000.00. > > ERROR: Failed to get valid NVDATA image from File! > > Firmware Image Validation Failed! > >Tried downgrading bios: > >Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) : > > Validating BIOS Image... > > BIOS Header Signature is Valid > > BIOS Image has a Valid Checksum. > > BIOS PCI Structure Signature Valid. > > BIOS Image Compatible with the SAS Controller. > > Attempting to Flash BIOS Image... > > BIOS Version in flash : 07.39.00.00 > BIOS Version from File : 07.35.00.00 > Skipping flash since file version is not greater than >existing. > > Flash BIOS Image Failed! > >I've tried sas2flash from P20 and P18. > >Can someone fill me in on the trick to downgrade? > >-Chip > > > > > >On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan >wrote: > >Thank you for your help Aaron! I downgraded firmware to P18 and there >are no >more errors today. > >It seems, that there is something bad with P20 firmware! > > > >You saved me a lot of time J > > > >Filip > > > > > >From: Aaron Curry [mailto:asc1111 at gmail.com] >Sent: Tuesday, November 25, 2014 5:46 PM >To: Filip Marvan >Cc: Dan McDonald; omnios-discuss at lists.omniti.com >Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2 > > > >I had this exact same problem recently when setting up a new home >server... >same controller, same firmware (P20). The errors were on all disks >attached to >the controller, but only on high read activity. Writes did not generate >the >errors. I downgraded the firmware to P18 (which is what we are using at >work) >and the errors went away. Has anyone had success with this controller >and the >P20 firmware? > > > >Aaron > > > >On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan >wrote: > >Hi Dan, > >thanks for reply. >Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on >other >disks in different pool but on the same SAS cable. >So maybe I only had bad luck with that disks. > >Filip > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Thanks all for the insightful thread. I found a usb image instead, with freedos and batch files prepared to reflash lsi - see http://joelinoff.com/blog/?p=588 HTH, Jim -- Typos courtesy of K-9 Mail on my Samsung Android From chip at innovates.com Fri Dec 12 14:33:55 2014 From: chip at innovates.com (Schweiss, Chip) Date: Fri, 12 Dec 2014 08:33:55 -0600 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <548AD071.5070306@jvm.de> References: <1418336750819.83182@steait.net> <1418337697672.85350@steait.net> <548AD071.5070306@jvm.de> Message-ID: There is one situation that svccfg changes are not turning off fast reboot. If the system has a panic the next boot will have an automated reboot that gets hung. I'm assuming this is the crash dump being collected then a reboot initiated. Anyone know how to fix this one or how to collect better information on where the fast reboot is coming from? -Chip On Fri, Dec 12, 2014 at 5:24 AM, Stephan Budach wrote: > > Am 12.12.14 um 08:58 schrieb Jim Klimov: > > 12 ??????? 2014 ?. 8:35:18 CET, Jim Klimov ?????: >> >>> 11 ??????? 2014 ?. 23:41:35 CET, Rune Tipsmark ?????: >>> >>>> NIC's are not shared with IPMI. >>>> >>>> >>>> >>>> I will try the target-mode first, however on the 3rd box I actually >>>> have Infiniband and it also caused the same issue without touching the >>>> target-mode for FC. >>>> >>>> >>>> >>>> I have a 4th system on an old HP server and it reboots perfect every >>>> time, no target-mode and has Infiniband as well... I am leaning >>>> >>> towards >>> >>>> something with the SuperMicro hardware but can't really pinpoint it. >>>> >>>> br, >>>> >>>> Rune >>>> >>>> ________________________________ >>>> From: Dan McDonald >>>> Sent: Thursday, December 11, 2014 11:39 PM >>>> To: Rune Tipsmark >>>> Cc: omnios-discuss at lists.omniti.com >>>> Subject: Re: [OmniOS-discuss] hangs on reboot >>>> >>>> Nothing printed out on the console? >>>> >>>> And try eliminating the target mode first - just in case. >>>> >>>> Also, is one of your NICs a dual IPMI/host NIC? If so, disable the >>>> IPMI portion, we don't cope with shared NIC ports like that. >>>> >>>> Dan >>>> >>>> Sent from my iPhone (typos, autocorrect, and all) >>>> >>>> On Dec 11, 2014, at 5:25 PM, Rune Tipsmark >>>> > wrote: >>>> >>>> >>>> hi all, >>>> >>>> >>>> >>>> I got a bunch (3) installations of omnios on SuperMicro hardware and >>>> all 3 have issues rebooting. They simply hang and never ever reboot. >>>> >>>> >>>> >>>> The install is latest version and I only added the storage-server >>>> package and installed napp-it and changed the fibre channel setting in >>>> /kernel/drv/emlxs.conf target-mode=1 >>>> >>>> >>>> >>>> two nics igb0 and igb1 configured as aggregation (aggr0) >>>> >>>> >>>> >>>> besides this its 100% default install, not 10 minutes old... >>>> >>>> >>>> >>>> Any ideas? >>>> >>>> br, >>>> >>>> Rune >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>>> >>>> ------------------------------------------------------------ >>>> ------------ >>>> >>>> _______________________________________________ >>>> OmniOS-discuss mailing list >>>> OmniOS-discuss at lists.omniti.com >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >>>> >>> I've installed latest bloody on an HP Z400 workstation last week with >>> an lsi 9212 addon hba, and it too needed to disable fastboot. >>> >>> IIRC this is somehow tied to ability of all drivers to quiesce like for >>> standby - if some one refuses to, suspend or fastboot fail. >>> IMHO the fastboot handler should have a reasonable/configurable timeout >>> after 'stand by for reboot' to do slowboot if that expires. >>> >>> Alternately on servers you can set up bmc-watchdog and have the mobo >>> reboot itself. But then use long timeouts (10-15 min) and/or take care >>> that the watchdog driver is among the first pieces of software that >>> runs in your bootup or especially 'live' repair-boots. >>> >>> HTH, >>> Jim >>> -- >>> Typos courtesy of K-9 Mail on my Samsung Android >>> >> Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also >> a signal for devs to improve at least nominal hw support for new usb3, >> video chips, maybe some sata HBAs? >> Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or >> usb1, or let radeon be a vesa-video, but let it work without errors) -- >> they should not impede system work either. >> >> Jim >> -- >> > I have recently bought a new SuperMicro which showed exactly the same > behaviour regarding fast reboots. Rebooting through the prom however, > worked each time I can remember trying it. That system had been originally > installed using the release 006 of OmniOS. After upgrading to 012 stable, > these issues seem to have gone. The only thing that is different on the new > host is, that it's equipped with a SSD DOM instead of a "traditional" HD,. > This DOM is stuffed in SATA0 and somewhat cabled to the MB, so it's not > just a dumb one. This cable seems to be a bit picky about its connection of > the plug that goes into the DOM. > > As for 012? I just performed 4 fast reboots in a row without issues on my > new SuperMicro box? > > Cheers, > budy > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.sproul at circonus.com Fri Dec 12 15:08:14 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Fri, 12 Dec 2014 10:08:14 -0500 Subject: [OmniOS-discuss] Java paths in OmniOS In-Reply-To: References: Message-ID: On Fri, Dec 12, 2014 at 2:52 AM, Jim Klimov wrote: > The new structure all but forbids such customization; was there and particular benefit from changing to it? The Java that ships with OmniOS is only there to satisfy runtime dependencies for certain things that need it. You are almost always better off getting your own JDK/JRE and installing it however you wish. The Oracle Java downloads work just fine, and so does OpenJDK, though that's trickier to build yourself (but you can look at the omnios-build script for hints). Eric From zmalone at omniti.com Fri Dec 12 15:25:32 2014 From: zmalone at omniti.com (Zach Malone) Date: Fri, 12 Dec 2014 10:25:32 -0500 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: /opt/ha is locally built, it isn't delivered by OmniOS. I'm not sure why it would go missing on upgrade, how did you initially create it? --Zach On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos wrote: > Hi > > The command that I execute : > root#crm > abort: couldn't find crm libraries in [/opt/ha/sbin > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip /usr/lib/python2.6 > /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] > (check your install and PYTHONPATH) > > I have the following environment variables set: > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH > export OCF_ROOT=/opt/ha/lib/ocf > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat > > I have exactly the same in an r10 release (pre upgrade to rr12) where there > is no problem > > I did a truss -d crm and it seems that many files it searches for are not > found. Snippets of the output (very long file) hope this helps: > > Below (as I understand it ) some searches which it can resolve: > > 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023) > = 18 > 0.0100 resolvepath("/usr/bin/amd64/python2.6", > "/usr/bin/amd64/python2.6", 1023) = 24 > 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 > 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT > 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", > 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) = > 0 > 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 > 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 > 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0113 close(3) = 0 > 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 > 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) > Err#2 ENOENT > 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", > "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 > 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0127 close(3) = 0 > 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 > 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2 > ENOENT > 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", > "/lib/amd64/libnsl.so.1", 1023) = 22 > 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 > 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0139 close(3) = 0 > 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 > 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 > ENOENT > 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2", > 1023) = 20 > 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 > 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0150 close(3) = 0 > 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 > 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 > ENOENT > 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1", > 1023) = 20 > 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 > 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > 0.0162 close(3) = 0 > 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 > 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0, > 0) = 0 > 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0168 resolvepath("/lib/64/libsocket.so.1", > "/lib/amd64/libsocket.so.1", 1023) = 25 > 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", > 1023) = 22 > 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = > 20 > 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) > Err#2 ENOENT > 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 > 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 > 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Below searches which it cannot resolve: > > 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 > 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 > 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22 > EINVAL > 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT > 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 > ENOENT > 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2 > ENOENT > 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 > 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT > 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) > Err#2 ENOENT > 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0 > 0.0351 brk(0x00485250) = 0 > > 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 > 0.0459 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) = > 0 > 0.0461 stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) = > 0 > 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY) > Err#2 ENOENT > 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", > O_RDONLY) Err#2 ENOENT > 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) > Err#2 ENOENT > 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) > Err#2 ENOENT > 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT > 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 > 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT > 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 > 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 > 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT > 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 ENOENT > 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 > ENOENT > 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 > 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 > 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 > 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 > 0.0494 brk(0x004D5250) = 0 > 0.0496 brk(0x004D9250) = 0 > 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 > 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY > 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 > > 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD320) > Err#2 ENOENT > 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) > Err#2 ENOENT > 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", > O_RDONLY) Err#2 ENOENT > 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) Err#2 > ENOENT > 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) > Err#2 ENOENT > 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT > 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT > 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT > 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 > 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 > 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 > 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 > 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 > 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY > 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 > > 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC000) Err#2 ENOENT > 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", > O_RDONLY) Err#2 ENOENT > 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0575 open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", > O_RDONLY) Err#2 ENOENT > 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", > O_RDONLY) Err#2 ENOENT > 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", > O_RDONLY) Err#2 ENOENT > 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 > ENOENT > 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT > 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2 > ENOENT > 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > ENOENT > 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 > 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 > 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 > 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 > > > And the list goes on. > > Hope there's a solution for this. > > > Joe > > > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald wrote: >> >> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos wrote: >> > >> > Hi, >> > >> > I have an application working on Omnios r10 which depends on certain >> > python libraries. There are certain environment variables in place which >> > point to the location of those libraries. >> > >> > I have updated the r10 machine to r12. The application now cannot be >> > started with errors stating that the needed libraries cannot be found in the >> > given paths. >> >> Share the errors please? I'll need more details. >> >> > Maybe somebody can tell me what the difference is in python version >> > between the two omnios releases. Python -V gives me 2.6.8. on both versions. >> >> We updated supplemental python libraries, which may contribute to what >> you're seeing. Also, we had not updated the "entire" metapackage on 010 to >> show we were actually running 2.6.8. >> >> Dan >> > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From info at houseofancients.nl Fri Dec 12 15:24:34 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 15:24:34 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> Hi All, this just keeps getting stranger by the minute. after removing the aggr's and setting a static ip on each interface, i can ping from the host itself, but not through a wire, direct or through a switch. plugged in a new intel i350, same deal... when looking at the driver, i see that the is a update date of november 6.... does anybody have an idea what could be going on ? Met vriendelijke groet / With kind regards, Floris van Essen Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::.. Verzonden: dinsdag 9 december 2014 0:15 Aan: omnios-discuss at lists.omniti.com Onderwerp: [OmniOS-discuss] LACP Omnios , igb and C3750 Guys, been having a odd issue with my Omnios box and c3750 the omnios box has 6 interfaces ( 2 e1000g and 4 igb) somehow the LACP bonding between the igb's and my cisco's will not stay in sync, while the same lacp config runs fine over the e1000's : channel 5 = e1000g's channel 6 and 7 are the igb's : Channel group 5 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi1/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x2 0x3F Gi2/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x1 0x3F Channel group 6 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi1/0/10 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x6 0x47 Gi1/0/12 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x5 0x47 Channel group 7 neighbors Partner's information: LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Gi2/0/10 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x4 0x47 Gi2/0/11 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x3 0x47 LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS aggr0 L4 auto active short ----- aggr1 L4 auto active short ----- aggr2 L4 auto active short ----- aggr0 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 e1000g0 attached up 1000Mb full - - - x:x:x:x:x:x 1500 e1000g1 attached up 1000Mb full - - - x:x:x:x:x:x 1500 aggr1 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 igb0 attached up 1000Mb full - - - x:x:x:x:x:x 1500 igb1 attached up 1000Mb full - - - x:x:x:x:x:x 1500 aggr2 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500 igb2 attached up 1000Mb full - - - x:x:x:x:x:x 1500 igb3 attached up 1000Mb full - - - x:x:x:x:x:x 1500 058673: Dec 8 22:15:51.593: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 1(selected) 058674: Dec 8 22:15:51.593: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING 058675: Dec 8 22:15:51.593: LACP: Gi2/0/10 lacp_action_mx_waiting entered 058676: Dec 8 22:15:51.593: LACP: timer lacp_w(Gi2/0/10) started with interval 2000. 058677: Dec 8 22:15:51.895: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/10 058678: Dec 8 22:15:51.895: LACP : packet size: 124 058679: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1 058680: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058681: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA, s-pri:0x0, s-mac:0000.0000.0000 058682: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA 058683: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0 058684: Dec 8 22:15:51.895: LACP: Gi2/0/10 LACP packet received, processing 058685: Dec 8 22:15:51.895: lacp_rx Gi2/0/10 - rx: during state CURRENT, got event 5(recv_lacpdu) 058686: Dec 8 22:15:51.895: @@@ lacp_rx Gi2/0/10 - rx: CURRENT -> CURRENT 058687: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_rx_current entered 058688: Dec 8 22:15:51.895: LACP: Gi2/0/10 set to UNSELECTED 058689: Dec 8 22:15:51.895: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 3(unselected) 058690: Dec 8 22:15:51.895: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> DETACHED 058691: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_mx_detached entered 058692: Dec 8 22:15:51.895: LACP: Gi2/0/10 Resetting hw_sw constraints 058693: Dec 8 22:15:51.895: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag. 058694: Dec 8 22:15:51.895: LACP: lacp_send_lacpdu: (Gi2/0/10) About to send the 110 LACPDU 058695: Dec 8 22:15:51.895: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/10 058696: Dec 8 22:15:51.895: LACP : packet size: 124 058697: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1 058698: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20B, p-state:0x5, s-pri:0x8000, s-mac:0019.aa89.b580 058699: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058700: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000 058701: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0 058702: Dec 8 22:15:51.895: LACP: lacp_write: LACP 124 bytes out Gi2/0/10 058703: Dec 8 22:15:51.895: lacp_handle_standby_port_internal called, depth = 1 058704: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 LACP PDU Rcvd. Partners oper state is hex 47 058705: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 Partner out of sync 058706: Dec 8 22:15:51.895: LACP: Gi2/0/10 Partners oper state is hex 47 058707: Dec 8 22:15:51.895: LACP: timer lacp_c_l(Gi2/0/10) started with interval 90000. 058708: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG_PARTNER_UP. 058709: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG unchanged 058710: Dec 8 22:15:51.903: LACP: Gi2/0/10 STANDBY aggregator hex address is 5FB71A4 058711: Dec 8 22:15:51.903: LACP: Gi2/0/10 set to STANDBY 058712: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state DETACHED, got event 2(standby) 058713: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: DETACHED -> WAITING 058714: Dec 8 22:15:51.903: LACP: Gi2/0/10 lacp_action_mx_waiting entered 058715: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING 058716: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 6(outof_sync) (ignored) 058717: Dec 8 22:15:51.903: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/11 058718: Dec 8 22:15:51.903: LACP : packet size: 124 058719: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1 058720: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 058721: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA, s-pri:0x0, s-mac:0000.0000.0000 058722: Dec 8 22:15:51.903: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA 058723: Dec 8 22:15:51.903: LACP: term-tlv:0 termr-tlv-len:0 058724: Dec 8 22:15:51.903: LACP: Gi2/0/11 LACP packet received, processing 058725: Dec 8 22:15:51.903: lacp_rx Gi2/0/11 - rx: during state CURRENT, got event 5(recv_lacpdu) 058726: Dec 8 22:15:51.903: @@@ lacp_rx Gi2/0/11 - rx: CURRENT -> CURRENT 058727: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_rx_current entered 058728: Dec 8 22:15:51.903: LACP: Gi2/0/11 set to UNSELECTED 058729: Dec 8 22:15:51.903: lacp_mux Gi2/0/11 - mux: during state WAITING, got event 3(unselected) 058730: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/11 - mux: WAITING -> DETACHED 058731: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_mx_detached entered 058732: Dec 8 22:15:51.903: LACP: Gi2/0/11 Resetting hw_sw constraints 058733: Dec 8 22:15:51.903: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag. 058734: Dec 8 22:15:51.903: LACP: lacp_send_lacpdu: (Gi2/0/11) About to send the 110 LACPDU 058735: Dec 8 22:15:51.903: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/11 058736: Dec 8 22:15:51.903: LACP : packet size: 124 058737: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1 058738: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20C, p-state:0x5, s-pri:0x8000, s-mac:0019.aa89.b580 058739: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47, s-pri:0x1000, s-mac:a036.9f02.c138 so in short, i see it coming up, shows out of sync, and down it goes... have tried short times, long times active , passive... nothing will work Met vriendelijke groet / With kind regards, Floris van Essen / Claudia Dufornee House of Ancients Amstaffs info at houseofancients.nl www.houseofancients.nl ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Fri Dec 12 16:35:25 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 11:35:25 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> Message-ID: Sorry for not replying earlier. > On Dec 12, 2014, at 10:24 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi All, > > this just keeps getting stranger by the minute. > after removing the aggr's and setting a static ip on each interface, > i can ping from the host itself, but not through a wire, direct or through a switch. > plugged in a new intel i350, same deal... > when looking at the driver, i see that the is a update date of november 6.... > > does anybody have an idea what could be going on ? You were smart to reduce it to individual igb links first. Please remind me what revision of OmniOS you have? Also, please share the output of: - dladm show-ether - dladm show-link - ifconfig -a - netstat -rnv Thanks, Dan From info at houseofancients.nl Fri Dec 12 16:49:26 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 16:49:26 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> Hi Dann, No problem, just happy you did :-) Right, so as might remember I had to reinstall because I was running bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... So here we are again, running bloody r13 , fully updated :-) # dladm show-ether LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE e1000g1 current up yes 1G-f bi e1000g0 current up yes 1G-f bi igb0 current up yes 1G-f bi igb1 current up yes 1G-f bi igb2 current up yes 1G-f bi igb3 current up yes 1G-f bi # dladm show-link LINK CLASS MTU STATE BRIDGE OVER e1000g1 phys 1500 up -- -- e1000g0 phys 1500 up -- -- aggr0 aggr 1500 up -- e1000g0 e1000g1 igb0 phys 1500 up -- -- igb1 phys 1500 up -- -- igb2 phys 1500 up -- -- igb3 phys 1500 up -- -- # ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 aggr0: flags=1000843 mtu 1500 index 2 inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 ether 0:30:48:d5:ec:94 igb0: flags=1000843 mtu 1500 index 3 inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 ether a0:36:9f:2:c2:6c igb1: flags=1000843 mtu 1500 index 4 inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 ether a0:36:9f:2:c2:6d igb2: flags=1000843 mtu 1500 index 5 inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 ether a0:36:9f:2:c2:6e igb3: flags=1000843 mtu 1500 index 6 inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 ether a0:36:9f:2:c2:6f lo0: flags=2002000849 mtu 8252 index 1 inet6 ::1/128 aggr0: flags=20002000840 mtu 1500 index 2 inet6 ::/0 ether 0:30:48:d5:ec:94 igb0: flags=20002000840 mtu 1500 index 3 inet6 ::/0 ether a0:36:9f:2:c2:6c igb1: flags=20002000840 mtu 1500 index 4 inet6 ::/0 ether a0:36:9f:2:c2:6d igb2: flags=20002000840 mtu 1500 index 5 inet6 ::/0 ether a0:36:9f:2:c2:6e igb3: flags=20002000840 mtu 1500 index 6 inet6 ::/0 ether a0:36:9f:2:c2:6f # netstat -rnv IRE Table: IPv4 Destination Mask Gateway Device MTU Ref Flg Out In/Fwd -------------------- --------------- -------------------- ------ ----- --- --- ----- ------ default 0.0.0.0 x.x.x.254 0 3 UG 68263 0 5.135.127.96 255.255.255.255 x.x.x.1 0 2 UHD 2 0 37.187.62.87 255.255.255.255 x.x.x.1 0 2 UHD 79 0 46.19.141.134 255.255.255.255 x.x.x.1 0 2 UHD 210 0 46.105.201.50 255.255.255.255 x.x.x.1 0 2 UHD 35 0 77.250.233.134 255.255.255.255 x.x.x.1 0 2 UHD 53 0 127.0.0.1 255.255.255.255 127.0.0.1 lo0 8232 2 UH 9921 9921 130.233.43.4 255.255.255.255 x.x.x.1 0 2 UHD 2 0 192.168.0.0 255.255.255.0 192.168.0.4 igb3 1500 3 U 18 0 192.168.0.0 255.255.255.0 192.168.0.3 igb2 1500 2 U 0 0 192.168.0.0 255.255.255.0 192.168.0.2 igb1 1500 2 U 0 0 192.168.0.0 255.255.255.0 192.168.0.1 igb0 1500 2 U 0 0 x.x.x.0 255.255.255.0 x.x.x.11 aggr0 1500 6 U 680 0 199.15.226.221 255.255.255.255 x.x.x.1 0 2 UHD 8 0 IRE Table: IPv6 Destination/Mask Gateway If MTU Ref Flags Out In/Fwd --------------------------- --------------------------- ----- ----- --- ----- ------ ------ ::1 ::1 lo0 8252 2 UH 1118 1118 # Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Dan McDonald [mailto:danmcd at omniti.com] Verzonden: vrijdag 12 december 2014 17:35 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 Sorry for not replying earlier. > On Dec 12, 2014, at 10:24 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi All, > > this just keeps getting stranger by the minute. > after removing the aggr's and setting a static ip on each interface, i > can ping from the host itself, but not through a wire, direct or through a switch. > plugged in a new intel i350, same deal... > when looking at the driver, i see that the is a update date of november 6.... > > does anybody have an idea what could be going on ? You were smart to reduce it to individual igb links first. Please remind me what revision of OmniOS you have? Also, please share the output of: - dladm show-ether - dladm show-link - ifconfig -a - netstat -rnv Thanks, Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Fri Dec 12 16:54:27 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 11:54:27 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> Message-ID: > On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dann, > > No problem, just happy you did :-) > > Right, so as might remember I had to reinstall because I was running bloody R11 , but there was a problem upgrading , so just did a fresh install > Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. > So here we are again, running bloody r13 , fully updated :-) > > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current up yes 1G-f bi > igb1 current up yes 1G-f bi > igb2 current up yes 1G-f bi > igb3 current up yes 1G-f bi > # dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 up -- -- > igb1 phys 1500 up -- -- > igb2 phys 1500 up -- -- > igb3 phys 1500 up -- -- Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > # ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 > ether 0:30:48:d5:ec:94 And you have no problems pinging x.x.x.0/24 addresses? Or do you? > igb0: flags=1000843 mtu 1500 index 3 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6c > igb1: flags=1000843 mtu 1500 index 4 > inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6d > igb2: flags=1000843 mtu 1500 index 5 > inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6e > igb3: flags=1000843 mtu 1500 index 6 > inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out another. If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. Dan From info at houseofancients.nl Fri Dec 12 17:10:36 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 17:10:36 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> > On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dann, > > No problem, just happy you did :-) > > Right, so as might remember I had to reinstall because I was running > bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... >> Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. Did the installation 2 weeks ago, so guessing that that was before you did the changes. No problem, I'll update to stable next release :-) > So here we are again, running bloody r13 , fully updated :-) > > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current up yes 1G-f bi > igb1 current up yes 1G-f bi > igb2 current up yes 1G-f bi > igb3 current up yes 1G-f bi > # dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 up -- -- > igb1 phys 1500 up -- -- > igb2 phys 1500 up -- -- > igb3 phys 1500 up -- -- >>Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > # ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 > ether 0:30:48:d5:ec:94 And you have no problems pinging x.x.x.0/24 addresses? Or do you? Non, actually connecting on that interface with ssh > igb0: flags=1000843 mtu 1500 index 3 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6c > igb1: flags=1000843 mtu 1500 index 4 > inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6d > igb2: flags=1000843 mtu 1500 index 5 > inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6e > igb3: flags=1000843 mtu 1500 index 6 > inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f >>Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another. >>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. That is exactly what i did.. for each test brought the others down Just to add, when doing a SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news >>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100% Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Fri Dec 12 17:21:24 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 12:21:24 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> Message-ID: > >>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. > > That is exactly what i did.. for each test brought the others down > Just to add, when doing a SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news And you've eliminated the switch as a problem? I ask because of this recent push (don't let the commit date fool you, this went in in November): commit b607c8a3bdb27d4fde6e3fc4bb6617a1d91bdca7 Author: Keith M Wesolowski Date: Tue Aug 5 22:08:11 2014 +0000 5337 igb/ixgbe mishandle raw packets if cable problem Reviewed by: Robert Mustacchi Reviewed by: Dan McDonald Reviewed by: Jason King Reviewed by: Garrett D'Amore Approved by: Richard Lowe usr/src/uts/common/io/igb/igb_tx.c | 10 +++++++--- usr/src/uts/common/io/ixgbe/ixgbe_tx.c | 6 ++++-- 2 files changed, 11 insertions(+), 5 deletions(-) This is the ONLY change since the February of 2014 support for I354 that's gone into the Intel GigE NIC code. >>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. > > They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. > To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100% The only difference between 011 & 012 and 013 is the above commit. You *could*, if you're feeling adventurous, swap out the "igb" driver in 013 with one from 012 or 011. And if you unplumb all of the igbs (both in IPv4 and IPv6), you MAY be able to do it without rebooting. Do you know which PCI ID your igb (it's an I350, right?) board is? "prtconf -d | grep -i network" should be a quick way to show me that. Dan From info at houseofancients.nl Fri Dec 12 17:27:42 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 17:27:42 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> > >>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. > > That is exactly what i did.. for each test brought the others down > Just to add, when doing a SH ARP on my cisco's, don't see the macs > popping up, which is even more worring , no arps on a switch usually > is very bad news And you've eliminated the switch as a problem? I ask because of this recent push (don't let the commit date fool you, this went in in November): Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect. Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-) commit b607c8a3bdb27d4fde6e3fc4bb6617a1d91bdca7 Author: Keith M Wesolowski Date: Tue Aug 5 22:08:11 2014 +0000 5337 igb/ixgbe mishandle raw packets if cable problem Reviewed by: Robert Mustacchi Reviewed by: Dan McDonald Reviewed by: Jason King Reviewed by: Garrett D'Amore Approved by: Richard Lowe usr/src/uts/common/io/igb/igb_tx.c | 10 +++++++--- usr/src/uts/common/io/ixgbe/ixgbe_tx.c | 6 ++++-- 2 files changed, 11 insertions(+), 5 deletions(-) This is the ONLY change since the February of 2014 support for I354 that's gone into the Intel GigE NIC code. >>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. > > They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. > To make it more interesting.. this was the same setup as I had on r11 > , and then it did seem to work 100% The only difference between 011 & 012 and 013 is the above commit. You *could*, if you're feeling adventurous, swap out the "igb" driver in 013 with one from 012 or 011. And if you unplumb all of the igbs (both in IPv4 and IPv6), you MAY be able to do it without rebooting. As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue. Just not sure as HOW to change that driver, so would be great if you'd have some pointers Do you know which PCI ID your igb (it's an I350, right?) board is? "prtconf -d | grep -i network" should be a quick way to show me that. pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #0 pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #1 pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #2 pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #3 floris ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Fri Dec 12 17:32:04 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 12:32:04 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> Message-ID: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> > > And you've eliminated the switch as a problem? I ask because of this recent push (don't let the commit date fool you, this went in in November): > > Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect. > Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-) Okay, if the e1000gs work, then fine. I'm assuming you've checked out the cables as well... > > As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue. > Just not sure as HOW to change that driver, so would be great if you'd have some pointers You're running bloody, so you can do these normally not-recommended-for-users steps: 1.) Backup the original igb: cp /kernel/drv/amd64/igb 2.) Find an igb from 011 or 012. Exercise left to the reader. If you have an old boot-environment, "beadm mount" it and grab it from there. 3.) Replace /kernel/drv/amd64/igb with the older version. 4.) Utter "bootadm update-archive" to make sure the replacement igb is there. 5.) Reboot. Dan From info at houseofancients.nl Fri Dec 12 17:36:02 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 17:36:02 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> Message-ID: <356582D1FC91784992ABB4265A16ED487EF46A07@vEX01.mindstorm-internet.local> > > And you've eliminated the switch as a problem? I ask because of this recent push (don't let the commit date fool you, this went in in November): > > Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect. > Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-) Okay, if the e1000gs work, then fine. I'm assuming you've checked out the cables as well... New cables, cables tested with e1000 also perfect > > As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue. > Just not sure as HOW to change that driver, so would be great if you'd have some pointers You're running bloody, so you can do these normally not-recommended-for-users steps: 1.) Backup the original igb: cp /kernel/drv/amd64/igb 2.) Find an igb from 011 or 012. Exercise left to the reader. If you have an old boot-environment, "beadm mount" it and grab it from there. As i had to do a reinstall , there's no r11 stuff left. R12 was never installed... Is there a way to download this somewere from github or maybe some source ? 3.) Replace /kernel/drv/amd64/igb with the older version. 4.) Utter "bootadm update-archive" to make sure the replacement igb is there. 5.) Reboot. Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From info at houseofancients.nl Fri Dec 12 19:05:23 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 19:05:23 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> Message-ID: <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ? Would like the igb file, for testing Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Dan McDonald [mailto:danmcd at omniti.com] Verzonden: vrijdag 12 december 2014 18:32 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > > And you've eliminated the switch as a problem? I ask because of this recent push (don't let the commit date fool you, this went in in November): > > Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect. > Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-) Okay, if the e1000gs work, then fine. I'm assuming you've checked out the cables as well... > > As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue. > Just not sure as HOW to change that driver, so would be great if you'd have some pointers You're running bloody, so you can do these normally not-recommended-for-users steps: 1.) Backup the original igb: cp /kernel/drv/amd64/igb 2.) Find an igb from 011 or 012. Exercise left to the reader. If you have an old boot-environment, "beadm mount" it and grab it from there. 3.) Replace /kernel/drv/amd64/igb with the older version. 4.) Utter "bootadm update-archive" to make sure the replacement igb is there. 5.) Reboot. Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From eric.sproul at circonus.com Fri Dec 12 19:15:18 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Fri, 12 Dec 2014 14:15:18 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> Message-ID: On Fri, Dec 12, 2014 at 2:05 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ? > Would like the igb file, for testing Hi Floris, You can fetch the file directly from the pkg server, without having any additional publisher URLs configured: $ pkg contents -mr -g http://pkg.omniti.com/omnios/r151012/ driver/network/igb | grep kernel/drv/amd64/igb file a4f79660b1593166be621166e8e1afc9b45af993 chash=c95cbda5496b6643ae331e0a1630eb74b55200c4 elfarch=i386 elfbits=64 elfhash=c10bc6c62e118456093689921d7794ae60684ae1 group=sys mode=0755 owner=root path=kernel/drv/amd64/igb pkg.csize=117774 pkg.size=384592 reboot-needed=true variant.opensolaris.zone=global The second field of the result is a SHA1 hash of the file which is its index in the repo, so you can: curl http://pkg.omniti.com/omnios/r151012/file/1/a4f79660b1593166be621166e8e1afc9b45af993 | gzip -dc > igb.r151012.amd64 and verify: $ sha1sum igb.r151012.amd64 a4f79660b1593166be621166e8e1afc9b45af993 igb.r151012.amd64 Then copy that file into place as Dan described. See http://omnios.omniti.com/wiki.php/FetchIPSFilesWithoutPkg for details. Eric From danmcd at omniti.com Fri Dec 12 19:23:00 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 14:23:00 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> Message-ID: <827529EA-5494-45FD-9181-B54B3553C6B4@omniti.com> I'd sent him an igb binary from 012 offline. Floris --> if you didn't get it, maybe your mail server filtered me out?! Dan From info at houseofancients.nl Fri Dec 12 20:06:40 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 20:06:40 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> Right, Wish i had better news, copied the file i got from Dan, create boot archive and rebooted Disable on the switch the ports not needed for first test Checked the boot log to see if it didn?t show funny stuff, of course I saw the e1000g's and their aggr passing correctly : Dec 12 20:40:43 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1096 (e1000g) instance 0 irq 0x33 vector 0x60 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: e1000g0 registered Dec 12 20:40:43 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1096 (e1000g) instance 1 irq 0x34 vector 0x61 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: e1000g1 registered Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: aggr1000 registered Then the IGB's : Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x37 vector 0x62 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x38 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb3 registered Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb3: igb 2.3.8-ish Seems 100% ok so After that removed if's and recreated just one of them (IGB3) : ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4 let's check : # dladm show-ether LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE e1000g1 current up yes 1G-f bi e1000g0 current up yes 1G-f bi igb0 current down yes 0M-h bi igb1 current unknown yes 0M-h bi igb2 current unknown yes 0M-h bi igb3 current up yes 1G-f bi awesome root at PSD01:/root# dladm show-link LINK CLASS MTU STATE BRIDGE OVER e1000g1 phys 1500 up -- -- e1000g0 phys 1500 up -- -- aggr0 aggr 1500 up -- e1000g0 e1000g1 igb0 phys 1500 down -- -- igb1 phys 1500 unknown -- -- igb2 phys 1500 unknown -- -- igb3 phys 1500 up -- -- awesome root at PSD01:/root# ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 aggr0: flags=1000843 mtu 1500 index 2 inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255 ether 0:30:48:d5:ec:94 igb3: flags=1000843 mtu 1500 index 10 inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 ether a0:36:9f:2:c2:6f lo0: flags=2002000849 mtu 8252 index 1 inet6 ::1/128 aggr0: flags=20002000840 mtu 1500 index 2 inet6 ::/0 ether 0:30:48:d5:ec:94 igb3: flags=20002000840 mtu 1500 index 10 inet6 ::/0 ether a0:36:9f:2:c2:6f awesome root at PSD01:/root# ping 192.168.0.1 192.168.0.1 is alive core-stack#sh arp | inc a036.9f:2.c26f core-stack# oh oh let's see if we can ping another host in the same subnet : root at PSD01:/root# ping 192.168.0.5 no answer from 192.168.0.5 root at PSD01:/root# let check the arp again : core-stack#sh arp | inc a036.9f:2.c26f core-stack# so I'm (again ) baffled Met vriendelijke groet -----Oorspronkelijk bericht----- Van: Eric Sproul [mailto:eric.sproul at circonus.com] Verzonden: vrijdag 12 december 2014 20:15 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: Dan McDonald; omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 On Fri, Dec 12, 2014 at 2:05 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ? > Would like the igb file, for testing Hi Floris, You can fetch the file directly from the pkg server, without having any additional publisher URLs configured: $ pkg contents -mr -g http://pkg.omniti.com/omnios/r151012/ driver/network/igb | grep kernel/drv/amd64/igb file a4f79660b1593166be621166e8e1afc9b45af993 chash=c95cbda5496b6643ae331e0a1630eb74b55200c4 elfarch=i386 elfbits=64 elfhash=c10bc6c62e118456093689921d7794ae60684ae1 group=sys mode=0755 owner=root path=kernel/drv/amd64/igb pkg.csize=117774 pkg.size=384592 reboot-needed=true variant.opensolaris.zone=global The second field of the result is a SHA1 hash of the file which is its index in the repo, so you can: curl http://pkg.omniti.com/omnios/r151012/file/1/a4f79660b1593166be621166e8e1afc9b45af993 | gzip -dc > igb.r151012.amd64 and verify: $ sha1sum igb.r151012.amd64 a4f79660b1593166be621166e8e1afc9b45af993 igb.r151012.amd64 Then copy that file into place as Dan described. See http://omnios.omniti.com/wiki.php/FetchIPSFilesWithoutPkg for details. Eric ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Fri Dec 12 20:10:21 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 15:10:21 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> Message-ID: <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com> > On Dec 12, 2014, at 3:06 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4 > > let's check : > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current down yes 0M-h bi > igb1 current unknown yes 0M-h bi > igb2 current unknown yes 0M-h bi > igb3 current up yes 1G-f bi > > awesome > > root at PSD01:/root# dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 down -- -- > igb1 phys 1500 unknown -- -- > igb2 phys 1500 unknown -- -- > igb3 phys 1500 up -- -- > > awesome > > root at PSD01:/root# ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255 > ether 0:30:48:d5:ec:94 > igb3: flags=1000843 mtu 1500 index 10 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f > lo0: flags=2002000849 mtu 8252 index 1 > inet6 ::1/128 > aggr0: flags=20002000840 mtu 1500 index 2 > inet6 ::/0 > ether 0:30:48:d5:ec:94 > igb3: flags=20002000840 mtu 1500 index 10 > inet6 ::/0 > ether a0:36:9f:2:c2:6f > > awesome > > root at PSD01:/root# ping 192.168.0.1 > 192.168.0.1 is alive NOTE: Pinging the local address doesn't prove anything about a NIC. > core-stack#sh arp | inc a036.9f:2.c26f > core-stack# > oh oh > > let's see if we can ping another host in the same subnet : > > root at PSD01:/root# ping 192.168.0.5 > no answer from 192.168.0.5 > root at PSD01:/root# > > let check the arp again : > > core-stack#sh arp | inc a036.9f:2.c26f > core-stack# > > so I'm (again ) baffled I'm actually relieved. I think your card is broken. The old driver doesn't improve your situation from what was previously happening. See if putting that card in another machine helps. Or perhaps just re-seating it. Dan From info at houseofancients.nl Fri Dec 12 20:36:16 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 20:36:16 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com> Message-ID: <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local> Hi Dan, So.. Pulled a working i350 ( connected to the same switch) from one of my vmware hosts. This would be the 3rd broke i350 Plugged it in, went into BIOS, reset config data Still no joy no ping, not arp My last option would be an HP NC quad nic, but I don't think that is supported by omnios Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Dan McDonald [mailto:danmcd at omniti.com] Verzonden: vrijdag 12 december 2014 21:10 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: Eric Sproul; omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > On Dec 12, 2014, at 3:06 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4 > > let's check : > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current down yes 0M-h bi > igb1 current unknown yes 0M-h bi > igb2 current unknown yes 0M-h bi > igb3 current up yes 1G-f bi > > awesome > > root at PSD01:/root# dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 down -- -- > igb1 phys 1500 unknown -- -- > igb2 phys 1500 unknown -- -- > igb3 phys 1500 up -- -- > > awesome > > root at PSD01:/root# ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255 > ether 0:30:48:d5:ec:94 > igb3: flags=1000843 mtu 1500 index 10 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f > lo0: flags=2002000849 mtu 8252 index 1 > inet6 ::1/128 > aggr0: flags=20002000840 mtu 1500 index 2 > inet6 ::/0 > ether 0:30:48:d5:ec:94 > igb3: flags=20002000840 mtu 1500 index 10 > inet6 ::/0 > ether a0:36:9f:2:c2:6f > > awesome > > root at PSD01:/root# ping 192.168.0.1 > 192.168.0.1 is alive NOTE: Pinging the local address doesn't prove anything about a NIC. > core-stack#sh arp | inc a036.9f:2.c26f core-stack# oh oh > > let's see if we can ping another host in the same subnet : > > root at PSD01:/root# ping 192.168.0.5 > no answer from 192.168.0.5 > root at PSD01:/root# > > let check the arp again : > > core-stack#sh arp | inc a036.9f:2.c26f core-stack# > > so I'm (again ) baffled I'm actually relieved. I think your card is broken. The old driver doesn't improve your situation from what was previously happening. See if putting that card in another machine helps. Or perhaps just re-seating it. Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Fri Dec 12 20:50:15 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 12 Dec 2014 15:50:15 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local> Message-ID: <25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com> > On Dec 12, 2014, at 3:36 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dan, > > So.. > Pulled a working i350 ( connected to the same switch) from one of my vmware hosts. > This would be the 3rd broke i350 > > Plugged it in, went into BIOS, reset config data > > Still no joy no ping, not arp grep igb /var/adm/messages --> any complaints from the kernel? Dan From info at houseofancients.nl Fri Dec 12 21:03:36 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Fri, 12 Dec 2014 21:03:36 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local> <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local> <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com> <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local> <25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com> Message-ID: <356582D1FC91784992ABB4265A16ED487EF47C4C@vEX01.mindstorm-internet.local> Looks good me : Dec 12 22:00:17 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x35 vector 0x62 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 22:00:17 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x36 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 22:00:17 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb3 registered Dec 12 22:00:17 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb3: igb 2.3.8-ish Dec 12 22:00:21 PSD01 genunix: [ID 435574 kern.info] NOTICE: igb3 link up, 1000 Mbps, full duplex Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb0 unregistered Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb1 unregistered Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb2 unregistered Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 1 Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 2 Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 3 Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 0 Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Dan McDonald [mailto:danmcd at omniti.com] Verzonden: vrijdag 12 december 2014 21:50 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: Eric Sproul; omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > On Dec 12, 2014, at 3:36 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dan, > > So.. > Pulled a working i350 ( connected to the same switch) from one of my vmware hosts. > This would be the 3rd broke i350 > > Plugged it in, went into BIOS, reset config data > > Still no joy no ping, not arp grep igb /var/adm/messages --> any complaints from the kernel? Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From info at houseofancients.nl Sat Dec 13 10:37:34 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Sat, 13 Dec 2014 10:37:34 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> So, To add to the weirdness. Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange : Sh arp Internet x.x.x.104 52 a036.9f17.abc0 ARPA Vlanx Wait, that is the mac of my IGB interface... did I miss something ? ifconfig igb0: flags=1004843 mtu 1500 index 13 inet 0.0.0.0 netmask ff000000 ether a0:36:9f:17:ab:c0 so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box... This means hardware, and drivers are ok! then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0 I think this is something within Omnios , however, would like some pointers where to look Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::.. Verzonden: vrijdag 12 december 2014 18:11 Aan: Dan McDonald CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dann, > > No problem, just happy you did :-) > > Right, so as might remember I had to reinstall because I was running > bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... >> Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. Did the installation 2 weeks ago, so guessing that that was before you did the changes. No problem, I'll update to stable next release :-) > So here we are again, running bloody r13 , fully updated :-) > > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current up yes 1G-f bi > igb1 current up yes 1G-f bi > igb2 current up yes 1G-f bi > igb3 current up yes 1G-f bi > # dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 up -- -- > igb1 phys 1500 up -- -- > igb2 phys 1500 up -- -- > igb3 phys 1500 up -- -- >>Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > # ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 > ether 0:30:48:d5:ec:94 And you have no problems pinging x.x.x.0/24 addresses? Or do you? Non, actually connecting on that interface with ssh > igb0: flags=1000843 mtu 1500 index 3 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6c > igb1: flags=1000843 mtu 1500 index 4 > inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6d > igb2: flags=1000843 mtu 1500 index 5 > inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6e > igb3: flags=1000843 mtu 1500 index 6 > inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f >>Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another. >>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. That is exactly what i did.. for each test brought the others down Just to add, when doing a SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news >>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100% Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From joeveliscos at gmail.com Sat Dec 13 13:50:48 2014 From: joeveliscos at gmail.com (Joe Veliscos) Date: Sat, 13 Dec 2014 14:50:48 +0100 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: /opt/ha was created by the HA install package. It is not missing after upgrade but many python files on the other hand, are removed from the system by the upgrade from r10 to r12. On Fri, Dec 12, 2014 at 4:25 PM, Zach Malone wrote: > > /opt/ha is locally built, it isn't delivered by OmniOS. I'm not sure > why it would go missing on upgrade, how did you initially create it? > --Zach > > On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos > wrote: > > Hi > > > > The command that I execute : > > root#crm > > abort: couldn't find crm libraries in [/opt/ha/sbin > > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg > > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip > /usr/lib/python2.6 > > /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk > > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload > > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] > > (check your install and PYTHONPATH) > > > > I have the following environment variables set: > > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages > > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH > > export OCF_ROOT=/opt/ha/lib/ocf > > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat > > > > I have exactly the same in an r10 release (pre upgrade to rr12) where > there > > is no problem > > > > I did a truss -d crm and it seems that many files it searches for are not > > found. Snippets of the output (very long file) hope this helps: > > > > Below (as I understand it ) some searches which it can resolve: > > > > 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", > 1023) > > = 18 > > 0.0100 resolvepath("/usr/bin/amd64/python2.6", > > "/usr/bin/amd64/python2.6", 1023) = 24 > > 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 > > 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT > > 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", > > 0xFFFFFD7FFFDFF000) Err#2 ENOENT > > 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", > 0xFFFFFD7FFFDFF000) = > > 0 > > 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", > > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 > > 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 > > 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, > > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > > 0.0113 close(3) = 0 > > 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 > > 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, > 0, > > 0) = 0 > > 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) > > Err#2 ENOENT > > 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", > > "/lib/amd64/libsocket.so.1", 1023) = 25 > > 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 > > 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, > > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > > 0.0127 close(3) = 0 > > 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 > > 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, > 0, > > 0) = 0 > > 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) > Err#2 > > ENOENT > > 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", > > "/lib/amd64/libnsl.so.1", 1023) = 22 > > 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 > > 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, > > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > > 0.0139 close(3) = 0 > > 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 > > 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, > 0, > > 0) = 0 > > 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 > > ENOENT > > 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > > 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", > "/lib/amd64/libm.so.2", > > 1023) = 20 > > 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 > > 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, > > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > > 0.0150 close(3) = 0 > > 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 > > 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, > 0, > > 0) = 0 > > 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 > > ENOENT > > 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", > "/lib/amd64/libc.so.1", > > 1023) = 20 > > 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 > > 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, > > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 > > 0.0162 close(3) = 0 > > 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, > > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 > > 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, > 0, > > 0) = 0 > > 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0168 resolvepath("/lib/64/libsocket.so.1", > > "/lib/amd64/libsocket.so.1", 1023) = 25 > > 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", > > 1023) = 22 > > 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 > > 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", > 1023) = > > 20 > > 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) > > Err#2 ENOENT > > 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT > > 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 > > 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", > > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 > > 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Below searches which it cannot resolve: > > > > 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 > > 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 > > 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) > Err#22 > > EINVAL > > 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 > ENOENT > > 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 > > ENOENT > > 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) > Err#2 > > ENOENT > > 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 > > 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 > ENOENT > > 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) > > Err#2 ENOENT > > 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0 > > 0.0351 brk(0x00485250) = 0 > > > > 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 > > 0.0459 stat("/opt/ha/lib/python2.6/site-packages", > 0xFFFFFD7FFFDFDF90) = > > 0 > > 0.0461 stat("/opt/ha/lib/python2.6/site-packages", > 0xFFFFFD7FFFDFE340) = > > 0 > > 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", > > 0xFFFFFD7FFFDFE640) Err#2 ENOENT > > 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", > > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > > 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", > O_RDONLY) > > Err#2 ENOENT > > 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", > > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > > 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", > > O_RDONLY) Err#2 ENOENT > > 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", > > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > > 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) > > Err#2 ENOENT > > 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", > > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT > > 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) > > Err#2 ENOENT > > 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT > > 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 > > 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT > > 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 > > 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 > > 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 > ENOENT > > 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 > ENOENT > > 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 > ENOENT > > 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 > ENOENT > > 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 > > ENOENT > > 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 > ENOENT > > 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 > > 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 > > 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 > > 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 > > 0.0494 brk(0x004D5250) = 0 > > 0.0496 brk(0x004D9250) = 0 > > 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 > > 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY > > 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 > > > > 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", > 0xFFFFFD7FFFDFD320) > > Err#2 ENOENT > > 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", > > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) > > Err#2 ENOENT > > 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", > > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", > > O_RDONLY) Err#2 ENOENT > > 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", > > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) > Err#2 > > ENOENT > > 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", > > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) > > Err#2 ENOENT > > 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT > > 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT > > 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 > ENOENT > > 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT > > 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 > > 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 > > 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 > > 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 > > 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 > > 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY > > 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 > > > > 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > > 0xFFFFFD7FFFDFC000) Err#2 ENOENT > > 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > > 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", > > O_RDONLY) Err#2 ENOENT > > 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > > 0.0575 > open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", > > O_RDONLY) Err#2 ENOENT > > 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > > 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", > > O_RDONLY) Err#2 ENOENT > > 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", > > 0xFFFFFD7FFFDFC490) Err#2 ENOENT > > 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", > > O_RDONLY) Err#2 ENOENT > > 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 > > ENOENT > > 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > > ENOENT > > 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 > ENOENT > > 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > > ENOENT > > 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) > Err#2 > > ENOENT > > 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 > > ENOENT > > 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 > > 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 > > 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 > > 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 > > > > > > And the list goes on. > > > > Hope there's a solution for this. > > > > > > Joe > > > > > > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald > wrote: > >> > >> > >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos > wrote: > >> > > >> > Hi, > >> > > >> > I have an application working on Omnios r10 which depends on certain > >> > python libraries. There are certain environment variables in place > which > >> > point to the location of those libraries. > >> > > >> > I have updated the r10 machine to r12. The application now cannot be > >> > started with errors stating that the needed libraries cannot be found > in the > >> > given paths. > >> > >> Share the errors please? I'll need more details. > >> > >> > Maybe somebody can tell me what the difference is in python version > >> > between the two omnios releases. Python -V gives me 2.6.8. on both > versions. > >> > >> We updated supplemental python libraries, which may contribute to what > >> you're seeing. Also, we had not updated the "entire" metapackage on > 010 to > >> show we were actually running 2.6.8. > >> > >> Dan > >> > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at houseofancients.nl Sat Dec 13 14:01:04 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Sat, 13 Dec 2014 14:01:04 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> Message-ID: <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone : ntxn0 ntxn0 Ethernet up 1000 full BOUND 192.168.249.104 255.255.255.0 78:ac:c0:11:d4:88 1500 ok ntxn1 ntxn1 Ethernet unknown 1000 full static unset - - 1500 ntxn2 ntxn2 Ethernet unknown 1000 full static unset - - 1500 ntxn3 ntxn3 Ethernet unknown 1000 full static unset - - 1500 this all lead me to conclude that there is something wrong with the IGB implementation as it sits Weather it be drivers, or something more to the background..don't know, but would love to find out Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Floris van Essen ..:: House of Ancients Amstafs ::.. Verzonden: zaterdag 13 december 2014 11:38 Aan: Dan McDonald CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 So, To add to the weirdness. Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange : Sh arp Internet x.x.x.104 52 a036.9f17.abc0 ARPA Vlanx Wait, that is the mac of my IGB interface... did I miss something ? ifconfig igb0: flags=1004843 mtu 1500 index 13 inet 0.0.0.0 netmask ff000000 ether a0:36:9f:17:ab:c0 so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box... This means hardware, and drivers are ok! then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0 I think this is something within Omnios , however, would like some pointers where to look Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::.. Verzonden: vrijdag 12 december 2014 18:11 Aan: Dan McDonald CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > Hi Dann, > > No problem, just happy you did :-) > > Right, so as might remember I had to reinstall because I was running > bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... >> Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. Did the installation 2 weeks ago, so guessing that that was before you did the changes. No problem, I'll update to stable next release :-) > So here we are again, running bloody r13 , fully updated :-) > > # dladm show-ether > LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE > e1000g1 current up yes 1G-f bi > e1000g0 current up yes 1G-f bi > igb0 current up yes 1G-f bi > igb1 current up yes 1G-f bi > igb2 current up yes 1G-f bi > igb3 current up yes 1G-f bi > # dladm show-link > LINK CLASS MTU STATE BRIDGE OVER > e1000g1 phys 1500 up -- -- > e1000g0 phys 1500 up -- -- > aggr0 aggr 1500 up -- e1000g0 e1000g1 > igb0 phys 1500 up -- -- > igb1 phys 1500 up -- -- > igb2 phys 1500 up -- -- > igb3 phys 1500 up -- -- >>Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > # ifconfig -a > lo0: flags=2001000849 mtu 8232 index 1 > inet 127.0.0.1 netmask ff000000 > aggr0: flags=1000843 mtu 1500 index 2 > inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 > ether 0:30:48:d5:ec:94 And you have no problems pinging x.x.x.0/24 addresses? Or do you? Non, actually connecting on that interface with ssh > igb0: flags=1000843 mtu 1500 index 3 > inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6c > igb1: flags=1000843 mtu 1500 index 4 > inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6d > igb2: flags=1000843 mtu 1500 index 5 > inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6e > igb3: flags=1000843 mtu 1500 index 6 > inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 > ether a0:36:9f:2:c2:6f >>Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another. >>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. That is exactly what i did.. for each test brought the others down Just to add, when doing a SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news >>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100% Dan ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From danmcd at omniti.com Sat Dec 13 16:12:35 2014 From: danmcd at omniti.com (Dan McDonald) Date: Sat, 13 Dec 2014 11:12:35 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> Message-ID: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> I thought I'd mentioned this on the thread, but does the machine attempt to use one of the IGB interfaces for IPMI as well? We do not support dual-use NICs like that. (Check your BIOS.) Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 13, 2014, at 9:01 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone : > > ntxn0 ntxn0 Ethernet up 1000 full BOUND 192.168.249.104 255.255.255.0 78:ac:c0:11:d4:88 1500 ok > ntxn1 ntxn1 Ethernet unknown 1000 full static unset - - 1500 > ntxn2 ntxn2 Ethernet unknown 1000 full static unset - - 1500 > ntxn3 ntxn3 Ethernet unknown 1000 full static unset - - 1500 > > this all lead me to conclude that there is something wrong with the IGB implementation as it sits > Weather it be drivers, or something more to the background..don't know, but would love to find out > > Met vriendelijke groet / With kind regards, > > > > Floris van Essen > > -----Oorspronkelijk bericht----- > Van: Floris van Essen ..:: House of Ancients Amstafs ::.. > Verzonden: zaterdag 13 december 2014 11:38 > Aan: Dan McDonald > CC: omnios-discuss at lists.omniti.com > Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > > So, > > To add to the weirdness. > > Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange : > > Sh arp > Internet x.x.x.104 52 a036.9f17.abc0 ARPA Vlanx > > Wait, that is the mac of my IGB interface... did I miss something ? > > ifconfig > igb0: flags=1004843 mtu 1500 index 13 > inet 0.0.0.0 netmask ff000000 > ether a0:36:9f:17:ab:c0 > > so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box... > This means hardware, and drivers are ok! > > then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0 > > I think this is something within Omnios , however, would like some pointers where to look > > Met vriendelijke groet / With kind regards, > > > > Floris van Essen > > -----Oorspronkelijk bericht----- > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::.. > Verzonden: vrijdag 12 december 2014 18:11 > Aan: Dan McDonald > CC: omnios-discuss at lists.omniti.com > Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > > > > > > >> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: >> >> Hi Dann, >> >> No problem, just happy you did :-) >> >> Right, so as might remember I had to reinstall because I was running >> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... > >>> Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. > > Did the installation 2 weeks ago, so guessing that that was before you did the changes. > No problem, I'll update to stable next release :-) > >> So here we are again, running bloody r13 , fully updated :-) >> >> # dladm show-ether >> LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE >> e1000g1 current up yes 1G-f bi >> e1000g0 current up yes 1G-f bi >> igb0 current up yes 1G-f bi >> igb1 current up yes 1G-f bi >> igb2 current up yes 1G-f bi >> igb3 current up yes 1G-f bi >> # dladm show-link >> LINK CLASS MTU STATE BRIDGE OVER >> e1000g1 phys 1500 up -- -- >> e1000g0 phys 1500 up -- -- >> aggr0 aggr 1500 up -- e1000g0 e1000g1 >> igb0 phys 1500 up -- -- >> igb1 phys 1500 up -- -- >> igb2 phys 1500 up -- -- >> igb3 phys 1500 up -- -- > >>> Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > >> # ifconfig -a >> lo0: flags=2001000849 mtu 8232 index 1 >> inet 127.0.0.1 netmask ff000000 >> aggr0: flags=1000843 mtu 1500 index 2 >> inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 >> ether 0:30:48:d5:ec:94 > > And you have no problems pinging x.x.x.0/24 addresses? Or do you? > > Non, actually connecting on that interface with ssh > >> igb0: flags=1000843 mtu 1500 index 3 >> inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6c >> igb1: flags=1000843 mtu 1500 index 4 >> inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6d >> igb2: flags=1000843 mtu 1500 index 5 >> inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6e >> igb3: flags=1000843 mtu 1500 index 6 >> inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6f > >>> Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another. > >>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. > > That is exactly what i did.. for each test brought the others down Just to add, when doing a SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news > >>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. > > They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. > To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100% > > Dan > > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl > > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl From jesus at omniti.com Sat Dec 13 17:14:27 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Sat, 13 Dec 2014 09:14:27 -0800 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: A package (like ha) should either rely on the system python packages / versions it needs to work (thus preventing your upgrade to 012) or more reasonably rely on its own known-good python. I don't believe OmniOS makes any release-to-release compatibility guarantees for python (or perl or java for that matter). The only reason they even exist in the core is because we have other core requirements that depend upon them. On Sat, Dec 13, 2014 at 5:50 AM, Joe Veliscos wrote: > > /opt/ha was created by the HA install package. > It is not missing after upgrade but many python files on the other hand, > are removed from the system by the upgrade from r10 to r12. > > > > > On Fri, Dec 12, 2014 at 4:25 PM, Zach Malone wrote: >> >> /opt/ha is locally built, it isn't delivered by OmniOS. I'm not sure >> why it would go missing on upgrade, how did you initially create it? >> --Zach >> >> On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos >> wrote: >> > Hi >> > >> > The command that I execute : >> > root#crm >> > abort: couldn't find crm libraries in [/opt/ha/sbin >> > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg >> > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip >> /usr/lib/python2.6 >> > /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk >> > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload >> > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages] >> > (check your install and PYTHONPATH) >> > >> > I have the following environment variables set: >> > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages >> > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH >> > export OCF_ROOT=/opt/ha/lib/ocf >> > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat >> > >> > I have exactly the same in an r10 release (pre upgrade to rr12) where >> there >> > is no problem >> > >> > I did a truss -d crm and it seems that many files it searches for are >> not >> > found. Snippets of the output (very long file) hope this helps: >> > >> > Below (as I understand it ) some searches which it can resolve: >> > >> > 0.0098 resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", >> 1023) >> > = 18 >> > 0.0100 resolvepath("/usr/bin/amd64/python2.6", >> > "/usr/bin/amd64/python2.6", 1023) = 24 >> > 0.0101 stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0 >> > 0.0103 open("/var/ld/64/ld.config", O_RDONLY) Err#2 ENOENT >> > 0.0105 stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", >> > 0xFFFFFD7FFFDFF000) Err#2 ENOENT >> > 0.0106 stat("/usr/lib/amd64/libpython2.6.so.1.0", >> 0xFFFFFD7FFFDFF000) = >> > 0 >> > 0.0108 resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", >> > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34 >> > 0.0110 open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3 >> > 0.0112 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, >> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 >> > 0.0113 close(3) = 0 >> > 0.0115 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, >> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000 >> > 0.0117 memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, >> MADV_WILLNEED, 0, >> > 0) = 0 >> > 0.0118 stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) >> > Err#2 ENOENT >> > 0.0120 stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0121 resolvepath("/usr/lib/amd64/libsocket.so.1", >> > "/lib/amd64/libsocket.so.1", 1023) = 25 >> > 0.0123 open("/usr/lib/amd64/libsocket.so.1", O_RDONLY) = 3 >> > 0.0125 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, >> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 >> > 0.0127 close(3) = 0 >> > 0.0128 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, >> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000 >> > 0.0129 memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, >> 0, >> > 0) = 0 >> > 0.0130 stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) >> Err#2 >> > ENOENT >> > 0.0132 stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0134 resolvepath("/usr/lib/amd64/libnsl.so.1", >> > "/lib/amd64/libnsl.so.1", 1023) = 22 >> > 0.0135 open("/usr/lib/amd64/libnsl.so.1", O_RDONLY) = 3 >> > 0.0137 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, >> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 >> > 0.0139 close(3) = 0 >> > 0.0140 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, >> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000 >> > 0.0141 memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, >> MADV_WILLNEED, 0, >> > 0) = 0 >> > 0.0142 stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) >> Err#2 >> > ENOENT >> > 0.0144 stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0145 resolvepath("/usr/lib/amd64/libm.so.2", >> "/lib/amd64/libm.so.2", >> > 1023) = 20 >> > 0.0147 open("/usr/lib/amd64/libm.so.2", O_RDONLY) = 3 >> > 0.0149 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, >> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 >> > 0.0150 close(3) = 0 >> > 0.0151 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, >> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000 >> > 0.0153 memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, >> 0, >> > 0) = 0 >> > 0.0154 stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) >> Err#2 >> > ENOENT >> > 0.0156 stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0157 resolvepath("/usr/lib/amd64/libc.so.1", >> "/lib/amd64/libc.so.1", >> > 1023) = 20 >> > 0.0159 open("/usr/lib/amd64/libc.so.1", O_RDONLY) = 3 >> > 0.0161 mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, >> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0 >> > 0.0162 close(3) = 0 >> > 0.0163 mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, >> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000 >> > 0.0165 memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, >> MADV_WILLNEED, 0, >> > 0) = 0 >> > 0.0166 stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0168 resolvepath("/lib/64/libsocket.so.1", >> > "/lib/amd64/libsocket.so.1", 1023) = 25 >> > 0.0170 stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0171 resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", >> > 1023) = 22 >> > 0.0173 stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0174 resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", >> 1023) = >> > 20 >> > 0.0177 stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) >> > Err#2 ENOENT >> > 0.0179 stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 >> ENOENT >> > 0.0180 stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0 >> > 0.0185 resolvepath("/usr/lib/64/libgcc_s.so.1", >> > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28 >> > 0.0187 open("/usr/lib/64/libgcc_s.so.1", O_RDONLY) = 3 >> > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > Below searches which it cannot resolve: >> > >> > 0.0329 fstat(2, 0xFFFFFD7FFFDFF920) = 0 >> > 0.0331 readlink("/usr/bin/python", "python2.6", 1024) = 9 >> > 0.0333 readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) >> Err#22 >> > EINVAL >> > 0.0335 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 >> ENOENT >> > 0.0336 stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) >> Err#2 >> > ENOENT >> > 0.0338 stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) >> Err#2 >> > ENOENT >> > 0.0342 stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0 >> > 0.0344 stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 >> ENOENT >> > 0.0345 stat("/usr/bin/lib/python2.6/lib-dynload", >> 0xFFFFFD7FFFDFF120) >> > Err#2 ENOENT >> > 0.0347 stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = >> 0 >> > 0.0351 brk(0x00485250) = 0 >> > >> > 0.0456 sysconfig(_CONFIG_SIGRT_MAX) = 73 >> > 0.0459 stat("/opt/ha/lib/python2.6/site-packages", >> 0xFFFFFD7FFFDFDF90) = >> > 0 >> > 0.0461 stat("/opt/ha/lib/python2.6/site-packages", >> 0xFFFFFD7FFFDFE340) = >> > 0 >> > 0.0462 stat("/opt/ha/lib/python2.6/site-packages/site", >> > 0xFFFFFD7FFFDFE640) Err#2 ENOENT >> > 0.0464 stat("/opt/ha/lib/python2.6/site-packages/site", >> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT >> > 0.0465 open("/opt/ha/lib/python2.6/site-packages/64/site.so", >> O_RDONLY) >> > Err#2 ENOENT >> > 0.0467 stat("/opt/ha/lib/python2.6/site-packages/site", >> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT >> > 0.0469 open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", >> > O_RDONLY) Err#2 ENOENT >> > 0.0470 stat("/opt/ha/lib/python2.6/site-packages/site", >> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT >> > 0.0471 open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) >> > Err#2 ENOENT >> > 0.0473 stat("/opt/ha/lib/python2.6/site-packages/site", >> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT >> > 0.0474 open("/opt/ha/lib/python2.6/site-packages/site.pyc", >> O_RDONLY) >> > Err#2 ENOENT >> > 0.0476 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 >> ENOENT >> > 0.0477 stat("/usr/lib", 0xFFFFFD7FFFDFDF90) = 0 >> > 0.0478 stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 >> ENOENT >> > 0.0480 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90) = 0 >> > 0.0481 stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340) = 0 >> > 0.0482 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 >> ENOENT >> > 0.0483 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 >> ENOENT >> > 0.0485 open("/usr/lib/python2.6/64/site.so", O_RDONLY) Err#2 >> ENOENT >> > 0.0486 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 >> ENOENT >> > 0.0487 open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 >> > ENOENT >> > 0.0488 stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 >> ENOENT >> > 0.0490 open("/usr/lib/python2.6/site.py", O_RDONLY) = 3 >> > 0.0491 fstat(3, 0xFFFFFD7FFFDFEA90) = 0 >> > 0.0492 open("/usr/lib/python2.6/site.pyc", O_RDONLY) = 4 >> > 0.0493 fstat(4, 0xFFFFFD7FFFDFE8D0) = 0 >> > 0.0494 brk(0x004D5250) = 0 >> > 0.0496 brk(0x004D9250) = 0 >> > 0.0498 fstat(4, 0xFFFFFD7FFFDFE800) = 0 >> > 0.0498 ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880) Err#25 ENOTTY >> > 0.0500 read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944) = 18651 >> > >> > 0.0514 stat("/opt/ha/lib/python2.6/site-packages/os", >> 0xFFFFFD7FFFDFD320) >> > Err#2 ENOENT >> > 0.0516 stat("/opt/ha/lib/python2.6/site-packages/os", >> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT >> > 0.0517 open("/opt/ha/lib/python2.6/site-packages/64/os.so", >> O_RDONLY) >> > Err#2 ENOENT >> > 0.0519 stat("/opt/ha/lib/python2.6/site-packages/os", >> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT >> > 0.0520 open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", >> > O_RDONLY) Err#2 ENOENT >> > 0.0521 stat("/opt/ha/lib/python2.6/site-packages/os", >> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT >> > 0.0522 open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) >> Err#2 >> > ENOENT >> > 0.0524 stat("/opt/ha/lib/python2.6/site-packages/os", >> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT >> > 0.0525 open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) >> > Err#2 ENOENT >> > 0.0527 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 >> ENOENT >> > 0.0528 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 >> ENOENT >> > 0.0529 open("/usr/lib/python2.6/64/os.so", O_RDONLY) Err#2 ENOENT >> > 0.0530 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 >> ENOENT >> > 0.0532 open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 >> ENOENT >> > 0.0533 stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 >> ENOENT >> > 0.0534 open("/usr/lib/python2.6/os.py", O_RDONLY) = 4 >> > 0.0536 fstat(4, 0xFFFFFD7FFFDFD770) = 0 >> > 0.0537 open("/usr/lib/python2.6/os.pyc", O_RDONLY) = 5 >> > 0.0538 fstat(5, 0xFFFFFD7FFFDFD5B0) = 0 >> > 0.0539 fstat(5, 0xFFFFFD7FFFDFD4E0) = 0 >> > 0.0540 ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560) Err#25 ENOTTY >> > 0.0541 read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112) = 25702 >> > >> > 0.0568 stat("/opt/ha/lib/python2.6/site-packages/posixpath", >> > 0xFFFFFD7FFFDFC000) Err#2 ENOENT >> > 0.0570 stat("/opt/ha/lib/python2.6/site-packages/posixpath", >> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT >> > 0.0572 open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", >> > O_RDONLY) Err#2 ENOENT >> > 0.0573 stat("/opt/ha/lib/python2.6/site-packages/posixpath", >> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT >> > 0.0575 >> open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", >> > O_RDONLY) Err#2 ENOENT >> > 0.0576 stat("/opt/ha/lib/python2.6/site-packages/posixpath", >> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT >> > 0.0577 open("/opt/ha/lib/python2.6/site-packages/posixpath.py", >> > O_RDONLY) Err#2 ENOENT >> > 0.0579 stat("/opt/ha/lib/python2.6/site-packages/posixpath", >> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT >> > 0.0580 open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", >> > O_RDONLY) Err#2 ENOENT >> > 0.0582 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) >> Err#2 >> > ENOENT >> > 0.0583 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) >> Err#2 >> > ENOENT >> > 0.0584 open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 >> ENOENT >> > 0.0585 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) >> Err#2 >> > ENOENT >> > 0.0587 open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) >> Err#2 >> > ENOENT >> > 0.0588 stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) >> Err#2 >> > ENOENT >> > 0.0589 open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5 >> > 0.0591 fstat(5, 0xFFFFFD7FFFDFC450) = 0 >> > 0.0592 open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6 >> > 0.0593 fstat(6, 0xFFFFFD7FFFDFC290) = 0 >> > >> > >> > And the list goes on. >> > >> > Hope there's a solution for this. >> > >> > >> > Joe >> > >> > >> > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald >> wrote: >> >> >> >> >> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos >> wrote: >> >> > >> >> > Hi, >> >> > >> >> > I have an application working on Omnios r10 which depends on certain >> >> > python libraries. There are certain environment variables in place >> which >> >> > point to the location of those libraries. >> >> > >> >> > I have updated the r10 machine to r12. The application now cannot be >> >> > started with errors stating that the needed libraries cannot be >> found in the >> >> > given paths. >> >> >> >> Share the errors please? I'll need more details. >> >> >> >> > Maybe somebody can tell me what the difference is in python version >> >> > between the two omnios releases. Python -V gives me 2.6.8. on both >> versions. >> >> >> >> We updated supplemental python libraries, which may contribute to what >> >> you're seeing. Also, we had not updated the "entire" metapackage on >> 010 to >> >> show we were actually running 2.6.8. >> >> >> >> Dan >> >> >> > >> > _______________________________________________ >> > OmniOS-discuss mailing list >> > OmniOS-discuss at lists.omniti.com >> > http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > >> > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at houseofancients.nl Sat Dec 13 17:34:47 2014 From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..) Date: Sat, 13 Dec 2014 17:34:47 +0000 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> Message-ID: <356582D1FC91784992ABB4265A16ED487F2CCF63@vEX01.mindstorm-internet.local> Hi Dan, Yes you did mention it,and I confirmed that is not the case. System Configuration: Supermicro X7DB8 BIOS Configuration: Phoenix Technologies LTD 2.1c 07/04/2011 BMC Configuration: IPMI 1.0 (unknown) Met vriendelijke groet / With kind regards, ???????????????Floris van Essen -----Oorspronkelijk bericht----- Van: Dan McDonald [mailto:danmcd at omniti.com] Verzonden: zaterdag 13 december 2014 17:13 Aan: Floris van Essen ..:: House of Ancients Amstafs ::.. CC: omnios-discuss at lists.omniti.com Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 I thought I'd mentioned this on the thread, but does the machine attempt to use one of the IGB interfaces for IPMI as well? We do not support dual-use NICs like that. (Check your BIOS.) Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 13, 2014, at 9:01 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: > > So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone : > > ntxn0 ntxn0 Ethernet up 1000 full BOUND 192.168.249.104 255.255.255.0 78:ac:c0:11:d4:88 1500 ok > ntxn1 ntxn1 Ethernet unknown 1000 full static unset - - 1500 > ntxn2 ntxn2 Ethernet unknown 1000 full static unset - - 1500 > ntxn3 ntxn3 Ethernet unknown 1000 full static unset - - 1500 > > this all lead me to conclude that there is something wrong with the > IGB implementation as it sits Weather it be drivers, or something more > to the background..don't know, but would love to find out > > Met vriendelijke groet / With kind regards, > > > > Floris van Essen > > -----Oorspronkelijk bericht----- > Van: Floris van Essen ..:: House of Ancients Amstafs ::.. > Verzonden: zaterdag 13 december 2014 11:38 > Aan: Dan McDonald > CC: omnios-discuss at lists.omniti.com > Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > > So, > > To add to the weirdness. > > Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange : > > Sh arp > Internet x.x.x.104 52 a036.9f17.abc0 ARPA Vlanx > > Wait, that is the mac of my IGB interface... did I miss something ? > > ifconfig > igb0: flags=1004843 mtu 1500 index 13 > inet 0.0.0.0 netmask ff000000 > ether a0:36:9f:17:ab:c0 > > so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box... > This means hardware, and drivers are ok! > > then I booted the machine using a KNOPPIX os, just to confirm my > suspicions about the hardware and got ip x.x.x.104 on IGB0 > > I think this is something within Omnios , however, would like some > pointers where to look > > Met vriendelijke groet / With kind regards, > > > > Floris van Essen > > -----Oorspronkelijk bericht----- > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::.. > Verzonden: vrijdag 12 december 2014 18:11 > Aan: Dan McDonald > CC: omnios-discuss at lists.omniti.com > Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750 > > > > > > >> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. wrote: >> >> Hi Dann, >> >> No problem, just happy you did :-) >> >> Right, so as might remember I had to reinstall because I was running >> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12... > >>> Weird. I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again. > > Did the installation 2 weeks ago, so guessing that that was before you did the changes. > No problem, I'll update to stable next release :-) > >> So here we are again, running bloody r13 , fully updated :-) >> >> # dladm show-ether >> LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE >> e1000g1 current up yes 1G-f bi >> e1000g0 current up yes 1G-f bi >> igb0 current up yes 1G-f bi >> igb1 current up yes 1G-f bi >> igb2 current up yes 1G-f bi >> igb3 current up yes 1G-f bi >> # dladm show-link >> LINK CLASS MTU STATE BRIDGE OVER >> e1000g1 phys 1500 up -- -- >> e1000g0 phys 1500 up -- -- >> aggr0 aggr 1500 up -- e1000g0 e1000g1 >> igb0 phys 1500 up -- -- >> igb1 phys 1500 up -- -- >> igb2 phys 1500 up -- -- >> igb3 phys 1500 up -- -- > >>> Tis all looks sane. You're showing 1Gig, full dupliex, and up. This seems sane. > >> # ifconfig -a >> lo0: flags=2001000849 mtu 8232 index 1 >> inet 127.0.0.1 netmask ff000000 >> aggr0: flags=1000843 mtu 1500 index 2 >> inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255 >> ether 0:30:48:d5:ec:94 > > And you have no problems pinging x.x.x.0/24 addresses? Or do you? > > Non, actually connecting on that interface with ssh > >> igb0: flags=1000843 mtu 1500 index 3 >> inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6c >> igb1: flags=1000843 mtu 1500 index 4 >> inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6d >> igb2: flags=1000843 mtu 1500 index 5 >> inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6e >> igb3: flags=1000843 mtu 1500 index 6 >> inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255 >> ether a0:36:9f:2:c2:6f > >>> Now I can totally see some potential confusion here. You've four addrs all in the same netstack and all with the same prefix. Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another. > >>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3. > > That is exactly what i did.. for each test brought the others down > Just to add, when doing a SH ARP on my cisco's, don't see the macs > popping up, which is even more worring , no arps on a switch usually > is very bad news > >>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links? If so, disable the sharing feature in the BIOS. We don't cope with shared-with-IPMI NICs. > > They are not... besides.. the e1000G's ( and their aggr) are working fully as expected. > To make it more interesting.. this was the same setup as I had on r11 > , and then it did seem to work 100% > > Dan > > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl > > ...:: House of Ancients ::... > American Staffordshire Terriers > > +31-628-161-350 > +31-614-198-389 > Het Perk 48 > 4903 RB > Oosterhout > Netherlands > www.houseofancients.nl ...:: House of Ancients ::... American Staffordshire Terriers +31-628-161-350 +31-614-198-389 Het Perk 48 4903 RB Oosterhout Netherlands www.houseofancients.nl From tim at multitalents.net Sat Dec 13 19:52:28 2014 From: tim at multitalents.net (Tim Rice) Date: Sat, 13 Dec 2014 11:52:28 -0800 (PST) Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: On Sat, 13 Dec 2014, Theo Schlossnagle wrote: | A package (like ha) should either rely on the system python packages / | versions it needs to work (thus preventing your upgrade to 012) or more | reasonably rely on its own known-good python. I don't believe OmniOS makes | any release-to-release compatibility guarantees for python (or perl or java | for that matter). The only reason they even exist in the core is because we | have other core requirements that depend upon them. Anyone coming from a UNIX background knowing that OmniOS has it's roots in OpenSolaris would reasonably expect that something depending on programs/libs found in /usr/bin,/lib,/usr/lib would continue to work on the next release. If omniti chooses not to have any compatibility guarantees for python (or perl or java), they should be moved to a separate tree (perhaps /usr/omnios or even /usr/omnios-private). Kind of like how Solaris put the unstable bits in /usr/sfw. -- Tim Rice Multitalents tim at multitalents.net From jesus at omniti.com Sat Dec 13 20:06:24 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Sat, 13 Dec 2014 12:06:24 -0800 Subject: [OmniOS-discuss] Python difference In-Reply-To: References: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com> Message-ID: Agreed. I think we've had a considerable amount of regret that python was not put into to a "system internals" path. And I know I regret having Java and Perl in there at all. It's been stated a few times that as soon as the two bits that depend on Java get removed or replaced that Java will go out the window. Also, having used distros from every modern and many not so modern unicies, I would say that any that provided perl or python in their distro did so with horrible woe (at least in hindsight). On Sat, Dec 13, 2014 at 11:52 AM, Tim Rice wrote: > > On Sat, 13 Dec 2014, Theo Schlossnagle wrote: > > | A package (like ha) should either rely on the system python packages / > | versions it needs to work (thus preventing your upgrade to 012) or more > | reasonably rely on its own known-good python. I don't believe OmniOS > makes > | any release-to-release compatibility guarantees for python (or perl or > java > | for that matter). The only reason they even exist in the core is because > we > | have other core requirements that depend upon them. > > Anyone coming from a UNIX background knowing that OmniOS has > it's roots in OpenSolaris would reasonably expect that something > depending on programs/libs found in /usr/bin,/lib,/usr/lib would > continue to work on the next release. If omniti chooses not to > have any compatibility guarantees for python (or perl or java), > they should be moved to a separate tree (perhaps /usr/omnios or > even /usr/omnios-private). Kind of like how Solaris put the unstable > bits in /usr/sfw. > > > -- > Tim Rice Multitalents > tim at multitalents.net > > > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: From henson at acm.org Sat Dec 13 21:53:58 2014 From: henson at acm.org (Paul B. Henson) Date: Sat, 13 Dec 2014 13:53:58 -0800 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> Message-ID: <20141213215358.GQ29549@bender.unx.csupomona.edu> On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote: > I thought I'd mentioned this on the thread, but does the machine > attempt to use one of the IGB interfaces for IPMI as well? We do not > support dual-use NICs like that. (Check your BIOS.) Hmm, I had thought that the "shared" IPMI port was invisible to the OS? That there was an internal three-port switch, with one port connected to the physical jack on the back of the server, and then the other two wired internally to the NIC on the motherboard and the NIC on the BMC? Do these boxes do something else that actually tweaks with the hardware on the motherboard NIC when the BIOS is set to shared? From doug at will.to Sat Dec 13 22:07:40 2014 From: doug at will.to (Doug Hughes) Date: Sat, 13 Dec 2014 17:07:40 -0500 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <20141213215358.GQ29549@bender.unx.csupomona.edu> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> <20141213215358.GQ29549@bender.unx.csupomona.edu> Message-ID: <548CB8AC.30205@will.to> On 12/13/2014 4:53 PM, Paul B. Henson wrote: > On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote: >> I thought I'd mentioned this on the thread, but does the machine >> attempt to use one of the IGB interfaces for IPMI as well? We do not >> support dual-use NICs like that. (Check your BIOS.) > Hmm, I had thought that the "shared" IPMI port was invisible to the OS? > That there was an internal three-port switch, with one port connected to > the physical jack on the back of the server, and then the other two > wired internally to the NIC on the motherboard and the NIC on the BMC? > > Do these boxes do something else that actually tweaks with the hardware > on the motherboard NIC when the BIOS is set to shared? > That's the typical setup with IPMI NIC = shared. The OS is totally unaware that there's another device using the same hardware. The amazing thing is that while the OS is using the physical port at 1gbit the BMC is using it at 100mbits/sec on the same physical port, all without either knowing about the other. IMHO, the best way to do this is to set the IPMI to use a different VLAN than the host, but that's dependent on the features of your BMC and network hardware. (tagged to BMC, untagged to host). The other options are 'dedicated' where there's a dedicated IPMI/BMC port, and 'failover' which has a dedicated port, but can failover to the motherboard/host port. (crazy: some of them even support bonding across the ports for the BMC; who thought this was a good idea?!) From rt at steait.net Sun Dec 14 13:27:39 2014 From: rt at steait.net (Rune Tipsmark) Date: Sun, 14 Dec 2014 13:27:39 +0000 Subject: [OmniOS-discuss] latency spikes ~every hour Message-ID: <1418563662763.71128@steait.net> hi all, All my vSphere (ESXi5.1) hosts experience a big spike in latency every hour or so. I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All exhibit the same behavior so I don't think its the connection that is causing this. When I modify the arc_shrink_shift 10 (192GB Ram in the System) it helps a bit, the spikes are still with the same regularity but latency peaks at about ~5000ms for the most part. if I leave arc_shrink_shift at the default value they can be higher - up to 15000ms. Looking at the latency as seen from the vSphere hosts the average is usually below 1ms for most datastores. Any ideas what can cause this or what can be done to fix? br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimklimov at cos.ru Sun Dec 14 15:28:48 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Sun, 14 Dec 2014 16:28:48 +0100 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <548CB8AC.30205@will.to> References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> <20141213215358.GQ29549@bender.unx.csupomona.edu> <548CB8AC.30205@will.to> Message-ID: <62E80049-87C8-4B24-8C42-7B8AF13F2589@cos.ru> 13 ??????? 2014??. 23:07:40 CET, Doug Hughes ?????: > >On 12/13/2014 4:53 PM, Paul B. Henson wrote: >> On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote: >>> I thought I'd mentioned this on the thread, but does the machine >>> attempt to use one of the IGB interfaces for IPMI as well? We do >not >>> support dual-use NICs like that. (Check your BIOS.) >> Hmm, I had thought that the "shared" IPMI port was invisible to the >OS? >> That there was an internal three-port switch, with one port connected >to >> the physical jack on the back of the server, and then the other two >> wired internally to the NIC on the motherboard and the NIC on the >BMC? >> >> Do these boxes do something else that actually tweaks with the >hardware >> on the motherboard NIC when the BIOS is set to shared? >> >That's the typical setup with IPMI NIC = shared. The OS is totally >unaware that there's another device using the same hardware. The >amazing >thing is that while the OS is using the physical port at 1gbit the BMC >is using it at 100mbits/sec on the same physical port, all without >either knowing about the other. IMHO, the best way to do this is to set > >the IPMI to use a different VLAN than the host, but that's dependent on > >the features of your BMC and network hardware. (tagged to BMC, untagged > >to host). > >The other options are 'dedicated' where there's a dedicated IPMI/BMC >port, and 'failover' which has a dedicated port, but can failover to >the >motherboard/host port. (crazy: some of them even support bonding across > >the ports for the BMC; who thought this was a good idea?!) > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Imho it varies with hardware and firmware. Recent supermicros apparently offer, among other configurable options, failover between a dedicated ipmi port (deemed more reliable) and the connection shared over a port of the server (saves on cabling) if both ports have a cable. According to forum critique, this implementation is currently prone to failure if the datacenter takes time to power on and for the switches (STP) to converge and begin passing frames. The ipmi module sticks to one choice it made during bootup and no longer fails over if that link goes down. Something of the sort, they say. What i did have experience with was with sun x2100, where they had 2*bge + 2*nge (less reliable in practice under load) interfaces, and one of the better ones was shared with ipmi, and a reboot of the host using the interface caused the ipmi module to lose connection. So essentially that port was to be left unconfigured in the OS, for remote management to remain usable when you need it. Jim -- Typos courtesy of K-9 Mail on my Samsung Android From rt at steait.net Sun Dec 14 15:44:57 2014 From: rt at steait.net (Rune Tipsmark) Date: Sun, 14 Dec 2014 15:44:57 +0000 Subject: [OmniOS-discuss] Fibre Target problems In-Reply-To: <5415604A.7030408@gmail.com> References: <541155D5.8080406@gmail.com> <562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>, <5415604A.7030408@gmail.com> Message-ID: <1418571900149.76065@steait.net> did you ever find a solution? I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... br, Rune ________________________________________ From: OmniOS-discuss on behalf of Mark Sent: Sunday, September 14, 2014 11:30 AM To: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Fibre Target problems On 11/09/2014 9:25 p.m., OSN | Marian Fischer wrote: > Hi, > > do you have Sync=disabled in ZFS / ZPOOL settings? > If not, this can cause the slow speed ... No, but I didn't with OI either, and it has no issues achieving 400+ Mbytes/sec and doesn't have link loss issues. > > Mit besten Gruessen > > Marian Fischer > -- > OSN Online Service Nuernberg GmbH http://www.osn.de > Bucher Str. 78 Tel: 0911/39905-0 > 90408 Nuernberg Fax: 0911/39905-55 > HRB 15022 Nuernberg, Ust-Id: DE189301263 GF: Joerg Goltermann > > -----Urspr?ngliche Nachricht----- > Von: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Im > Auftrag von Mark > Gesendet: Donnerstag, 11. September 2014 09:57 > An: omnios-discuss at lists.omniti.com > Betreff: [OmniOS-discuss] Fibre Target problems > > I have a Supermicro X7DWN+ based server with 24 x 4Tb SAS disks on a LSI > 6G IT mode hba. > > After configuring a bunch of 10Tb Luns and presenting to a Win2012 > server, write is very slow < 6 Mb/sec, and writing causes the Fc link to > drop repeatedly. > Reads reach about 400Mb/sec. > > OmniOS version is r151006_057. > > I've tried two different 4G and an 8G QLogic adapters, via switch or > direct, but there is no change in behaviour. Only 1 HBA and path. > > The odd thing is the exact same hardware and os setup (fct, zpool etc.) > works well with OI or Solaris 11/11, with writes getting around 4-500 > Mb/sec. > I had to start with OI147 and work up, as the later text installer is > very buggy. Upgrading OI with qlt installed fails. > > I'm at a bit of a loss as to a likely cause. > > Anyone have any pointers ? > > > Some details and log: > > HBA Port WWN: 2100001b320aba27 > Port Mode: Target > Port ID: 10100 > OS Device Name: Not Applicable > Manufacturer: QLogic Corp. > Model: QLE2460 > Firmware Version: 5.2.1 > FCode/BIOS Version: N/A > Serial Number: not available > Driver Name: COMSTAR QLT > Driver Version: 20100505-1.05 > Type: F-port > State: online > Supported Speeds: 1Gb 2Gb 4Gb > Current Speed: 4Gb > Node WWN: 2000001b320aba27 > Link Error Statistics: > Link Failure Count: 0 > Loss of Sync Count: 0 > Loss of Signal Count: 0 > Primitive Seq Protocol Error Count: 0 > Invalid Tx Word Count: 4 > Invalid CRC Count: 0 > > > > prtconf > > value='ISP2432-based 4Gb Fibre Channel to PCI Express HBA' > name='subsystem-name' type=string items=1 > value='unknown subsystem' > Device Minor Nodes:dev=(3,1) > dev_path=/pci at 0,0/pci8086,4027 at 7/pci8086,3500 at 0/pci8086,3510 at 0/pci1077,137 at 0 > :qlt1 > > echo "*stmf_trace_buf/s" | mdb -k > > 0xffffff090f7c0000: qlt1,0:0001318: iport is ffffff090f1921b8 > > :0003718: Imported the LU 600144f09cdd9224000053f56a5c0005 > :0003719: Imported the LU 600144f09cdd9224000053f56a5d0006 > :0003721: Imported the LU 600144f09cdd9224000053f56a5d0007 > :0003722: Imported the LU 600144f09cdd9224000053f56a5e0008 > :0003725: Imported the LU 600144f09cdd9224000053f56a5e0009 > :0003726: Imported the LU 600144f09cdd9224000053f56a5f000a > :0003728: Imported the LU 600144f09cdd9224000053f56a5f000b > qlt1,0:0003815: Async event 8010 mb1=f8e8 mb2=c108, mb3=0, mb5=3362, mb6=0 > qlt1,0:0003815: Async event 8011 mb1=3 mb2=c108, mb3=0, mb5=3362, mb6=0 > qlt1,0:0003815: port state change from 0 to e > qlt1,0:0003815: Async event 8014 mb1=ffff mb2=6, mb3=0, mb5=3362, mb6=0 > qlt1,0:0003815: Posting unsol ELS 3 (PLOGI) rp_id=e8 lp_id=ef > qlt1,0:0003815: Rcvd PLOGI with wrong lportid ef, expecting 0. Killing ELS. > qlt1,0:0003815: port state change from e to 4 > qlt1,0:0004216: Posting unsol ELS 3 (PLOGI) rp_id=e8 lp_id=ef > qlt1,0:0004216: Processing unsol ELS 3 (PLOGI) rp_id=e8 > qlt1,0:0004216: Posting unsol ELS 20 (PRLI) rp_id=e8 lp_id=ef > qlt1,0:0004216: Processing unsol ELS 20 (PRLI) rp_id=e8 > qlt1,0:7247736: Posting unsol ELS 5 (LOGO) rp_id=e8 lp_id=ef > qlt1,0:7247736: Processing unsol ELS 5 (LOGO) rp_id=e8 > qlt1,0:7247736: handling LOGO rp_id e8. Triggering cleanup > :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00 > qlt1,0:7248836: port state change from 4 to 11 > :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00 > qlt1,0:7248836: port state change from 11 to 11 > :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00 > qlt1,0:7248936: port state change from 11 to 11 > :7248936: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00 > qlt1,0:7249036: port state change from 11 to 11 > :7249036: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00 > qlt1,0:7249136: port state change from 11 to 11 > qlt1,0:7249136: Killing ELS 5 cond 1 > qlt1,0:7249136: Processing sol ELS 5 (LOGO) rp_id=e8 > qlt1,0:7249136: handling LOGO rp_id e8. Triggering cleanup > qlt1,0:7249236: port state change from 11 to 11 > qlt1,0:7249336: port state change from 11 to 11 > qlt1,0:7249336: port state change from 11 to 11 > qlt1,0:7249436: port state change from 11 to 11 > qlt1,0:7249536: port state change from 11 to 11 > qlt1,0:7249536: handling LOGO rp_id e8, waiting for cmds to drain > qlt1,0:7249636: port state change from 11 to 11 > qlt1,0:7249736: port state change from 11 to 11 > qlt1,0:7249836: port state change from 11 to 11 > qlt1,0:7249936: port state change from 11 to 11 > qlt1,0:7250036: port state change from 11 to 11 > qlt1,0:7250136: port state change from 11 to 11 > qlt1,0:7250236: port state change from 11 to 11 > :7250236: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250336: port state change from 11 to 11 > :7250336: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250336: port state change from 11 to 11 > :7250336: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250436: port state change from 11 to 11 > :7250436: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250536: port state change from 11 to 11 > :7250536: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250636: port state change from 11 to 11 > :7250636: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250736: port state change from 11 to 11 > :7250736: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250836: port state change from 11 to 11 > :7250836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7250936: port state change from 11 to 11 > :7250936: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7251136: port state change from 11 to 11 > :7251136: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo: > unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0 > qlt1,0:7251236: port state change from 11 to 11 > > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From richard.elling at richardelling.com Sun Dec 14 15:48:10 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Sun, 14 Dec 2014 07:48:10 -0800 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> References: <1418336750819.83182@steait.net> <893101d01592$59168140$0b4383c0$@acm.org> <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> Message-ID: <1372E447-00C5-4BA6-953F-154178050022@RichardElling.com> > On Dec 11, 2014, at 2:40 PM, Dan McDonald wrote: > > Thanks for the fast reboot catch! > > Yes, were contemplating it for '014 AND for upstreaming. Not sure yet if it'll happen, but this thread makes a compelling case. FWIW, even Oracle has not been successful in getting fastboot to work consistently in x86. We always disable it, as do some other illumos distros. Make your life easier, disable by default. -- richard > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Dec 11, 2014, at 5:32 PM, Paul B. Henson wrote: > >>> Rune Tipsmark >>> Sent: Thursday, December 11, 2014 2:26 PM >>> >>> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 >>> have issues rebooting. They simply hang and never ever reboot. >> >> Disable fast reboot: >> >> svccfg -s "system/boot-config:default" setprop >> config/fastreboot_default=false >> svccfg -s "system/boot-config:default" setprop >> config/fastreboot_onpanic=false >> svcadm refresh svc:/system/boot-config:default >> >> Some systems (it seems particularly common on supermicro hardware) tend to >> wedge up with fast reboot enabled. If you want to verify this is the issue >> before changing the configuration, try 'reboot -p' which should do a one >> time regular reboot without changing the default. >> >> Dan, there was some talk of making fast reboot disabled the default, and >> having people who wanted it need to enable it, rather than the other way >> around? Any thoughts? >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From henson at acm.org Mon Dec 15 01:36:59 2014 From: henson at acm.org (Paul B. Henson) Date: Sun, 14 Dec 2014 17:36:59 -0800 Subject: [OmniOS-discuss] LACP Omnios , igb and C3750 In-Reply-To: <548CB8AC.30205@will.to> References: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local> <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local> <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local> <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com> <20141213215358.GQ29549@bender.unx.csupomona.edu> <548CB8AC.30205@will.to> Message-ID: <20141215013659.GZ29549@bender.unx.csupomona.edu> On Sat, Dec 13, 2014 at 05:07:40PM -0500, Doug Hughes wrote: > That's the typical setup with IPMI NIC = shared. The OS is totally > unaware that there's another device using the same hardware. Yeah, that's why I'm confused about Dan saying omnios doesn't support it, I'm not sure how it would know it was happening. Unless newer Intel chipsets do something weird to make it happen rather than having a separate mini-switch. > The amazing > thing is that while the OS is using the physical port at 1gbit the BMC > is using it at 100mbits/sec on the same physical port, all without > either knowing about the other. If you think of it as an embedded 3 port switch it's not that amazing :). From henson at acm.org Mon Dec 15 04:30:37 2014 From: henson at acm.org (Paul B. Henson) Date: Sun, 14 Dec 2014 20:30:37 -0800 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable Message-ID: <20141215043037.GA29549@bender.unx.csupomona.edu> One of my winter break plans is to get an illumos-gate dev environment going again :). A while back there was some talk of possibly putting up some wiki pages on configuring omnios-stable to build upstream illumos-gate, but I still see only the "Setting Up A Basic Dev Environment" at the moment. I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks about how to get an OmniOS r151012 zone configured for illumos development, and seems like a good starting point. Just wondering if anybody had any further advice or possible hangups to keep an eye out for, or if maybe there was some official omnios wiki page about to be pushed out ;). Thanks much... From danmcd at omniti.com Mon Dec 15 04:38:45 2014 From: danmcd at omniti.com (Dan McDonald) Date: Sun, 14 Dec 2014 23:38:45 -0500 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu> References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: Upstream needs several changes. See http://kebe.com/~danmcd/webrevs/for-omnios Dan Sent from my iPhone (typos, autocorrect, and all) > On Dec 14, 2014, at 11:30 PM, Paul B. Henson wrote: > > One of my winter break plans is to get an illumos-gate dev environment > going again :). A while back there was some talk of possibly putting up > some wiki pages on configuring omnios-stable to build upstream > illumos-gate, but I still see only the "Setting Up A Basic Dev > Environment" at the moment. > > I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks > about how to get an OmniOS r151012 zone configured for illumos > development, and seems like a good starting point. Just wondering if > anybody had any further advice or possible hangups to keep an eye out > for, or if maybe there was some official omnios wiki page about to be > pushed out ;). > > Thanks much... > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mark0x01 at gmail.com Mon Dec 15 06:19:18 2014 From: mark0x01 at gmail.com (Mark) Date: Mon, 15 Dec 2014 19:19:18 +1300 Subject: [OmniOS-discuss] Fibre Target problems In-Reply-To: <1418571900149.76065@steait.net> References: <541155D5.8080406@gmail.com> <562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>, <5415604A.7030408@gmail.com> <1418571900149.76065@steait.net> Message-ID: <548E7D66.6080208@gmail.com> On 15/12/2014 4:44 a.m., Rune Tipsmark wrote: > did you ever find a solution? > I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... > br, > Rune I installed an older OpenIndiana version to work around the issue. One thing I didn't check was if the fc target cache was disabled (may be the current default), so may be worth checking that. Mark. From henson at acm.org Mon Dec 15 08:33:40 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 15 Dec 2014 00:33:40 -0800 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: <20141215083340.GB29549@bender.unx.csupomona.edu> On Sun, Dec 14, 2014 at 11:38:45PM -0500, Dan McDonald wrote: > Upstream needs several changes. See http://kebe.com/~danmcd/webrevs/for-omnios Thanks for the pointer. As I recall the last thread on this subject, I thought it was possible to build upstream as-is with the caveat you needed to disable building one or two components and that lint was noisier than one would like? Hmm, or maybe I'm misremembering, and that was the state if these not as yet accepted upstream patches were actually committed. How's progress on pushing that upstream :)? There's a handful of things I'd really like to do but I just can't bring myself to spend the time to spin up an OI box . Is my impression that stable OI is dead and hipster is a never-ending alpha mistaken? If not, I'm surprised there isn't more interest in getting the gate to build on a stable and well maintained illumos distribution 8-/. Thanks... From richard at netbsd.org Mon Dec 15 10:50:08 2014 From: richard at netbsd.org (Richard PALO) Date: Mon, 15 Dec 2014 11:50:08 +0100 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu> References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: Le 15/12/14 05:30, Paul B. Henson a ?crit : > One of my winter break plans is to get an illumos-gate dev environment > going again :). A while back there was some talk of possibly putting up > some wiki pages on configuring omnios-stable to build upstream > illumos-gate, but I still see only the "Setting Up A Basic Dev > Environment" at the moment. > > I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks > about how to get an OmniOS r151012 zone configured for illumos > development, and seems like a good starting point. Just wondering if > anybody had any further advice or possible hangups to keep an eye out > for, or if maybe there was some official omnios wiki page about to be > pushed out ;). > > Thanks much... > One possibility, that is if you use what you build, is to start from omnios bloody. For example, git remote -v (doctored up for ) > illumos-joyent git://github.com/joyent/illumos-joyent (fetch) > illumos-joyent git://github.com/joyent/illumos-joyent (push) > illumos-omnios git://github.com/omniti-labs/illumos-omnios.git (push) > illumos-omnios git://github.com/omniti-labs/illumos-omnios.git (fetch) > origin ssh://git at github.com//illumos-gate (fetch) > origin ssh://git at github.com//illumos-gate (push) > upstream git://github.com/illumos/illumos-gate (fetch) > upstream git://github.com/illumos/illumos-gate (push) by having your development branches rebased or merged from illumos-omnios, you can now test with 'pkg update -v --be-name=' For upstream, you can then create a branch using upstream and then cherry picking your patchset. I've been doing this since summer after dropping hipster. For myself, my diffs for illumos.sh and nightly.sh: > richard at omnis:/home/richard/src/illumos-gate$ diff illumos.sh usr/src/tools/env/illumos.sh > 47c47 > < export NIGHTLY_OPTIONS='-FnCDAmprt' > --- > > export NIGHTLY_OPTIONS='-FnCDAlmprt' > 60c60 > < export GATE='illumos-gate' > --- > > export GATE='testws' > 162,164d161 > < if [[ -d "$CODEMGR_WS/.git" ]]; then > < export VERSION="$GATE-$(git log -1 --format=%h)" > < else > 166d162 > < fi > 235d230 > < # export ENABLE_SMB_PRINTING='#' > 237,244d231 > < > < export GCC_ROOT=/opt/gcc-4.4.4 > < export CW_NO_SHADOW=1 > < export ONNV_BUILDNUM=151013 > < export RELEASE_DATE=`git log --pretty=format:%cd --date=short -n 1 | tr - .` > < export i386_LINT=/opt/SUNWspro/bin/lint > < export amd64_LINT=/opt/SUNWspro/bin/lint > < export MULTI_PROTO=yes and > richard at omnis:/home/richard/src/illumos-gate$ diff nightly.sh usr/src/tools/scripts/nightly.sh > 2159,2161c2159,2161 > < checkpaths $arg $checkroot > $SRC/check-paths.out 2>&1 > < if [[ -s $SRC/check-paths.out ]]; then > < tee -a $LOGFILE < $SRC/check-paths.out >> $mail_msg_file > --- > > checkpaths $arg $checkroot > $SRC/checkpaths.out 2>&1 > > if [[ -s $SRC/checkpaths.out ]]; then > > tee -a $LOGFILE < $SRC/checkpaths.out >> $mail_msg_file (I also reverted .gitignore from illumos-omnios to use upstream...) If you have any particular questions about my config, offline is perhaps best. It will be great when illumos-omnios and upstream converge 'au maximum'. From sjorge+ml at blackdot.be Mon Dec 15 12:26:02 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Mon, 15 Dec 2014 13:26:02 +0100 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu> References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be> Hi Paul, If while setting up your env you found obvious things lacking on https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop me a line on #omnios (sjorge) or here on the mailing list and I will update the page. Other than that, good luck! Regards Jorge On 2014-12-15 05:30, Paul B. Henson wrote: > One of my winter break plans is to get an illumos-gate dev environment > going again :). A while back there was some talk of possibly putting up > some wiki pages on configuring omnios-stable to build upstream > illumos-gate, but I still see only the "Setting Up A Basic Dev > Environment" at the moment. > > I found https://docu.blackdot.be/snipets/solaris/build-zone, which > talks > about how to get an OmniOS r151012 zone configured for illumos > development, and seems like a good starting point. Just wondering if > anybody had any further advice or possible hangups to keep an eye out > for, or if maybe there was some official omnios wiki page about to be > pushed out ;). > > Thanks much... > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From rt at steait.net Mon Dec 15 12:27:39 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 15 Dec 2014 12:27:39 +0000 Subject: [OmniOS-discuss] latency spikes ~every hour In-Reply-To: <1418563662763.71128@steait.net> References: <1418563662763.71128@steait.net> Message-ID: <1418646462467.87062@steait.net> ok I removed some of my SLOG devices and currently I am only using a single SLOG (no mirror or anything) and no spikes seen since. I wonder why multiple SLOG devices would cause this. br. Rune ________________________________ From: OmniOS-discuss on behalf of Rune Tipsmark Sent: Sunday, December 14, 2014 2:27 PM To: omnios-discuss at lists.omniti.com Subject: [OmniOS-discuss] latency spikes ~every hour hi all, All my vSphere (ESXi5.1) hosts experience a big spike in latency every hour or so. I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All exhibit the same behavior so I don't think its the connection that is causing this. When I modify the arc_shrink_shift 10 (192GB Ram in the System) it helps a bit, the spikes are still with the same regularity but latency peaks at about ~5000ms for the most part. if I leave arc_shrink_shift at the default value they can be higher - up to 15000ms. Looking at the latency as seen from the vSphere hosts the average is usually below 1ms for most datastores. Any ideas what can cause this or what can be done to fix? br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Mon Dec 15 12:45:15 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 15 Dec 2014 12:45:15 +0000 Subject: [OmniOS-discuss] Fibre Target problems In-Reply-To: <548E7D66.6080208@gmail.com> References: <541155D5.8080406@gmail.com> <562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>, <5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>,<548E7D66.6080208@gmail.com> Message-ID: <1418647518088.75378@steait.net> where do you check that? br, Rune ________________________________________ From: Mark Sent: Monday, December 15, 2014 7:19 AM To: Rune Tipsmark; omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] Fibre Target problems On 15/12/2014 4:44 a.m., Rune Tipsmark wrote: > did you ever find a solution? > I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... > br, > Rune I installed an older OpenIndiana version to work around the issue. One thing I didn't check was if the fc target cache was disabled (may be the current default), so may be worth checking that. Mark. From alexmcwhirter at vantagetitle.com Thu Dec 11 00:45:48 2014 From: alexmcwhirter at vantagetitle.com (Alex McWhirter) Date: Wed, 10 Dec 2014 19:45:48 -0500 Subject: [OmniOS-discuss] Change Zone Hostname Message-ID: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? I had a look at /etc/nodname and it is an empty file when used in a zone. From jorge at blackdot.be Fri Dec 12 08:10:53 2014 From: jorge at blackdot.be (Jorge Schrauwen) Date: Fri, 12 Dec 2014 09:10:53 +0100 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> References: <1418336750819.83182@steait.net> <893101d01592$59168140$0b4383c0$@acm.org> <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> Message-ID: SuperMicro boards seem quite popular too... anybody ever figure out why they are hanging on fast reboot? I have the same issue but it only hangs about 1/3 of the time. (Fastreboot is now off though) --- ~ sjorge On 2014-12-11 23:40, Dan McDonald wrote: > Thanks for the fast reboot catch! > > Yes, were contemplating it for '014 AND for upstreaming. Not sure yet > if it'll happen, but this thread makes a compelling case. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Dec 11, 2014, at 5:32 PM, Paul B. Henson wrote: > >>> Rune Tipsmark >>> Sent: Thursday, December 11, 2014 2:26 PM >>> >>> I got a bunch (3) installations of omnios on SuperMicro hardware and >>> all 3 >>> have issues rebooting. They simply hang and never ever reboot. >> >> Disable fast reboot: >> >> svccfg -s "system/boot-config:default" setprop >> config/fastreboot_default=false >> svccfg -s "system/boot-config:default" setprop >> config/fastreboot_onpanic=false >> svcadm refresh svc:/system/boot-config:default >> >> Some systems (it seems particularly common on supermicro hardware) >> tend to >> wedge up with fast reboot enabled. If you want to verify this is the >> issue >> before changing the configuration, try 'reboot -p' which should do a >> one >> time regular reboot without changing the default. >> >> Dan, there was some talk of making fast reboot disabled the default, >> and >> having people who wanted it need to enable it, rather than the other >> way >> around? Any thoughts? >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From matthew.lagoe at subrigo.net Mon Dec 15 19:16:14 2014 From: matthew.lagoe at subrigo.net (Matthew Lagoe) Date: Mon, 15 Dec 2014 11:16:14 -0800 Subject: [OmniOS-discuss] hangs on reboot In-Reply-To: References: <1418336750819.83182@steait.net> <893101d01592$59168140$0b4383c0$@acm.org> <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com> Message-ID: <005e01d0189b$99e2f5d0$cda8e170$@subrigo.net> Ive had the same issue with dell servers as well as supermicro Seems fast reboot just has issues, I always disable it as I have yet to see it work -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Jorge Schrauwen Sent: Friday, December 12, 2014 12:11 AM To: Dan McDonald Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] hangs on reboot SuperMicro boards seem quite popular too... anybody ever figure out why they are hanging on fast reboot? I have the same issue but it only hangs about 1/3 of the time. (Fastreboot is now off though) --- ~ sjorge On 2014-12-11 23:40, Dan McDonald wrote: > Thanks for the fast reboot catch! > > Yes, were contemplating it for '014 AND for upstreaming. Not sure yet > if it'll happen, but this thread makes a compelling case. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Dec 11, 2014, at 5:32 PM, Paul B. Henson wrote: > >>> Rune Tipsmark >>> Sent: Thursday, December 11, 2014 2:26 PM >>> >>> I got a bunch (3) installations of omnios on SuperMicro hardware and >>> all 3 have issues rebooting. They simply hang and never ever reboot. >> >> Disable fast reboot: >> >> svccfg -s "system/boot-config:default" setprop >> config/fastreboot_default=false svccfg -s >> "system/boot-config:default" setprop config/fastreboot_onpanic=false >> svcadm refresh svc:/system/boot-config:default >> >> Some systems (it seems particularly common on supermicro hardware) >> tend to wedge up with fast reboot enabled. If you want to verify this >> is the issue before changing the configuration, try 'reboot -p' which >> should do a one time regular reboot without changing the default. >> >> Dan, there was some talk of making fast reboot disabled the default, >> and having people who wanted it need to enable it, rather than the >> other way around? Any thoughts? >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Mon Dec 15 20:34:15 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 15:34:15 -0500 Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI In-Reply-To: References: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net> Message-ID: <5B181158-1EA2-441D-8FD9-9C8ADB8F85F4@omniti.com> Sorry for not getting back to you on this sooner. Have you tried putting your patches into the omnios-build tree? What you'd have to do is: 1.) mkdir $OMNIOS_BUILD/build/isc-dhcp/patches/ 2.) Put your patch in a file in the aforementioned directory: say "use-dlpi.patch". 3.) Edit .../patches/series so it has one line: "use-dlpi.patch". 4.) Run ./build.sh in "isc-dhcp". It should generate the right binaries. I might be able to do this later in the week, but since you have experience with it already, I thought maybe you could try it. Thanks, Dan From danmcd at omniti.com Mon Dec 15 20:35:15 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 15:35:15 -0500 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com> References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com> Message-ID: <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com> > On Dec 10, 2014, at 7:45 PM, Alex McWhirter wrote: > > I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name? > > I had a look at /etc/nodname and it is an empty file when used in a zone. Did you ask this earlier and I'm seeing a duplicate? Or did someone else ask this? You can have /etc/nodename override the zone's name for a given zone. E.g. my "webserver" zone isn't hostnamed "webserver", because I have a different name in /etc/nodename for that zone. You will have to reboot the zone, however, for the change to stick. Dan From rt at steait.net Mon Dec 15 20:43:58 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 15 Dec 2014 20:43:58 +0000 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. Message-ID: <1418676240949.15076@steait.net> hi all, got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online. the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash. what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Dec 15 20:53:47 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 15:53:47 -0500 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. In-Reply-To: <1418676240949.15076@steait.net> References: <1418676240949.15076@steait.net> Message-ID: > On Dec 15, 2014, at 3:43 PM, Rune Tipsmark wrote: > > hi all, > > got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online. > the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash. > > what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. Dedup is a memory pig and not very well implemented in ZFS. I'd highly recommend against it in production. Either that, or really increase your memory for your system in question. There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled. Sorry, Dan From henson at acm.org Mon Dec 15 20:55:13 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 15 Dec 2014 12:55:13 -0800 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com> References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com> <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com> Message-ID: > From: Dan McDonald > Sent: Monday, December 15, 2014 12:35 PM > > Did you ask this earlier and I'm seeing a duplicate? Or did someone else ask > this? It does seem familiar, plus it has an old date? I believe earlier this month I saw what appeared to be some list reposts too. > You can have /etc/nodename override the zone's name for a given zone. > E.g. my "webserver" zone isn't hostnamed "webserver", because I have a > different name in /etc/nodename for that zone. You will have to reboot the > zone, however, for the change to stick. Just as a data point, I set up a couple of zones over the weekend, and after updating nodename and rebooting the zone, the replacement hostname worked fine. From jtyocum at uw.edu Mon Dec 15 20:51:36 2014 From: jtyocum at uw.edu (John Yocum) Date: Mon, 15 Dec 2014 12:51:36 -0800 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. In-Reply-To: <1418676240949.15076@steait.net> References: <1418676240949.15076@steait.net> Message-ID: <548F49D8.9060601@uw.edu> On 12/15/2014 12:43 PM, Rune Tipsmark wrote: > hi all, > > > > got a new system I was intending on using as backup repository. Whenever > dedup is enabled it dies after anywhere between 5 and 30 minutes. I need > to reboot OmniOS to get it back online. > > the files being copied onto the zfs vols are rather large, about ~2TB > each... if I copy smaller files, say 400GB or so, it takes longer for it > to crash. > > > > what can be done to fix this? after Windows (initiator) looses the > connection (both Fibre Channel and iSCSI) I still see a lot of disk > activity using iostat - disks remain active for minutes after the > copying has died... its like ZFS cannot handle dedup of large files.. > > br, > > Rune > How much RAM do you have? And, how big is your ZFS pool? I ask, as dedup requires a ton of RAM. Helpful reference on the issue http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDedupMemoryProblem -- John Yocum, Systems Administrator, DEOHS From henson at acm.org Mon Dec 15 21:00:08 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 15 Dec 2014 13:00:08 -0800 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be> References: <20141215043037.GA29549@bender.unx.csupomona.edu> <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be> Message-ID: > From: Jorge Schrauwen > Sent: Monday, December 15, 2014 4:26 AM > > If while setting up your env you found obvious things lacking on > https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop > me a line on #omnios (sjorge) or here on the mailing list and I will > update the page. Cool, thanks. Just to clarify, did you apply the pending illumos-gate patches Dan pointed out to get things to compile? As it sounded like without them it wouldn't compile. From richard at netbsd.org Mon Dec 15 21:00:48 2014 From: richard at netbsd.org (Richard PALO) Date: Mon, 15 Dec 2014 22:00:48 +0100 Subject: [OmniOS-discuss] pull request: revert a6f10d64 Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dan suggested I post this here to get a go-ahead. This gist effectively reverts the following which seems unnecessary, at least nowadays: > commit a6f10d643d66d62130a867a42efa06ce8cb93f70 > Author: Theo Schlossnagle > Date: Fri Mar 30 03:42:02 2012 +0000 > > two .gitignore files that cover a completed build > > diff --git a/.gitignore b/.gitignore https://gist.github.com/risto3/adf3f9068eff759f06a7 cheers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJUj0wAAAoJECAB22fHtp27ZtAH+wZk2tjcGXnlzKD4DNf5r9s/ 4fBJ++ojxTPbO3jQuE5e0XpMAbADPQLgR+A6vT07AYoruRIx81uUHg3nihJNlrXx Um8DRTKViQcyUHWsiRXYUI8NgF4oeGryKohyfphZGbZYLPlHqmjxB4S3WpFKNkPx TgKOpHzInc9uMqQbNTQNM/SA9h8ydo3pWhl1Y/SYQl2TyxvYmHCOpcImsqaLmiUT wF/OhoQxVYHy1LHZwDOSR4rHAvqYhaHwhWCtnVqVZbkbGpjhHBHBSLN8G+/UDKd3 GM9PxZGJyB1axsP2I5YwuQFFQXDnjP5veZONaMsc9y5YbLAseOU45uXI9QbCtGM= =6G0k -----END PGP SIGNATURE----- From hasslerd at gmx.li Mon Dec 15 21:11:55 2014 From: hasslerd at gmx.li (Dominik Hassler) Date: Mon, 15 Dec 2014 22:11:55 +0100 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. In-Reply-To: References: <1418676240949.15076@steait.net> Message-ID: <548F4E9B.6060904@gmx.li> Hi, we used dedup on a production machine w/ 256 GB RAM, but disabled it after a couple of days due to huge performance impact. I would not recommend to use dedup even when having "enough" RAM. On 12/15/2014 09:53 PM, Dan McDonald wrote: > >> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark wrote: >> >> hi all, >> >> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online. >> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash. >> >> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. > > Dedup is a memory pig and not very well implemented in ZFS. I'd highly recommend against it in production. Either that, or really increase your memory for your system in question. There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled. > > Sorry, > Dan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0xF9ECC6A5.asc Type: application/pgp-keys Size: 2686 bytes Desc: not available URL: From eric.sproul at circonus.com Mon Dec 15 21:15:51 2014 From: eric.sproul at circonus.com (Eric Sproul) Date: Mon, 15 Dec 2014 16:15:51 -0500 Subject: [OmniOS-discuss] Change Zone Hostname In-Reply-To: References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com> <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com> Message-ID: FYI, the mail that appeared today was held in the moderator queue, having been sent from a non-subscribed address. I just went through the queue earlier today. From danmcd at omniti.com Mon Dec 15 21:22:14 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 16:22:14 -0500 Subject: [OmniOS-discuss] pull request: revert a6f10d64 In-Reply-To: References: Message-ID: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com> > On Dec 15, 2014, at 4:00 PM, Richard PALO wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Dan suggested I post this here to get a go-ahead. > This gist effectively reverts the following which seems unnecessary, at least nowadays: >> commit a6f10d643d66d62130a867a42efa06ce8cb93f70 >> Author: Theo Schlossnagle >> Date: Fri Mar 30 03:42:02 2012 +0000 >> >> two .gitignore files that cover a completed build >> >> diff --git a/.gitignore b/.gitignore > > https://gist.github.com/risto3/adf3f9068eff759f06a7 I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix. If it's silent, then I'm okay taking this in. If there's historical reasons this happened originally, I'd love to hear them on this thread. Dan From rt at steait.net Mon Dec 15 22:17:09 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 15 Dec 2014 22:17:09 +0000 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. In-Reply-To: <548F4E9B.6060904@gmx.li> References: <1418676240949.15076@steait.net> , <548F4E9B.6060904@gmx.li> Message-ID: <1418681831295.66175@steait.net> we have only 24GB of ram on this system... I was under the impression it would not require much when the block size was larger, we are on 64kb, so would expect around 2GB per 1TB. yet we cannot even get more than a few TB on the system before it dies. the main purpose with this system was dedup, no problem with slow speed... its for backup only, so could not care less if its rather slow - but not responding is not acceptable. br, Rune ________________________________________ From: OmniOS-discuss on behalf of Dominik Hassler Sent: Monday, December 15, 2014 10:11 PM To: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. Hi, we used dedup on a production machine w/ 256 GB RAM, but disabled it after a couple of days due to huge performance impact. I would not recommend to use dedup even when having "enough" RAM. On 12/15/2014 09:53 PM, Dan McDonald wrote: > >> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark wrote: >> >> hi all, >> >> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online. >> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash. >> >> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. > > Dedup is a memory pig and not very well implemented in ZFS. I'd highly recommend against it in production. Either that, or really increase your memory for your system in question. There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled. > > Sorry, > Dan > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From sjorge+ml at blackdot.be Tue Dec 16 00:15:58 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Tue, 16 Dec 2014 01:15:58 +0100 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: References: <20141215043037.GA29549@bender.unx.csupomona.edu> <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be> Message-ID: illumos-omnios works fine without. illumos-gate needs dan's patches to compile (but it still gives lint message IIRC) On 2014-12-15 22:00, Paul B. Henson wrote: >> From: Jorge Schrauwen >> Sent: Monday, December 15, 2014 4:26 AM >> >> If while setting up your env you found obvious things lacking on >> https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop >> me a line on #omnios (sjorge) or here on the mailing list and I will >> update the page. > > Cool, thanks. Just to clarify, did you apply the pending illumos-gate > patches Dan pointed out to get things to compile? As it sounded like > without > them it wouldn't compile. From danmcd at omniti.com Tue Dec 16 02:10:32 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 21:10:32 -0500 Subject: [OmniOS-discuss] pull request: revert a6f10d64 In-Reply-To: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com> References: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com> Message-ID: <5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com> > On Dec 15, 2014, at 4:22 PM, Dan McDonald wrote: >> >> https://gist.github.com/risto3/adf3f9068eff759f06a7 > > I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix. If it's silent, then I'm okay taking this in. > > If there's historical reasons this happened originally, I'd love to hear them on this thread. illumos-omnios build went through w/o a hitch. Unless I hear strong objections, I'm taking this one in tomorrow (Tuesday). Dan From henson at acm.org Tue Dec 16 02:20:10 2014 From: henson at acm.org (Paul B. Henson) Date: Mon, 15 Dec 2014 18:20:10 -0800 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: > From: Richard PALO > Sent: Monday, December 15, 2014 2:50 AM > > One possibility, that is if you use what you build, is to start from > omnios bloody. Hmm, do you mean actually develop against and test with illumos-omnios bloody, and then apply the resultant patch to illumos-gate, and RTI using the build log from omnios bloody? > For myself, my diffs for illumos.sh and nightly.sh: Thanks, hopefully I can take this and some of the other information I've gotten and get something going. From danmcd at omniti.com Tue Dec 16 02:24:07 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 21:24:07 -0500 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: References: <20141215043037.GA29549@bender.unx.csupomona.edu> Message-ID: <80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com> > On Dec 15, 2014, at 9:20 PM, Paul B. Henson wrote: > >> From: Richard PALO >> Sent: Monday, December 15, 2014 2:50 AM >> >> One possibility, that is if you use what you build, is to start from >> omnios bloody. > > Hmm, do you mean actually develop against and test with illumos-omnios > bloody, and then apply the resultant patch to illumos-gate, and RTI using > the build log from omnios bloody? Depending on what you do, that MAY be sufficient. I might do the dead-chicken dance and build your submitted diff against stock illumos-gate to be sure, but that's my own RTI advocate opinion. >> For myself, my diffs for illumos.sh and nightly.sh: > > Thanks, hopefully I can take this and some of the other information I've > gotten and get something going. Go for it! Dan From danmcd at omniti.com Tue Dec 16 02:25:43 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 15 Dec 2014 21:25:43 -0500 Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable In-Reply-To: <80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com> References: <20141215043037.GA29549@bender.unx.csupomona.edu> <80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com> Message-ID: > On Dec 15, 2014, at 9:24 PM, Dan McDonald wrote: >> >> Thanks, hopefully I can take this and some of the other information I've >> gotten and get something going. > > Go for it! One important safety tip: ** If you're doing an illumos-omnios nightly you want to actually ONU over a given build of OmniOS, make sure ONNV_BUILDNUM is set to the release number of your target OmniOS. E.g. If I have a bloody machine I wish to ONU with new bits, I better make sure ONNV_BUILDNUM=151013, or if I have a current-stable machine I wish to ONU with new bits, I better make sure ONNV_BUILDNUM=151012. FYI, Dan From chip at innovates.com Tue Dec 16 14:02:12 2014 From: chip at innovates.com (Schweiss, Chip) Date: Tue, 16 Dec 2014 08:02:12 -0600 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: <6e8301d0132f$e1702c40$a45084c0$@acm.org> References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> <674b01d01324$d0e28180$72a78480$@acm.org> <6e8301d0132f$e1702c40$a45084c0$@acm.org> Message-ID: It seems there a many ways to map ID in NFSv4, is there a way to not map them at all? I'm working to configure an OmniOS NFS server that will serve several domains simultaneously. Each which has their own ID map with many conflicting uid/gid across the domains. All the current file systems being migrated are NFSv3 with AUTH_SYS. I'd consider moving them all to kerberos authentication, but something tells me that may be impossible with the multiple domains. -Chip On Mon, Dec 8, 2014 at 3:42 PM, Paul B. Henson wrote: > > > From: Ian Kaufman > > Sent: Monday, December 08, 2014 12:37 PM > > > > Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX > > does not support NFSv4. AUTH_SYS uses the UID, not the name, so the > > mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL. > > I apologize for being a pedant; it's a character flaw :). Your wording > implies that you cannot use AUTH_SYS with NFSv4, which is not true. NFSv4 > works perfectly fine with AUTH_SYS as long as you maintain synchronization > of uid/gid's on both sides. I would pick NFSv4 with AUTH_SYS over NFSv3 > with AUTH_SYS. Both require that you maintain uid synchronization, so it's > not like you're gaining something by falling back, and why miss out on the > other features of NFSv4? > > > Supposedly. at least on the client side, this has been fixed somewhere > > upstream. However, the server side is not. > > > > https://bugzilla.linux-nfs.org/show_bug.cgi?id=226 > > I don't know if I would call this "fixed" 8-/. They are basically just > disabling the idmapper and passing raw uid/gid info at the NFS level to > match the raw info at the RPC level. I guess it makes it less confusing in > such a scenario, because you're always broken if they don't match on each > side rather than only broken in some cases. An actual fix would be > introducing an RPC mechanism using names such as AUTH_SYS_NAME, one of > these days maybe I'll find the time to go hassle some NFS developers and > see why they don't just do that and make everything simpler. > > > Regardless, my point is that this is not a Solaris/Linux issue, as a > > Linux server and Linux clients would be in the same boat. > > Agreed. It is a deficiency in the NFSv4 protocol :(... > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cscoman at gmail.com Tue Dec 16 15:12:43 2014 From: cscoman at gmail.com (Jason Cox) Date: Tue, 16 Dec 2014 07:12:43 -0800 Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. In-Reply-To: <1418681831295.66175@steait.net> References: <1418676240949.15076@steait.net> <548F4E9B.6060904@gmx.li> <1418681831295.66175@steait.net> Message-ID: I have found that turning on compression gets more savings when storing my backups then I got with dedup on. I run gzip-9 for my backup dataset. I think dedup is good if you are storing a lot of files that really are the same. Your mileage may vary compared to mine. I do a full backup once a week and incrementals daily. On Mon, Dec 15, 2014 at 2:17 PM, Rune Tipsmark wrote: > > we have only 24GB of ram on this system... > I was under the impression it would not require much when the block size > was larger, we are on 64kb, so would expect around 2GB per 1TB. > yet we cannot even get more than a few TB on the system before it dies. > > the main purpose with this system was dedup, no problem with slow speed... > its for backup only, so could not care less if its rather slow - but not > responding is not acceptable. > br, > Rune > ________________________________________ > From: OmniOS-discuss on behalf > of Dominik Hassler > Sent: Monday, December 15, 2014 10:11 PM > To: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] dedup causes zfs/omnios to drop connections. > > Hi, > > we used dedup on a production machine w/ 256 GB RAM, but disabled it > after a couple of days due to huge performance impact. > > I would not recommend to use dedup even when having "enough" RAM. > > On 12/15/2014 09:53 PM, Dan McDonald wrote: > > > >> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark wrote: > >> > >> hi all, > >> > >> got a new system I was intending on using as backup repository. > Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. > I need to reboot OmniOS to get it back online. > >> the files being copied onto the zfs vols are rather large, about ~2TB > each... if I copy smaller files, say 400GB or so, it takes longer for it to > crash. > >> > >> what can be done to fix this? after Windows (initiator) looses the > connection (both Fibre Channel and iSCSI) I still see a lot of disk > activity using iostat - disks remain active for minutes after the copying > has died... its like ZFS cannot handle dedup of large files.. > > > > Dedup is a memory pig and not very well implemented in ZFS. I'd highly > recommend against it in production. Either that, or really increase your > memory for your system in question. There was some work going on at > Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like > device, but I believe that work stalled. > > > > Sorry, > > Dan > > > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Jason Cox -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Dec 16 19:33:23 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 16 Dec 2014 14:33:23 -0500 Subject: [OmniOS-discuss] pull request: revert a6f10d64 In-Reply-To: <5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com> References: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com> <5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com> Message-ID: > On Dec 15, 2014, at 9:10 PM, Dan McDonald wrote: > > >> On Dec 15, 2014, at 4:22 PM, Dan McDonald wrote: >>> >>> https://gist.github.com/risto3/adf3f9068eff759f06a7 >> >> I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix. If it's silent, then I'm okay taking this in. >> >> If there's historical reasons this happened originally, I'd love to hear them on this thread. > > illumos-omnios build went through w/o a hitch. > > Unless I hear strong objections, I'm taking this one in tomorrow (Tuesday). It's in now: https://github.com/omniti-labs/illumos-omnios/commit/0c8258bb76d866ec46d30d1b3429fbd766237ef6 Dan From henson at acm.org Wed Dec 17 00:27:35 2014 From: henson at acm.org (Paul B. Henson) Date: Tue, 16 Dec 2014 16:27:35 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> <674b01d01324$d0e28180$72a78480$@acm.org> <6e8301d0132f$e1702c40$a45084c0$@acm.org> Message-ID: > From: Schweiss, Chip > Sent: Tuesday, December 16, 2014 6:02 AM > > It seems there a many ways to map ID in NFSv4, is there a way to not map > them at all? I believe linux supports disabling ID mapping and using raw uid/gids on the wire instead of strings, but I don't think illumos does? > All the current file systems being migrated are NFSv3 with AUTH_SYS. I'd > consider moving them all to kerberos authentication, but something tells me > that may be impossible with the multiple domains. Multiple Kerberos realms too? I don't think illumos can have more than one kerberos realm defined for NFS... From ikaufman at eng.ucsd.edu Wed Dec 17 00:29:39 2014 From: ikaufman at eng.ucsd.edu (Ian Kaufman) Date: Tue, 16 Dec 2014 16:29:39 -0800 Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not server? In-Reply-To: References: <20141204193711.03524183@sleipner.datanom.net> <20141207224353.GA13916@warlock.deepthought.com> <674b01d01324$d0e28180$72a78480$@acm.org> <6e8301d0132f$e1702c40$a45084c0$@acm.org> Message-ID: There has been some work on multi-domain Kerberos in NFSv4, going back to 2010. Not sure where things stand though. https://www.ietf.org/proceedings/10mar/slides/nfsv4-5.pdf Ian On Tue, Dec 16, 2014 at 4:27 PM, Paul B. Henson wrote: >> From: Schweiss, Chip >> Sent: Tuesday, December 16, 2014 6:02 AM >> >> It seems there a many ways to map ID in NFSv4, is there a way to not map >> them at all? > > I believe linux supports disabling ID mapping and using raw uid/gids on the wire instead of strings, but I don't think illumos does? > >> All the current file systems being migrated are NFSv3 with AUTH_SYS. I'd >> consider moving them all to kerberos authentication, but something tells me >> that may be impossible with the multiple domains. > > Multiple Kerberos realms too? I don't think illumos can have more than one kerberos realm defined for NFS... > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu From tim at multitalents.net Wed Dec 17 06:31:10 2014 From: tim at multitalents.net (Tim Rice) Date: Tue, 16 Dec 2014 22:31:10 -0800 (PST) Subject: [OmniOS-discuss] anyone doing DirectPath I/O? In-Reply-To: <54807EC4.6020201@kateley.com> References: <54807EC4.6020201@kateley.com> Message-ID: On Thu, 4 Dec 2014, Linda Kateley wrote: | Just curious, how fast is the nfs in this config? Is fast enough a good enough answer? :-) The only actual hard data I have (over any length of time) is what my backup program reports. So picture 3 evolving configurations. First the system common to all 3, a Solaris Express Community Edition box with 4 SATA drives in a raidz configuration. That's where the backups get written via ftp. (Yes I know you asked about NFS, we'll get to that) Now to establish a baseline, when the machine being backed up, a UnixWare 7.1.4 box, was running native backups would average 29.85G/hr and verify would average 49.10G/hr. Now we introduce the all in one box mentioned below. The OmniOS r151006 VM is providing NFS storage to it's ESXi host for the UnixWare 7.1.4 VM. Both VMs using E1000 nics. Now backups are averaging 31.19G/hr and verify 50.44G/hr. This is with no compression (on the UnixWare backup program) so the increase in CPU power would not be helping there. So obviously NFS is not getting in the way here. Faster then the native 6 drive raid in the HP DL385. Now let's say your ESXi was a little behind in it's patches and you load all the current patches. Oops. Your entire system falls on it's face. Network throughput is at least an order of magnitude slower. More info here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2078636 http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2092809 So after switching to vmxnet3 nics on the OmniOS VM and reconfiguring, things were back to normal. Actually it seemed better than before. The current numbers seem to confirm this. Backups are averaging 33.32G/hr and verify 56.42G/hr. Now what I have noticed over the life of this system (installed July 2013) although not recently, is what another poster described where from time to time everything seems to lock up for about 5 to 15 seconds. I hope this helps answer your question. | | linda | | On 12/4/14, 1:11 AM, Tim Rice wrote: | > On Wed, 3 Dec 2014, Schweiss, Chip wrote: | > | > | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 | > years | > | to an OpenIndiana server. It's been very stable: | > | > My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for | > about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since | > updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS | > storage to the ESXi host for (currently) 15 running VMS as well | > as well as NFS home directories for various other computers around | > the office. Other than drives failing, it's been working reasonably | > well for what it is. | > | > | | > | root at zfs01:~# uptime | > | 21:01pm up 649 days 6:58, 2 users, load average: 0.46, 0.30, 0.25 | > | | > | | > | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren | > wrote: | > | | > | > Greetings, | > | > | > | > I was wondering if anyone was using DirectPath, specifically for | > exclusive | > | > use of a drive controller and attached drives to a specific VM. I | > have a | > | > use case that would seem to be a good fit for this, so I played around | > with | > | > a couple of RAID controllers I had, and was able to get one (3ware | > 9650se) | > | > configured for directpath, but none of the drives attached would show to | > | > OmniOS, regardless of how I configured them in the raid bois (JBOD, | > | > individual disks, etc). I know that controller is poorly supported and | > I | > | > was curious if anyone was using DirectPath this way in production and | > what | > | > kind of drive controller/HBA/whatever was working. Also any "for the | > love | > | > of god don't do it this way" scenarios? I seem to be really adept at | > | > finding and trying those out first.... | > | > | > | > Thanks a ton and best regards, | > | > | > | > -jb- | > | > *Joseph Boren* | > | > | > | > IT Specialist | > | > *DRAKE COOPER* | > | > + c: (208) 891-2128 + o: (208) 342-0925 | > | > + 416 S. 8th St., Boise, ID 83702 | > | > + w: drakecooper.com + f: /drakecooper | > + | > | > t: @drakecooper | > | > | > | > | > | > _______________________________________________ | > | > OmniOS-discuss mailing list | > | > OmniOS-discuss at lists.omniti.com | > | > http://lists.omniti.com/mailman/listinfo/omnios-discuss | > | > | > | > | > | | > | | _______________________________________________ | OmniOS-discuss mailing list | OmniOS-discuss at lists.omniti.com | http://lists.omniti.com/mailman/listinfo/omnios-discuss | | | -- Tim Rice Multitalents (707) 456-1146 tim at multitalents.net From mark0x01 at gmail.com Wed Dec 17 06:44:17 2014 From: mark0x01 at gmail.com (Mark) Date: Wed, 17 Dec 2014 19:44:17 +1300 Subject: [OmniOS-discuss] Fibre Target problems In-Reply-To: <1418647518088.75378@steait.net> References: <541155D5.8080406@gmail.com> <562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>, <5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>, <548E7D66.6080208@gmail.com> <1418647518088.75378@steait.net> Message-ID: <54912641.9020402@gmail.com> On 16/12/2014 1:45 a.m., Rune Tipsmark wrote: > where do you check that? > br, > Rune > ________________________________________ > From: Mark > Sent: Monday, December 15, 2014 7:19 AM > To: Rune Tipsmark; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] Fibre Target problems > > On 15/12/2014 4:44 a.m., Rune Tipsmark wrote: >> did you ever find a solution? >> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... >> br, >> Rune > > I installed an older OpenIndiana version to work around the issue. > > One thing I didn't check was if the fc target cache was disabled (may be > the current default), so may be worth checking that. > > Mark. > To check stmfadm list-lu -v LU Name: 600144F0EBE24300000051118E770001 Operational Status: Online Provider Name : sbd Alias : /dev/zvol/rdsk/dpool/vol1 View Entry Count : 1 Data File : /dev/zvol/rdsk/dpool/vol1 Meta File : not set Size : 2147483648 Block Size : 512 Management URL : not set Vendor ID : OI Product ID : COMSTAR Serial Num : not set Write Protect : Disabled Writeback Cache : Enabled <<<<<<<<<<<<<<< Access State : Active you can change properties with stmfadm modify-lu I'm not sure of the exact syntax, so check the man page. From tobi at oetiker.ch Wed Dec 17 07:54:39 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 17 Dec 2014 08:54:39 +0100 (CET) Subject: [OmniOS-discuss] ipadm issues Message-ID: I seem to run into trouble with ipadm ... here is the latest root at simplon:~# ipadm show-addr ADDROBJ TYPE STATE ADDR lo0/v4 static ok 127.0.0.1/8 mplon0/v4static static ok 118.124.138.203/28 mplon0/v4intern static ok 192.168.1.4/24 lo0/v6 static ok ::1/128 mplon0/v6 addrconf disabled :: mplon0/v6a static disabled 2001:2620:102d::cb/64 root at mplon:~# ipadm enable-addr -t mplon0/v6 ipadm: could not enable address: Object not found root at mplon:~# ipadm enable-addr -t mplon0/v6a ipadm: could not enable address: Object not found I was able to solve this to an extent by removing the ip6 stuff from /etc/ipadm.conf, and then running ipdam again but it would not let me add permanent config: root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6a ipadm: Could not create address: Persistent operation on temporary object At least adding a temporary address worked: ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a how to make ipadm see the light ? cheers tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From johan.kragsterman at capvert.se Wed Dec 17 10:02:29 2014 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Wed, 17 Dec 2014 11:02:29 +0100 Subject: [OmniOS-discuss] Ang: Re: Fibre Target problems In-Reply-To: <54912641.9020402@gmail.com> References: <54912641.9020402@gmail.com>, <541155D5.8080406@gmail.com> <562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>, <5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>, <548E7D66.6080208@gmail.com> <1418647518088.75378@steait.net> Message-ID: Hi! -----"OmniOS-discuss" skrev: ----- Till: Rune Tipsmark , "omnios-discuss at lists.omniti.com" Fr?n: Mark S?nt av: "OmniOS-discuss" Datum: 2014-12-17 07:45 ?rende: Re: [OmniOS-discuss] Fibre Target problems On 16/12/2014 1:45 a.m., Rune Tipsmark wrote: > where do you check that? > br, > Rune > ________________________________________ > From: Mark > Sent: Monday, December 15, 2014 7:19 AM > To: Rune Tipsmark; omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] Fibre Target problems > > On 15/12/2014 4:44 a.m., Rune Tipsmark wrote: >> did you ever find a solution? >> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... >> br, >> Rune > > I installed an older OpenIndiana version to work around the issue. > > One thing I didn't check was if the fc target cache was disabled (may be > the current default), so may be worth checking that. > > Mark. > To check stmfadm list-lu -v LU Name: 600144F0EBE24300000051118E770001 ?? ? Operational Status: Online ?? ? Provider Name ? ? : sbd ?? ? Alias ? ? ? ? ? ? : /dev/zvol/rdsk/dpool/vol1 ?? ? View Entry Count ?: 1 ?? ? Data File ? ? ? ? : /dev/zvol/rdsk/dpool/vol1 ?? ? Meta File ? ? ? ? : not set ?? ? Size ? ? ? ? ? ? ?: 2147483648 ?? ? Block Size ? ? ? ?: 512 ?? ? Management URL ? ?: not set ?? ? Vendor ID ? ? ? ? : OI ?? ? Product ID ? ? ? ?: COMSTAR ?? ? Serial Num ? ? ? ?: not set ?? ? Write Protect ? ? : Disabled ?? ? Writeback Cache ? : Enabled ? ? ? ? ? ? ?<<<<<<<<<<<<<<< ?? ? Access State ? ? ?: Active you can change properties with stmfadm modify-lu I'm not sure of the exact syntax, so check the man page. I've followed this thread, since it is in my line of work. I'd be interested to know the progress of this. Are you guys still having this problem? And about the setup: The hosts seem to be windows, right? Have you seen this problem with other host OS's? Are they windows on hardware, or windows VM's? If VM's, which hypervisor and are the LU's presented directly to the VM's or through the hypervisor? Do you use NPIV? Other interesting information? Regards Johan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From jesus at omniti.com Wed Dec 17 13:48:57 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Wed, 17 Dec 2014 08:48:57 -0500 Subject: [OmniOS-discuss] ipadm issues In-Reply-To: References: Message-ID: I think you have to have an addrconf ipv6 first, then you can add the static one. ipadm create-addr -T addrconf mplon0/v6a ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6static On Wed, Dec 17, 2014 at 2:54 AM, Tobias Oetiker wrote: > > I seem to run into trouble with ipadm ... > > here is the latest > > root at simplon:~# ipadm show-addr > ADDROBJ TYPE STATE ADDR > lo0/v4 static ok 127.0.0.1/8 > mplon0/v4static static ok 118.124.138.203/28 > mplon0/v4intern static ok 192.168.1.4/24 > lo0/v6 static ok ::1/128 > mplon0/v6 addrconf disabled :: > mplon0/v6a static disabled 2001:2620:102d::cb/64 > root at mplon:~# ipadm enable-addr -t mplon0/v6 > ipadm: could not enable address: Object not found > root at mplon:~# ipadm enable-addr -t mplon0/v6a > ipadm: could not enable address: Object not found > > I was able to solve this to an extent by removing the ip6 stuff > from /etc/ipadm.conf, and then running ipdam again but it would not > let me add permanent config: > > root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64 > mplon0/v6a > ipadm: Could not create address: Persistent operation on temporary object > > At least adding a temporary address worked: > > ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a > > how to make ipadm see the light ? > > cheers > tobi > > -- > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobi at oetiker.ch Wed Dec 17 13:52:50 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 17 Dec 2014 14:52:50 +0100 (CET) Subject: [OmniOS-discuss] ipadm issues In-Reply-To: References: Message-ID: Today Theo Schlossnagle wrote: > I think you have to have an addrconf ipv6 first, then you can add the > static one. > > ipadm create-addr -T addrconf mplon0/v6a > ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6static oh ... I did that ... sorry ... not mentioned ... same problem ... ipadm only accepts the setting with the -t switch cheers tobi > > On Wed, Dec 17, 2014 at 2:54 AM, Tobias Oetiker wrote: > > > > I seem to run into trouble with ipadm ... > > > > here is the latest > > > > root at simplon:~# ipadm show-addr > > ADDROBJ TYPE STATE ADDR > > lo0/v4 static ok 127.0.0.1/8 > > mplon0/v4static static ok 118.124.138.203/28 > > mplon0/v4intern static ok 192.168.1.4/24 > > lo0/v6 static ok ::1/128 > > mplon0/v6 addrconf disabled :: > > mplon0/v6a static disabled 2001:2620:102d::cb/64 > > root at mplon:~# ipadm enable-addr -t mplon0/v6 > > ipadm: could not enable address: Object not found > > root at mplon:~# ipadm enable-addr -t mplon0/v6a > > ipadm: could not enable address: Object not found > > > > I was able to solve this to an extent by removing the ip6 stuff > > from /etc/ipadm.conf, and then running ipdam again but it would not > > let me add permanent config: > > > > root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64 > > mplon0/v6a > > ipadm: Could not create address: Persistent operation on temporary object > > > > At least adding a temporary address worked: > > > > ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a > > > > how to make ipadm see the light ? > > > > cheers > > tobi > > > > -- > > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland > > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From danmcd at omniti.com Wed Dec 17 15:47:19 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 10:47:19 -0500 Subject: [OmniOS-discuss] ipadm issues In-Reply-To: References: Message-ID: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> > On Dec 17, 2014, at 2:54 AM, Tobias Oetiker wrote: > > I seem to run into trouble with ipadm ... > > here is the latest Silly question --> what does "ipadm show-if" say? Dan From danmcd at omniti.com Wed Dec 17 16:00:18 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 11:00:18 -0500 Subject: [OmniOS-discuss] ipadm issues In-Reply-To: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> References: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> Message-ID: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> > On Dec 17, 2014, at 10:47 AM, Dan McDonald wrote: > > >> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker wrote: >> >> I seem to run into trouble with ipadm ... >> >> here is the latest > > Silly question --> what does "ipadm show-if" say? Also, I noticed this: mplon0/v6 addrconf disabled :: Try this: ipadm enable-if mplon0/v6 Dan From danmcd at omniti.com Wed Dec 17 16:15:12 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 11:15:12 -0500 Subject: [OmniOS-discuss] ipadm issues In-Reply-To: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> References: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> Message-ID: <4B6A601F-8DE0-47AB-84AD-0C3ABC47C559@omniti.com> > On Dec 17, 2014, at 11:00 AM, Dan McDonald wrote: > > >> On Dec 17, 2014, at 10:47 AM, Dan McDonald wrote: >> >> >>> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker wrote: >>> >>> I seem to run into trouble with ipadm ... >>> >>> here is the latest >> >> Silly question --> what does "ipadm show-if" say? > > Also, I noticed this: > > mplon0/v6 addrconf disabled :: Yeah... it's the disabled part that's likely getting you. I tried some experiments on one of my VMs, and once things were "enabled" life got a lot better. Dan From jimklimov at cos.ru Wed Dec 17 21:16:37 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Wed, 17 Dec 2014 22:16:37 +0100 Subject: [OmniOS-discuss] latency spikes ~every hour In-Reply-To: <1418646462467.87062@steait.net> References: <1418563662763.71128@steait.net> <1418646462467.87062@steait.net> Message-ID: <7429C727-AFAF-4C6E-9EA6-C27B4F291CA2@cos.ru> 15 ??????? 2014??. 13:27:39 CET, Rune Tipsmark ?????: >ok I removed some of my SLOG devices and currently I am only using a >single SLOG (no mirror or anything) and no spikes seen since. > >I wonder why multiple SLOG devices would cause this. > >br. > >Rune > >________________________________ >From: OmniOS-discuss on >behalf of Rune Tipsmark >Sent: Sunday, December 14, 2014 2:27 PM >To: omnios-discuss at lists.omniti.com >Subject: [OmniOS-discuss] latency spikes ~every hour > > >hi all, > > > >All my vSphere (ESXi5.1) hosts experience a big spike in latency every >hour or so. > >I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All >exhibit the same behavior so I don't think its the connection that is >causing this. > >When I modify the arc_shrink_shift 10 (192GB Ram in the System) it >helps a bit, the spikes are still with the same regularity but latency >peaks at about ~5000ms for the most part. if I leave arc_shrink_shift >at the default value they can be higher - up to 15000ms. > >Looking at the latency as seen from the vSphere hosts the average is >usually below 1ms for most datastores. > > > >Any ideas what can cause this or what can be done to fix? > >br, > >Rune > > >------------------------------------------------------------------------ > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss One idea stems from old experience: there used to be two triggers to start a TXG sync, where cached (async) writes do actually get to disk: a timeout and a filled-up buffer. While the sync is underway, a system indeed feels locked, new i/o's are delayed, etc. In such cases the tuning was to decrease time or rather size limits, to make the txg syncs shorter and faster, though at risk of more fragmentation of on-disk data. This is easily seen in iostat or zpool iostat as a considerable spike in write traffic (with your ram size, the metadata is probably cached and there are not many reads needed to complete the write), occurring regularly (i.e. 5secs). Your hourly hiccups = backups starting up, maybe? It may be that the mechanism or the tuning handles were changed in the past year or two in illumos-zfs, but it may be that your (unnamed?) distrp still has that behavior. -- Typos courtesy of K-9 Mail on my Samsung Android From tobi at oetiker.ch Wed Dec 17 23:30:58 2014 From: tobi at oetiker.ch (Tobias Oetiker) Date: Thu, 18 Dec 2014 00:30:58 +0100 (CET) Subject: [OmniOS-discuss] ipadm issues In-Reply-To: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> References: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> Message-ID: Hi Dan, Yesterday Dan McDonald wrote: > > > On Dec 17, 2014, at 10:47 AM, Dan McDonald wrote: > > > > > >> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker wrote: > >> > >> I seem to run into trouble with ipadm ... > >> > >> here is the latest > > > > Silly question --> what does "ipadm show-if" say? > > Also, I noticed this: > > mplon0/v6 addrconf disabled :: > > Try this: > > ipadm enable-if mplon0/v6 I tried enable-addr ... maybe that was the problem ... since I have now setup the temporary interface ... I can't test this ... on another note though ... as I setup addr conf, I got both the linklocal address as well as a selfassigned global address ... how can I tell ipadm NOT todo that and to be happy with the linklocal address, since I am assigning a fixed address by hand cheers tobi > Dan > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch tobi at oetiker.ch +41 62 775 9902 From danmcd at omniti.com Wed Dec 17 23:55:27 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 18:55:27 -0500 Subject: [OmniOS-discuss] ipadm issues In-Reply-To: References: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com> <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com> Message-ID: <7D2BC64A-F3BC-4F86-9A97-B119A0646927@omniti.com> > On Dec 17, 2014, at 6:30 PM, Tobias Oetiker wrote: > > Hi Dan, > > Yesterday Dan McDonald wrote: > >> >>> On Dec 17, 2014, at 10:47 AM, Dan McDonald wrote: >>> >>> >>>> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker wrote: >>>> >>>> I seem to run into trouble with ipadm ... >>>> >>>> here is the latest >>> >>> Silly question --> what does "ipadm show-if" say? >> >> Also, I noticed this: >> >> mplon0/v6 addrconf disabled :: >> >> Try this: >> >> ipadm enable-if mplon0/v6 > > I tried enable-addr ... maybe that was the problem ... > since I have now setup the temporary interface ... I can't test > this ... > > on another note though ... > > as I setup addr conf, I got both the linklocal address as well as a > selfassigned global address ... how can I tell ipadm NOT todo that > and to be happy with the linklocal address, since I am assigning a > fixed address by hand The ipadm(1M) page says: ipadm create-addr [-t] -T addrconf [-i interface-id] [-p {stateful|stateless}={yes|no},..] addrobj The -p option (also --prop) indicates which method of auto-configuration should be used. I believe you'll want to disable both stateful and stateless, or at the very least, stateful (which grabs prefixes via in.ndpd). Dan From danmcd at omniti.com Thu Dec 18 02:44:34 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 21:44:34 -0500 Subject: [OmniOS-discuss] Followon review --> Changes that allow illumos-gate to build on OmniOS Message-ID: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> Now that the java fixes are their own thing, let's revisit the remaining changes in: http://kebe.com/~danmcd/webrevs/for-omnios/ All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated. The aforementioned directory contains: check-rtime/ * Some check_rtime modifications for the version of things in OmniOS ig-on-omnios.env * A sample .env file used for building illumos-gate on OmniOS. Look for the extra export variables you'll need. java-updates * Now points to illumos 4719, see previous mail. lose-ipp-listener * Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS. * See the sample .env file above for how it works. lose-shell-styleguide * Lose the dependency on docbooks, only existing in the Shell Style Guide. * Apparently not only is this out in OmniOS already, but in SmartOS as well. lose-sysidtool-dep * Lose the SMF dependency on sysidtool. * I've been told this only ever affected people trying to move from old OpenSolaris to OI. modern-netsmnp * Allow linkage with more recent versions of net-snmp. more-lintfixes * Lint fixes that the OmniOS sunstudio12.1 lint are picky about. * Work done by Rich Lowe. Thank you, Rich! omnios-mail-msg.txt * See the small number of remaining lint errors, ALL having to do with openssl versions. perl-fixes * More perl fixes to work with OmniOS's alternative approach to perl support. * This has also been tested, I believe, by the OpenIndiana Hipster people. After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well. They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well. Thanks, Dan McDonald From danmcd at omniti.com Thu Dec 18 02:45:52 2014 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 17 Dec 2014 21:45:52 -0500 Subject: [OmniOS-discuss] Followon review --> Changes that allow illumos-gate to build on OmniOS In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> Message-ID: <0054A484-17B3-4B6E-821F-6056388E10AD@omniti.com> Two other things: 1.) I believe these changes could go back independently of 4719, but without the benefit of helping OmniOS directly. 2.) Also, I understand I'll need to file bugs for these diffs. Thanks, Dan From stephan.budach at JVM.DE Thu Dec 18 07:54:05 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Thu, 18 Dec 2014 08:54:05 +0100 Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on r012 Message-ID: <5492881D.5060504@jvm.de> Good morning, I wanted to install gcc to compile znapzend on my r012 box, so I issued: root at nfsvmpool02:/root# pkg install developer/gcc48 Packages to install: 6 Create boot environment: No Create backup boot environment: No DOWNLOAD PKGS FILES XFER (MB) Completed 6/6 1537/1537 165.1/165.1 PHASE ACTIONS Install Phase 2104/2104 PHASE ITEMS Package State Update Phase 6/6 Image State Update Phase 2/2 root at nfsvmpool02:/root# pkg install gnu-make Packages to install: 1 Create boot environment: No Create backup boot environment: No DOWNLOAD PKGS FILES XFER (MB) Completed 1/1 31/31 0.5/0.5 PHASE ACTIONS Install Phase 98/98 PHASE ITEMS Package State Update Phase 1/1 Image State Update Phase 2/2 However, gcc is not available afterwards: root at nfsvmpool02:/root# gcc -bash: gcc: command not found I did the same on my r006 box using pkg install developer/gcc47, which installed gcc as expected. What did I do wrong? ;) Thanks, budy From henson at acm.org Thu Dec 18 08:15:39 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 18 Dec 2014 00:15:39 -0800 Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on r012 In-Reply-To: <5492881D.5060504@jvm.de> References: <5492881D.5060504@jvm.de> Message-ID: <3268807D-987E-459B-AB4D-D651121F8801@acm.org> gcc on omnios isn't installed in the default path, you need to add /opt/gccXX/bin to your path. I'm pretty sure it's always been like that, so if you installed and used gcc on an earlier version, it's likely you added it to your path to do so. > On Dec 17, 2014, at 11:54 PM, Stephan Budach wrote: > > Good morning, > > I wanted to install gcc to compile znapzend on my r012 box, so I issued: > > root at nfsvmpool02:/root# pkg install developer/gcc48 > Packages to install: 6 > Create boot environment: No > Create backup boot environment: No > > DOWNLOAD PKGS FILES XFER (MB) > Completed 6/6 1537/1537 165.1/165.1 > > PHASE ACTIONS > Install Phase 2104/2104 > > PHASE ITEMS > Package State Update Phase 6/6 > Image State Update Phase 2/2 > root at nfsvmpool02:/root# pkg install gnu-make > Packages to install: 1 > Create boot environment: No > Create backup boot environment: No > > DOWNLOAD PKGS FILES XFER (MB) > Completed 1/1 31/31 0.5/0.5 > > PHASE ACTIONS > Install Phase 98/98 > > PHASE ITEMS > Package State Update Phase 1/1 > Image State Update Phase 2/2 > > > However, gcc is not available afterwards: > > root at nfsvmpool02:/root# gcc > -bash: gcc: command not found > > I did the same on my r006 box using pkg install developer/gcc47, which installed gcc as expected. > What did I do wrong? ;) > > Thanks, > budy > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From ml.omnios-discuss at valo.at Thu Dec 18 08:53:52 2014 From: ml.omnios-discuss at valo.at (Christian Kivalo) Date: Thu, 18 Dec 2014 09:53:52 +0100 Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on r012 In-Reply-To: <5492881D.5060504@jvm.de> References: <5492881D.5060504@jvm.de> Message-ID: Good Morning, I ran into the exact same problem yesterday and found this very informative page about setting up a basic dev environment on OmniOS: Link http://omnios.omniti.com/wiki.php/DevEnv I added the gcc bin dir to my $PATH and installed all the packages mentioned on that site and znapzend compiled. -christian On Thu, 18 Dec 2014 08:54:05 +0100 Stephan Budach wrote > Good morning, > > I wanted to install gcc to compile znapzend on my r012 box, so I issued: > > root at nfsvmpool02:/root# pkg install developer/gcc48 > Packages to install: 6 > Create boot environment: No > Create backup boot environment: No > > DOWNLOAD PKGS FILES XFER (MB) > Completed 6/6 1537/1537 165.1/165.1 > > PHASE ACTIONS > Install Phase 2104/2104 > > PHASE ITEMS > Package State Update Phase 6/6 > Image State Update Phase 2/2 > root at nfsvmpool02:/root# pkg install gnu-make > Packages to install: 1 > Create boot environment: No > Create backup boot environment: No > > DOWNLOAD PKGS FILES XFER (MB) > Completed 1/1 31/31 0.5/0.5 > > PHASE ACTIONS > Install Phase 98/98 > > PHASE ITEMS > Package State Update Phase 1/1 > Image State Update Phase 2/2 > > > However, gcc is not available afterwards: > > root at nfsvmpool02:/root# gcc > -bash: gcc: command not found > > I did the same on my r006 box using pkg install developer/gcc47, which > installed gcc as expected. > What did I do wrong? ;) > > Thanks, > budy > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From stephan.budach at JVM.DE Thu Dec 18 10:42:35 2014 From: stephan.budach at JVM.DE (Stephan Budach) Date: Thu, 18 Dec 2014 11:42:35 +0100 Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on r012 In-Reply-To: References: <5492881D.5060504@jvm.de> Message-ID: <5492AF9B.8020907@jvm.de> Am 18.12.14 um 09:53 schrieb Christian Kivalo: > Good Morning, > > I ran into the exact same problem yesterday and found this very informative > page about setting up a basic dev environment on OmniOS: > Link http://omnios.omniti.com/wiki.php/DevEnv > > I added the gcc bin dir to my $PATH and installed all the packages mentioned on > that site and znapzend compiled. > > -christian Thanks - it seems, that I rushed over those extra packages. Thanks for the heads-up! After actually completing the installation of the dev tools, I am also able to compile znapzend, which is a very neat tool, btw.! Thanks, budy > > On Thu, 18 Dec 2014 08:54:05 +0100 Stephan Budach wrote > >> Good morning, >> >> I wanted to install gcc to compile znapzend on my r012 box, so I issued: >> >> root at nfsvmpool02:/root# pkg install developer/gcc48 >> Packages to install: 6 >> Create boot environment: No >> Create backup boot environment: No >> >> DOWNLOAD PKGS FILES XFER (MB) >> Completed 6/6 1537/1537 165.1/165.1 >> >> PHASE ACTIONS >> Install Phase 2104/2104 >> >> PHASE ITEMS >> Package State Update Phase 6/6 >> Image State Update Phase 2/2 >> root at nfsvmpool02:/root# pkg install gnu-make >> Packages to install: 1 >> Create boot environment: No >> Create backup boot environment: No >> >> DOWNLOAD PKGS FILES XFER (MB) >> Completed 1/1 31/31 0.5/0.5 >> >> PHASE ACTIONS >> Install Phase 98/98 >> >> PHASE ITEMS >> Package State Update Phase 1/1 >> Image State Update Phase 2/2 >> >> >> However, gcc is not available afterwards: >> >> root at nfsvmpool02:/root# gcc >> -bash: gcc: command not found >> >> I did the same on my r006 box using pkg install developer/gcc47, which >> installed gcc as expected. >> What did I do wrong? ;) >> >> Thanks, >> budy >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Stephan Budach Deputy Managing Director Jung von Matt/it-services GmbH Glash?ttenstra?e 79 20357 Hamburg Tel: +49 40-4321-1353 Fax: +49 40-4321-1114 E-Mail: stephan.budach at jvm.de Internet: http://www.jvm.com Gesch?ftsf?hrer: Frank Wilhelm, Stephan Budach (stellv.) AG HH HRB 98380 From johan.kragsterman at capvert.se Thu Dec 18 15:57:59 2014 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 18 Dec 2014 16:57:59 +0100 Subject: [OmniOS-discuss] CoreOS Message-ID: Hi! I've been looking at CoreOS and finds it interesting! Since I'd like to have OmniOS as the platform, I need to run CoreOS as a KVM guest. Haven't tested yet, but I downloaded the startscript for qemu, and it looks a little bit "too much" for Illumos KVM... It would be nice to get some views on people that have been considering this as well, perhaps some already tested or already running...? I've seen that Frederic Alix on this list been blogging about it, but haven't seen if he managed to run it as a KVM guest on OmniOS. For me it seems to be some complications at first startup, mainly. It doesn't seem to be reachable by VNC... Hope to get some input from you guys... Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert From doug at will.to Thu Dec 18 16:26:40 2014 From: doug at will.to (Doug Hughes) Date: Thu, 18 Dec 2014 11:26:40 -0500 Subject: [OmniOS-discuss] latent problem with networking Message-ID: the machine has been up for a while. It seems like something funky is going on in the network stack. it pings ok.. ssh inbound new sessions no longer work. The master ssh process isn't bound to port 22 any more (pfiles). Active ssh session to this machine continues to work, however. Also, pkg install gets stuck, looks like on DNS. But the DNS server is responsive from other hosts, and I can see all request/response from the client on the DNS server: the client omniOS machine cannot see the responses from the server, though. Ping between omnios machine and DNS server is fully functional (initiated from either side). There is no firewall between them. I can see existing ssh packets (pre-existing this strange new condition) going back and forth quite well (also to DNS server) - demonstrating that there is no intervening network problem. I can see tcp/udp packets reaching the server from the omniOS machine and responses going back (but not showing to snoop on omniOS) Here's the simplest test... I start up ttcp -r on the server, it binds to port 5001, listening. I run snoop.. Then I try to connect to 5001 from another machine. I see the packets in snoop, but the accept call on the omniOS machine never returns. Something seems wonky in network land. Has anybody seen this? THe machine has been up for weeks without any problems. OmniOS v11 r151012 Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved. Use is subject to license terms. Regular/plain Intel chipset: e1000g0: root at xyr-r:/root# dladm show-link e1000g0 LINK CLASS MTU STATE BRIDGE OVER e1000g0 phys 1500 up -- -- root at xyr-r:/root# dladm show-phys e1000g0 LINK MEDIA STATE SPEED DUPLEX DEVICE e1000g0 Ethernet up 1000 full e1000g0 e1000 prtdiag excerpt: name='device-name' type=string items=1 value='82574L Gigabit Network Connection' name='subsystem-name' type=string items=1 value='unknown subsystem' Device Minor Nodes: dev=(112,1) dev_path=/pci at 0,0/pci8086,1d14 at 1c,2/pci122e,10d3 at 0 :e1000g0 Ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjorge+ml at blackdot.be Thu Dec 18 16:37:17 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Thu, 18 Dec 2014 17:37:17 +0100 Subject: [OmniOS-discuss] CoreOS In-Reply-To: References: Message-ID: Something like this will probably work: /usr/bin/qemu-system-x86_64 -name coreos \ -enable-kvm \ -no-hpet \ -m 4096 -cpu Nehalem \ -smp sockets=1,cores=4,threads=2 \ -rtc base=utc,driftfix=slew \ -pidfile /tank/coreo/coreos.pid \ -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay \ -vga std \ -vnc :1 \ -nographic \ -drive file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \ -drive file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on \ -boot order=cd,once=d \ -device virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 \ -net vnic,vlan=0,name=net1,ifname=vcoreos0 \ -chardev socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \ -serial chardev:serial0 \ -usb \ -usbdevice tablet \ -daemonize You should get vnc at port 5901, seemed to boot for me but I did not complete the install. Regards Jorge On 2014-12-18 16:57, Johan Kragsterman wrote: > Hi! > > > I've been looking at CoreOS and finds it interesting! Since I'd like > to have OmniOS as the platform, I need to run CoreOS as a KVM guest. > Haven't tested yet, but I downloaded the startscript for qemu, and it > looks a little bit "too much" for Illumos KVM... > > It would be nice to get some views on people that have been > considering this as well, perhaps some already tested or already > running...? > > I've seen that Frederic Alix on this list been blogging about it, but > haven't seen if he managed to run it as a KVM guest on OmniOS. > > For me it seems to be some complications at first startup, mainly. It > doesn't seem to be reachable by VNC... > > Hope to get some input from you guys... > > > Best regards from/Med v?nliga h?lsningar fr?n > > Johan Kragsterman > > Capvert > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From johan.kragsterman at capvert.se Thu Dec 18 16:57:11 2014 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 18 Dec 2014 17:57:11 +0100 Subject: [OmniOS-discuss] Ang: Re: CoreOS In-Reply-To: References: , Message-ID: Jorge, I was thinking about you when I posted this! I thought you would be a possible contributor to this thread... More furhter down... -----Jorge Schrauwen skrev: ----- Till: Johan Kragsterman Fr?n: Jorge Schrauwen Datum: 2014-12-18 17:38 Kopia: omnios-discuss at lists.omniti.com ?rende: Re: [OmniOS-discuss] CoreOS Something like this will probably work: ??/usr/bin/qemu-system-x86_64 ?? -name coreos \ ?? -enable-kvm \ ?? -no-hpet \ ?? -m 4096 ?? -cpu Nehalem \ ?? -smp sockets=1,cores=4,threads=2 \ ?? -rtc base=utc,driftfix=slew \ ?? -pidfile /tank/coreo/coreos.pid ?\ ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\ ?? -vga std ?\ ?? -vnc :1 ?\ ?? -nographic \ ?? -drive file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \ ?? -drive file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on \ ?? -boot order=cd,once=d \ ?? -device virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 \ ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \ ?? -chardev socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \ ?? -serial chardev:serial0 \ ?? -usb \ ?? -usbdevice tablet \ ?? -daemonize You should get vnc at port 5901, seemed to boot for me but I did not complete the install. At the CoreOS site they say: Start like this: ./coreos_production_qemu.sh -nographic and they pass on that string -nographic ...? It makes me wonder, because they tell you to connect with the instans only over ssh with: ssh -l core -p 2222 localhost ... So I'm not sure if it is possible to connect via VNC...did you actually check VNC, to confirm you had a VNC connection? It should boot and run from the image r/o, so perhaps you just need one "disk"? I can see you got two configured, or at least the iso file, and then a disk. Don't you think it would be enough with just the image file? Perhaps I just try... Regards Jorge On 2014-12-18 16:57, Johan Kragsterman wrote: > Hi! > > > ?I've been looking at CoreOS and finds it interesting! Since I'd like > to have OmniOS as the platform, I need to run CoreOS as a KVM guest. > Haven't tested yet, but I downloaded the startscript for qemu, and it > looks a little bit "too much" for Illumos KVM... > > ?It would be nice to get some views on people that have been > considering this as well, perhaps some already tested or already > running...? > > ?I've seen that Frederic Alix on this list been blogging about it, but > haven't seen if he managed to run it as a KVM guest on OmniOS. > > ?For me it seems to be some complications at first startup, mainly. It > doesn't seem to be reachable by VNC... > > ?Hope to get some input from you guys... > > > Best regards from/Med v?nliga h?lsningar fr?n > > Johan Kragsterman > > Capvert > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From sjorge+ml at blackdot.be Thu Dec 18 17:06:17 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Thu, 18 Dec 2014 18:06:17 +0100 Subject: [OmniOS-discuss] Ang: Re: CoreOS In-Reply-To: References: , Message-ID: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be> On 2014-12-18 17:57, Johan Kragsterman wrote: > Jorge, I was thinking about you when I posted this! I thought you > would be a possible contributor to this thread... More furhter > down... > > > -----Jorge Schrauwen skrev: ----- > Till: Johan Kragsterman > Fr?n: Jorge Schrauwen > Datum: 2014-12-18 17:38 > Kopia: omnios-discuss at lists.omniti.com > ?rende: Re: [OmniOS-discuss] CoreOS > > Something like this will probably work: > > > ??/usr/bin/qemu-system-x86_64 > ?? -name coreos \ > ?? -enable-kvm \ > ?? -no-hpet \ > ?? -m 4096 > ?? -cpu Nehalem \ > ?? -smp sockets=1,cores=4,threads=2 \ > ?? -rtc base=utc,driftfix=slew \ > ?? -pidfile /tank/coreo/coreos.pid ?\ > ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\ > ?? -vga std ?\ > ?? -vnc :1 ?\ > ?? -nographic \ > ?? -drive > file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \ > ?? -drive > file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on > \ > ?? -boot order=cd,once=d \ > ?? -device > virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 > \ > ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \ > ?? -chardev > socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \ > ?? -serial chardev:serial0 \ > ?? -usb \ > ?? -usbdevice tablet \ > ?? -daemonize > > You should get vnc at port 5901, seemed to boot for me but I did not > complete the install. > > > > At the CoreOS site they say: Start like this: > > ./coreos_production_qemu.sh -nographic > > and they pass on that string -nographic ...? > > It makes me wonder, because they tell you to connect with the instans > only over ssh with: ssh -l core -p 2222 localhost ... > > So I'm not sure if it is possible to connect via VNC...did you > actually check VNC, to confirm you had a VNC connection? > > It should boot and run from the image r/o, so perhaps you just need > one "disk"? I can see you got two configured, or at least the iso > file, and then a disk. Don't you think it would be enough with just > the image file? > > Perhaps I just try... I used the install iso to see if it booted. -nographic just mean don't spawn a graphical console AKA SDL or simular window. It does not prevent '-vnc :1' from working. > > > > Regards > > Jorge > > > > > > > > > > > > > On 2014-12-18 16:57, Johan Kragsterman wrote: >> Hi! >> >> >> ?I've been looking at CoreOS and finds it interesting! Since I'd like >> to have OmniOS as the platform, I need to run CoreOS as a KVM guest. >> Haven't tested yet, but I downloaded the startscript for qemu, and it >> looks a little bit "too much" for Illumos KVM... >> >> ?It would be nice to get some views on people that have been >> considering this as well, perhaps some already tested or already >> running...? >> >> ?I've seen that Frederic Alix on this list been blogging about it, but >> haven't seen if he managed to run it as a KVM guest on OmniOS. >> >> ?For me it seems to be some complications at first startup, mainly. It >> doesn't seem to be reachable by VNC... >> >> ?Hope to get some input from you guys... >> >> >> Best regards from/Med v?nliga h?lsningar fr?n >> >> Johan Kragsterman >> >> Capvert >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Thu Dec 18 18:21:09 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 18 Dec 2014 13:21:09 -0500 Subject: [OmniOS-discuss] latent problem with networking In-Reply-To: References: Message-ID: > On Dec 18, 2014, at 11:26 AM, Doug Hughes wrote: > > > Here's the simplest test... I start up ttcp -r on the server, it binds to port 5001, listening. I run snoop.. Then I try to connect to 5001 from another machine. I see the packets in snoop, but the accept call on the omniOS machine never returns. Something seems wonky in network land. Has anybody seen this? THe machine has been up for weeks without any problems. > > OmniOS v11 r151012 > Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved. > Use is subject to license terms. > > Regular/plain Intel chipset: > e1000g0: > root at xyr-r:/root# dladm show-link e1000g0 > LINK CLASS MTU STATE BRIDGE OVER > e1000g0 phys 1500 up -- -- > root at xyr-r:/root# dladm show-phys e1000g0 > LINK MEDIA STATE SPEED DUPLEX DEVICE > e1000g0 Ethernet up 1000 full e1000g0 > > e1000 prtdiag excerpt: > name='device-name' type=string items=1 > value='82574L Gigabit Network Connection' I can't recall if this chipset has problems or not. I want to say it might, BUT I'm not sure, so I won't point fingers. > name='subsystem-name' type=string items=1 > value='unknown subsystem' > Device Minor Nodes: > dev=(112,1) > dev_path=/pci at 0,0/pci8086,1d14 at 1c,2/pci122e,10d3 at 0:e1000g0 > Ideas? If you've the disk space, please utter "savecore -L" while your machine is in this state. It might be nice to have the system state while things are failing. Do you see any complaints from e1000g in /var/adm/messages? It's like the NIC or the driver stopped receiving packets. One thing you could do is unplumb and replumb the interface. That may make the kernel reset the driver. ifconfig e1000g0 unplumb ifconfig e1000g0 plumb up If that doesn't work, you may also need to modunload the driver before replumbing. ifconfig e1000g0 unplumb modinfo | grep e1000g modunload -i ifconfig e1000g0 plumb .... If modunload complains, you will need to unplumb the v6 interface ("ifconfig e1000g0 inet6 unplumb") or maybe disable some other services temporarily. Dan From doug at will.to Thu Dec 18 18:30:39 2014 From: doug at will.to (Doug Hughes) Date: Thu, 18 Dec 2014 13:30:39 -0500 Subject: [OmniOS-discuss] latent problem with networking In-Reply-To: References: Message-ID: well, we figured it out.. it was pretty silly actually.. It looks like for this machine, at this location, and without the network/routing/route disabled, it was picking up a *second* default route.. so some of the packets (seemingly acks and other TCP activity -- somewhat important!) were ending up at this other router which belongs to a peer organization and we're not making it all the way to the remote side under certain circumstances. Once that second default route was removed, everything was fixed. It never affected ping, and my existing ssh was working fine. I have no idea why this suddenly started causing a problem! I'm glad it turned out to be something simple. On Thu, Dec 18, 2014 at 1:21 PM, Dan McDonald wrote: > > > > On Dec 18, 2014, at 11:26 AM, Doug Hughes wrote: > > > > > > Here's the simplest test... I start up ttcp -r on the server, it binds > to port 5001, listening. I run snoop.. Then I try to connect to 5001 from > another machine. I see the packets in snoop, but the accept call on the > omniOS machine never returns. Something seems wonky in network land. Has > anybody seen this? THe machine has been up for weeks without any problems. > > > > OmniOS v11 r151012 > > Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved. > > Use is subject to license terms. > > > > Regular/plain Intel chipset: > > e1000g0: > > root at xyr-r:/root# dladm show-link e1000g0 > > LINK CLASS MTU STATE BRIDGE OVER > > e1000g0 phys 1500 up -- -- > > root at xyr-r:/root# dladm show-phys e1000g0 > > LINK MEDIA STATE SPEED DUPLEX DEVICE > > e1000g0 Ethernet up 1000 full e1000g0 > > > > e1000 prtdiag excerpt: > > name='device-name' type=string items=1 > > value='82574L Gigabit Network Connection' > > I can't recall if this chipset has problems or not. I want to say it > might, BUT I'm not sure, so I won't point fingers. > > > name='subsystem-name' type=string items=1 > > value='unknown subsystem' > > Device Minor Nodes: > > dev=(112,1) > > dev_path=/pci at 0,0/pci8086,1d14 at 1c > ,2/pci122e,10d3 at 0:e1000g0 > > Ideas? > > If you've the disk space, please utter "savecore -L" while your machine is > in this state. It might be nice to have the system state while things are > failing. > > Do you see any complaints from e1000g in /var/adm/messages? > > It's like the NIC or the driver stopped receiving packets. > > One thing you could do is unplumb and replumb the interface. That may > make the kernel reset the driver. > > ifconfig e1000g0 unplumb > ifconfig e1000g0 plumb up > > If that doesn't work, you may also need to modunload the driver before > replumbing. > > ifconfig e1000g0 unplumb > modinfo | grep e1000g > modunload -i > ifconfig e1000g0 plumb .... > > If modunload complains, you will need to unplumb the v6 interface > ("ifconfig e1000g0 inet6 unplumb") or maybe disable some other services > temporarily. > > Dan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Dec 18 18:34:18 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 18 Dec 2014 13:34:18 -0500 Subject: [OmniOS-discuss] latent problem with networking In-Reply-To: References: Message-ID: <59A588F7-8C4D-4198-94A1-3CCF9BE91E22@omniti.com> > On Dec 18, 2014, at 1:30 PM, Doug Hughes wrote: > > well, we figured it out.. > > it was pretty silly actually.. It looks like for this machine, at this location, and without the network/routing/route disabled, it was picking up a *second* default route.. so some of the packets (seemingly acks and other TCP activity -- somewhat important!) were ending up at this other router which belongs to a peer organization and we're not making it all the way to the remote side under certain circumstances. Once that second default route was removed, everything was fixed. It never affected ping, and my existing ssh was working fine. I have no idea why this suddenly started causing a problem! > > I'm glad it turned out to be something simple. Me too. When I see questions about drivers, I assume the routing situation had already been inspected. Glad you figured that out on your own, and thank you for the update! Dan From mir at miras.org Thu Dec 18 18:39:17 2014 From: mir at miras.org (Michael Rasmussen) Date: Thu, 18 Dec 2014 19:39:17 +0100 Subject: [OmniOS-discuss] latent problem with networking In-Reply-To: References: Message-ID: <20141218193917.7f2e2138@sleipner.datanom.net> On Thu, 18 Dec 2014 13:21:09 -0500 Dan McDonald wrote: > > I can't recall if this chipset has problems or not. I want to say it might, BUT I'm not sure, so I won't point fingers. > I have run with these chipsets on a Hudson based AMD board since 151008 was bloody (Sep/Oct 2013). It has been rock solid since then and still is. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: You can't have everything. Where would you put it? -- Steven Wright -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From danmcd at omniti.com Thu Dec 18 19:12:20 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 18 Dec 2014 14:12:20 -0500 Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long, please read) Message-ID: This will be the last OmniOS bloody update for the calendar year, AND you likely won't see the next one until the end of January. Please read on for what's new, and why the big upcoming gap. Only out to the pkg servers this time, this OmniOS bloody update (r151013-20141218): - omnios-build master branch, revision 9d22dd3 - Fix for os.mknod() in 64-bit runtimes. If any python wizards in the audience can help me upstream this, I'd appreciate it. - New M2Crypto features needed for CRL handling, part of the pkg(5) modernization effort. - illumos-omnios master branch, revision eb4c8f3 (last illumos-gate merge 9c30721) - Fast reboot is now DISABLED by default. - Non-interactive mode for ypinit(1M) - NFS bugfixes (illumos 5436 & 5440) - Several man page updates - Hung mount in one zone no longer affects boot/halt of another (illumos 5419) - Underlying infrastructure for global-zone rulesets for zones using ipfilter. (If this interests you, please let me know so an RFE can be tracked in the OmniOS ipkg brand) - One mpt_sas hang-on-reset bug fixed (illumos 5297) - Two ZFS bugfixes (illumos 5377 & 5369-5370). . . . We are suspending bloody updates during the month of January to run tests on a modernization of pkg(5). I may be altering the bloody pkg repo during this time, but I would HIGHLY recommend that users who update to this version save a backup BE afterwards (using beadm(1m)) in case they accidentially take unannounced experimental updates during the month of January. This modernization of pkg(5) has been bootstrapped by becoming a downstream child of the OpenIndiana Hipster work. I am in contact with OI pkg(5) people, and of course everything we do will be open-sourced (testing it all is the hard part). I've found bugs in Python thanks to this work, so I know it's being helpful. We have the pkg(5) test suite passing 100% in API tests, and ~90% in CLI tests, where almost all of those failures are either to either assumptions about where apache22 lives, or by having the existence of python2.7 in /usr/lib/python2.7... which runs a bit counter to OmniOS's "Keep your sh*t to yourself" philosophy. This pkg(5) work is difficult. Once it's in an OmniOS bloody update, I hope to aid illumos community members in upstreaming pieces from other distros, as well as our own. During January, I hope to define what will be in r151014, which is not only our next Stable release, but also our next Long-Term Support release. This bloody period is critical to r151014's success, so if you have spare machines to run this bloody on, PLEASE do so, as you'll be helping the OmniOS community. Thanks, Dan McDonald -- OmniOS Engineering From danmcd at omniti.com Thu Dec 18 22:28:43 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 18 Dec 2014 17:28:43 -0500 Subject: [OmniOS-discuss] UPDATE NOW: git vulnerability Message-ID: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com> See here: https://github.com/blog/1938-vulnerability-announced-update-your-git-clients All supported repos (006/LTS, 010/previous-stable, 012/current-stable, 013/bloody) are now updated with an appropriately patched version of git. Dan From kangcool2002 at yahoo.co.uk Thu Dec 18 22:48:05 2014 From: kangcool2002 at yahoo.co.uk (a a) Date: Thu, 18 Dec 2014 22:48:05 +0000 (UTC) Subject: [OmniOS-discuss] Who is good at recovering zfs pools/files? Message-ID: <922776323.500535.1418942885007.JavaMail.yahoo@jws11122.mail.ir2.yahoo.com> Hi. I am currently working for a London based charity who had bought an NAS appliance which is based on OmniOS. Unfortunately its rolled over taking lots of video files before it could be backed up. I am not sure if they have a support contract with Omni, and I have told them to talk to Omni, but they really need some of these video files back. So who is good at this as its been a while since I last looked or had an issue with zfs? -------------- next part -------------- An HTML attachment was scrubbed... URL: From henson at acm.org Fri Dec 19 00:53:28 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 18 Dec 2014 16:53:28 -0800 Subject: [OmniOS-discuss] UPDATE NOW: git vulnerability In-Reply-To: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com> References: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com> Message-ID: <19e401d01b26$36899d80$a39cd880$@acm.org> > From: Dan McDonald > Sent: Thursday, December 18, 2014 2:29 PM > > All supported repos (006/LTS, 010/previous-stable, 012/current-stable, > 013/bloody) are now updated with an appropriately patched version of git. This bug only impacts users on case insensitive filesystems, so unless you're running it on a zfs filesystem with casesensitivity= insensitive, you're not going to be vulnerable to this issue on an illumos system... Not that you shouldn't update anyway as a general precaution (and a big thanks to Dan for making it available so quickly), but if for some reason updating would be problematic it's probably not a crisis for you. From henson at acm.org Fri Dec 19 03:05:24 2014 From: henson at acm.org (Paul B. Henson) Date: Thu, 18 Dec 2014 19:05:24 -0800 Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long, please read) In-Reply-To: References: Message-ID: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org> > From: Dan McDonald > Sent: Thursday, December 18, 2014 11:12 AM > > - Underlying infrastructure for global-zone rulesets for zones using ipfilter. Hmm, what exactly is this? Right now my zones are using exclusive stacks and just running ipfilter in the zone... From danmcd at omniti.com Fri Dec 19 03:20:31 2014 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 18 Dec 2014 22:20:31 -0500 Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long, please read) In-Reply-To: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org> References: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org> Message-ID: > On Dec 18, 2014, at 10:05 PM, Paul B. Henson wrote: > >> From: Dan McDonald >> Sent: Thursday, December 18, 2014 11:12 AM >> >> - Underlying infrastructure for global-zone rulesets for zones using > ipfilter. > > Hmm, what exactly is this? Right now my zones are using exclusive stacks and > just running ipfilter in the zone... This came from Joyent. Imagine if you, the GZ admin, want to further clamp down a zone you're renting or just transferring root for. That's what this feature does. It doesn't have set-at-boot time properties, because that needs to be added to your zone's brand code. Brand code is still outside illumos-gate for illumos distros, including OmniOS. Dan From garrett at damore.org Fri Dec 19 17:37:58 2014 From: garrett at damore.org (Garrett D'Amore) Date: Fri, 19 Dec 2014 09:37:58 -0800 Subject: [OmniOS-discuss] [developer] Followon review --> Changes that allow illumos-gate to build on OmniOS In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> Message-ID: <6200A265-2B7C-4DF6-A64F-449A21DEE8F5@damore.org> I?d like to see the shell style guide go entirely not just the html. The docbook is hard to use, but more importantly the guide itself got little review, and I think it may refer to ksh93isms that I think we may wish to avoid in the future. Shell style should only make use of POSIX shell compatible capabilities. Its my hope someday to extract ksh93 from the gate in favor of a lighter weight alternative such as dash. I have already done some work on this front, though its not ready for wider consumption yet. - Garrett > On Dec 17, 2014, at 6:44 PM, Dan McDonald via illumos-developer wrote: > > Now that the java fixes are their own thing, let's revisit the remaining changes in: > > http://kebe.com/~danmcd/webrevs/for-omnios/ > > All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated. > > The aforementioned directory contains: > > check-rtime/ > > * Some check_rtime modifications for the version of things in OmniOS > > ig-on-omnios.env > > * A sample .env file used for building illumos-gate on OmniOS. Look for the extra export variables you'll need. > > java-updates > > * Now points to illumos 4719, see previous mail. > > lose-ipp-listener > > * Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS. > * See the sample .env file above for how it works. > > lose-shell-styleguide > > * Lose the dependency on docbooks, only existing in the Shell Style Guide. > * Apparently not only is this out in OmniOS already, but in SmartOS as well. > > lose-sysidtool-dep > > * Lose the SMF dependency on sysidtool. > * I've been told this only ever affected people trying to move from old OpenSolaris to OI. > > modern-netsmnp > > * Allow linkage with more recent versions of net-snmp. > > more-lintfixes > > * Lint fixes that the OmniOS sunstudio12.1 lint are picky about. > * Work done by Rich Lowe. Thank you, Rich! > > omnios-mail-msg.txt > > * See the small number of remaining lint errors, ALL having to do with openssl versions. > > perl-fixes > > * More perl fixes to work with OmniOS's alternative approach to perl support. > * This has also been tested, I believe, by the OpenIndiana Hipster people. > > After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well. They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well. > > Thanks, > Dan McDonald > > ------------------------------------------- > illumos-developer > Archives: https://www.listbox.com/member/archive/182179/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182179/21239177-3604570e > Modify Your Subscription: https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-2d0c9337 > Powered by Listbox: http://www.listbox.com From garrett at damore.org Fri Dec 19 17:34:07 2014 From: garrett at damore.org (Garrett D'Amore) Date: Fri, 19 Dec 2014 09:34:07 -0800 Subject: [OmniOS-discuss] [developer] Followon review --> Changes that allow illumos-gate to build on OmniOS In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com> Message-ID: <906F536C-0EAE-4D13-B315-1576888CF093@damore.org> on the ipp listener cleanup, I think you can probably avoid a two step calculation of the package list in pkg/Makefile by declaring a variable automatically that is the inverse of ENABLE_IPP_PRINTING. Somehting like this: $(ENABLE_IPP_PIRNTING)NO_ENABLE_IPP_PRINTING=$(POUND_SIGN) But really as I think about this, making more packages optional seems like a good thing. So probably we should instead create a list of packages to suppress, append to it, and filter that way. (Think scaling this to more than just the IPP printing problem.) - Garrett > On Dec 17, 2014, at 6:44 PM, Dan McDonald via illumos-developer wrote: > > Now that the java fixes are their own thing, let's revisit the remaining changes in: > > http://kebe.com/~danmcd/webrevs/for-omnios/ > > All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated. > > The aforementioned directory contains: > > check-rtime/ > > * Some check_rtime modifications for the version of things in OmniOS > > ig-on-omnios.env > > * A sample .env file used for building illumos-gate on OmniOS. Look for the extra export variables you'll need. > > java-updates > > * Now points to illumos 4719, see previous mail. > > lose-ipp-listener > > * Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS. > * See the sample .env file above for how it works. > > lose-shell-styleguide > > * Lose the dependency on docbooks, only existing in the Shell Style Guide. > * Apparently not only is this out in OmniOS already, but in SmartOS as well. > > lose-sysidtool-dep > > * Lose the SMF dependency on sysidtool. > * I've been told this only ever affected people trying to move from old OpenSolaris to OI. > > modern-netsmnp > > * Allow linkage with more recent versions of net-snmp. > > more-lintfixes > > * Lint fixes that the OmniOS sunstudio12.1 lint are picky about. > * Work done by Rich Lowe. Thank you, Rich! > > omnios-mail-msg.txt > > * See the small number of remaining lint errors, ALL having to do with openssl versions. > > perl-fixes > > * More perl fixes to work with OmniOS's alternative approach to perl support. > * This has also been tested, I believe, by the OpenIndiana Hipster people. > > After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well. They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well. > > Thanks, > Dan McDonald > > ------------------------------------------- > illumos-developer > Archives: https://www.listbox.com/member/archive/182179/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182179/21239177-3604570e > Modify Your Subscription: https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-2d0c9337 > Powered by Listbox: http://www.listbox.com From danmcd at omniti.com Fri Dec 19 20:39:43 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 19 Dec 2014 15:39:43 -0500 Subject: [OmniOS-discuss] UPDATE TIME (again)! Message-ID: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> The NTP server version is now 4.2.8, which I've gone ahead and placed on ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable, 013/bloody). Please utter "pkg update" at your earliest convenience. This is not a requires-reboot update. Relevant CVEs are: CVE-2014-9296, 9295, 9294, 9293 Thanks, Dan From danmcd at omniti.com Fri Dec 19 20:52:35 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 19 Dec 2014 15:52:35 -0500 Subject: [OmniOS-discuss] UPDATE TIME (again)! In-Reply-To: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> Message-ID: <6E6F9316-ABE7-43C0-862E-F18C1C7C6D2E@omniti.com> > On Dec 19, 2014, at 3:39 PM, Dan McDonald wrote: > > The NTP server version is now 4.2.8, which I've gone ahead and placed on ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable, 013/bloody). Please utter "pkg update" at your earliest convenience. This is not a requires-reboot update. > > Relevant CVEs are: CVE-2014-9296, 9295, 9294, 9293 NOTE: You will also need to "svcadm restart ntp" after this update. Dan From henson at acm.org Sat Dec 20 02:46:31 2014 From: henson at acm.org (Paul B. Henson) Date: Fri, 19 Dec 2014 18:46:31 -0800 Subject: [OmniOS-discuss] UPDATE TIME (again)! In-Reply-To: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> Message-ID: <1e3201d01bff$2b8521a0$828f64e0$@acm.org> > From: Dan McDonald > Sent: Friday, December 19, 2014 12:40 PM > > The NTP server version is now 4.2.8, which I've gone ahead and placed on > ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable, > 013/bloody). Please utter "pkg update" at your earliest convenience. This is > not a requires-reboot update. The only thing I've used classic ntpd on for ages are boxes with actual hardware time devices on them. It's got a history of being bloated with the occasional security issue like this 8-/. I've been using openntpd for pretty much everything else, but the portable version is unfortunately getting pretty dated :(. I've been thinking of looking into Chrony (http://chrony.tuxfamily.org/ ) as a possible simple/lightweight replacement for openntpd. Has anybody tried that under omnios? From danmcd at omniti.com Sat Dec 20 02:51:06 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 19 Dec 2014 21:51:06 -0500 Subject: [OmniOS-discuss] UPDATE TIME (again)! In-Reply-To: <1e3201d01bff$2b8521a0$828f64e0$@acm.org> References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com> <1e3201d01bff$2b8521a0$828f64e0$@acm.org> Message-ID: > On Dec 19, 2014, at 9:46 PM, Paul B. Henson wrote: > I've been thinking of looking into Chrony > (http://chrony.tuxfamily.org/ ) as a possible simple/lightweight replacement > for openntpd. Has anybody tried that under omnios? I don't think so, but if you can get it to work, you should try and upstream your diffs. I've done this for other open-source projects (ptp2d, e.g.), and it's been rather rewarding. If you have problems getting it working, let people know here, or on the illumos developer's list. Dan From henson at acm.org Sat Dec 20 02:55:02 2014 From: henson at acm.org (Paul B. Henson) Date: Fri, 19 Dec 2014 18:55:02 -0800 Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long, please read) In-Reply-To: References: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org> Message-ID: <211001d01c00$5c74f190$155ed4b0$@acm.org> > From: Dan McDonald > Sent: Thursday, December 18, 2014 7:21 PM > > This came from Joyent. Imagine if you, the GZ admin, want to further clamp > down a zone you're renting or just transferring root for. That's what this > feature does. Ooh, cool. I can see how that would be helpful if you are deploying a zone that will have an untrustworthy entity with privileges and you want to keep them from abusing the network. From danmcd at omniti.com Sat Dec 20 03:01:10 2014 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 19 Dec 2014 22:01:10 -0500 Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long, please read) In-Reply-To: <211001d01c00$5c74f190$155ed4b0$@acm.org> References: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org> <211001d01c00$5c74f190$155ed4b0$@acm.org> Message-ID: <54E99D52-1987-42AA-BAB3-F476C4192A97@omniti.com> > On Dec 19, 2014, at 9:55 PM, Paul B. Henson wrote: > , cool. I can see how that would be helpful if you are deploying a zone > that will have an untrustworthy entity with privileges and you want to keep > them from abusing the network. > For it to be REALLY useful, some brand code will have to happen. You need something to read /etc/ipf/zones/ from the global zone, and only the brand code that handles "boot" methods can do it right. I'm not doing that right now. Dan From johan.kragsterman at capvert.se Sat Dec 20 11:47:47 2014 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Sat, 20 Dec 2014 12:47:47 +0100 Subject: [OmniOS-discuss] Ang: Re: Ang: Re: CoreOS In-Reply-To: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be> References: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>, , Message-ID: Hi, Jorge and all! I would be interested in discussing this further, but perhaps omnios-discuss isn't the right place? Since I don't know if this is omnios/illumos/coreos specific... I did some experimenting: I only used CoreOS stable in my tests. I tried the iso, but the iso isn't full featured, and doesn't run docker out of the box. And the docker implementation is of coarse what everybody is interested in. I got it to boot without problems, but I had big problems with VNC keymapping due to my Swedish keyboard and perhaps my Swedish client computer. So I could actually never do something with it, and since it is not full featured, it is not what I want to use. So instead, I downloaded the img file for qemu, created a volume, and dd'ed the image to the volume, and then set this volume as boot. That went fine, to get it to boot. But then, with the default boot option in grub, it panicked, and restarted every 60 seconds. I stopped the grub booting, and chosed the B option. That didn't panic, but it didn't work either, it was too much that didn't work. But option A went fine, no panic, and everything seem to work more or less without problems. The only problem here seem to be that I can't log in, due to the "first log in"-principles they seem to have: It is only possible to log in via ssh, which means the network have to be up, and I couldn't get the network to come up....so there I am right now... Regards Johan -----Jorge Schrauwen skrev: ----- Till: Johan Kragsterman Fr?n: Jorge Schrauwen Datum: 2014-12-18 18:07 Kopia: omnios-discuss at lists.omniti.com ?rende: Re: Ang: Re: [OmniOS-discuss] CoreOS On 2014-12-18 17:57, Johan Kragsterman wrote: > Jorge, I was thinking about you when I posted this! I thought you > would be a possible contributor to this thread... ?More furhter > down... > > > -----Jorge Schrauwen skrev: ----- > Till: Johan Kragsterman > Fr?n: Jorge Schrauwen > Datum: 2014-12-18 17:38 > Kopia: omnios-discuss at lists.omniti.com > ?rende: Re: [OmniOS-discuss] CoreOS > > Something like this will probably work: > > > ??/usr/bin/qemu-system-x86_64 > ?? -name coreos \ > ?? -enable-kvm \ > ?? -no-hpet \ > ?? -m 4096 > ?? -cpu Nehalem \ > ?? -smp sockets=1,cores=4,threads=2 \ > ?? -rtc base=utc,driftfix=slew \ > ?? -pidfile /tank/coreo/coreos.pid ?\ > ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\ > ?? -vga std ?\ > ?? -vnc :1 ?\ > ?? -nographic \ > ?? -drive > file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \ > ?? -drive > file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on > \ > ?? -boot order=cd,once=d \ > ?? -device > virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 > \ > ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \ > ?? -chardev > socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \ > ?? -serial chardev:serial0 \ > ?? -usb \ > ?? -usbdevice tablet \ > ?? -daemonize > > You should get vnc at port 5901, seemed to boot for me but I did not > complete the install. > > > > At the CoreOS site they say: Start like this: > > ./coreos_production_qemu.sh -nographic > > and they pass on that string -nographic ?...? > > It makes me wonder, because they tell you to connect with the instans > only over ssh with: ssh -l core -p 2222 localhost ? ... > > So I'm not sure if it is possible to connect via VNC...did you > actually check VNC, to confirm you had a VNC connection? > > It should boot and run from the image r/o, so perhaps you just need > one "disk"? I can see you got two configured, or at least the iso > file, and then a disk. Don't you think it would be enough with just > the image file? > > Perhaps I just try... I used the install iso to see if it booted. -nographic just mean don't spawn a graphical console AKA SDL or simular window. It does not prevent '-vnc :1' from working. > > > > Regards > > Jorge > > > > > > > > > > > > > On 2014-12-18 16:57, Johan Kragsterman wrote: >> Hi! >> >> >> ?I've been looking at CoreOS and finds it interesting! Since I'd like >> to have OmniOS as the platform, I need to run CoreOS as a KVM guest. >> Haven't tested yet, but I downloaded the startscript for qemu, and it >> looks a little bit "too much" for Illumos KVM... >> >> ?It would be nice to get some views on people that have been >> considering this as well, perhaps some already tested or already >> running...? >> >> ?I've seen that Frederic Alix on this list been blogging about it, but >> haven't seen if he managed to run it as a KVM guest on OmniOS. >> >> ?For me it seems to be some complications at first startup, mainly. It >> doesn't seem to be reachable by VNC... >> >> ?Hope to get some input from you guys... >> >> >> Best regards from/Med v?nliga h?lsningar fr?n >> >> Johan Kragsterman >> >> Capvert >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From sjorge+ml at blackdot.be Sat Dec 20 13:36:50 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Sat, 20 Dec 2014 14:36:50 +0100 Subject: [OmniOS-discuss] Ang: Re: Ang: Re: CoreOS In-Reply-To: References: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>, , Message-ID: Hey Johan, I just poked at the qemu image... it seems it wants some stuff not in our old qemu-kvm fork. e.g. fsdev (mouting a filesystem from host to guest). But let's try anyway! # convert qcow2 to raw qemu-img convert coreos_production_qemu_image.img coreos_production_qemu_image.dd # dump this on our zvol dd if=coreos_production_qemu_image.dd of=/dev/zvol/rdsk/core/vms/hosts/coreos/disk0 We now have the correctly formatted data on our zvol... On the plus side it does output nicely to ttya if added to a vm :) So... here is where the kernel dies: (oh it does some kexec bits which are a PITA) --- [ 0.001000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.17.2 #2 [ 0.001000] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [ 0.001000] 0000000000000008 ffff88007a3e7db8 ffffffff814e8915 000000000000e [ 0.001000] ffffffff81798190 ffff88007a3e7e38 ffffffff814e4c97 0000000000006 [ 0.001000] 0000000000000008 ffff88007a3e7e48 ffff88007a3e7de8 00000000fffb0 [ 0.001000] Call Trace: [ 0.001000] [] dump_stack+0x46/0x58 [ 0.001000] [] panic+0xc1/0x1f5 [ 0.001000] [] setup_IO_APIC+0x7d6/0x83d [ 0.001000] [] native_smp_prepare_cpus+0x2bc/0x337 [ 0.001000] [] kernel_init_freeable+0xcd/0x212 [ 0.001000] [] ? rest_init+0x80/0x80 [ 0.001000] [] kernel_init+0xe/0xf0 [ 0.001000] [] ret_from_fork+0x7c/0xb0 [ 0.001000] [] ? rest_init+0x80/0x80 [ 0.001000] Rebooting in 60 seconds.. --- I actually also have this on a ubuntu vm I am using, it needs noapic kernel option... on the grub prompt (really nice is coreos seems to have grub + console on both tty0 (vga) and ttyS0 (serial ttya). Woo hoo we got past that bit where it fails on IO-APIC, now we just hang on smpboot :( --- [ 0.001000] CPU: Physical Processor ID: 0 [ 0.001000] CPU: Processor Core ID: 0 [ 0.001000] mce: CPU supports 10 MCE banks [ 0.001000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.001000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.001000] Freeing SMP alternatives memory: 20K (ffffffff82fa1000 - fffffff) [ 0.001000] ftrace: allocating 19518 entries in 77 pages [ 0.001000] smpboot: CPU0: Intel QEMU Virtual CPU version 0.14.1 (fam: 06, m --- Pretty much stuck here... I tried some variations of cpu type (qemu64, Nehalem and host) I also tried using one vcpu but still stuck. Let's just cripple the entire thing and plow are way through: adding 'nosmp noapic noacpi' So yeah at this point coreos is pretty useless... but we fly past smpboot! And... land here: --- [ 0.239823] scsi host0: ata_piix [ 0.239823] scsi host1: ata_piix [ 0.239823] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14 [ 0.239823] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15 --- If it is docker you want, you may as well look at SmartOS's LX Brand stuff, they are racing towards workable docker zones. But since I came this far, lets see if I can make it to the finish... I am using virtio... lets try scsi... nothing... ide... nothing... So this is were it ends. Our qemu-kvm fork is probably just too old. Regards Jorge On 2014-12-20 12:47, Johan Kragsterman wrote: > Hi, Jorge and all! > > > I would be interested in discussing this further, but perhaps > omnios-discuss isn't the right place? Since I don't know if this is > omnios/illumos/coreos specific... > > I did some experimenting: > > I only used CoreOS stable in my tests. > > I tried the iso, but the iso isn't full featured, and doesn't run > docker out of the box. And the docker implementation is of coarse what > everybody is interested in. I got it to boot without problems, but I > had big problems with VNC keymapping due to my Swedish keyboard and > perhaps my Swedish client computer. So I could actually never do > something with it, and since it is not full featured, it is not what I > want to use. > > So instead, I downloaded the img file for qemu, created a volume, and > dd'ed the image to the volume, and then set this volume as boot. That > went fine, to get it to boot. But then, with the default boot option > in grub, it panicked, and restarted every 60 seconds. > > I stopped the grub booting, and chosed the B option. That didn't > panic, but it didn't work either, it was too much that didn't work. > But option A went fine, no panic, and everything seem to work more or > less without problems. The only problem here seem to be that I can't > log in, due to the "first log in"-principles they seem to have: It is > only possible to log in via ssh, which means the network have to be > up, and I couldn't get the network to come up....so there I am right > now... > > Regards Johan > > > -----Jorge Schrauwen skrev: ----- > Till: Johan Kragsterman > Fr?n: Jorge Schrauwen > Datum: 2014-12-18 18:07 > Kopia: omnios-discuss at lists.omniti.com > ?rende: Re: Ang: Re: [OmniOS-discuss] CoreOS > > > On 2014-12-18 17:57, Johan Kragsterman wrote: >> Jorge, I was thinking about you when I posted this! I thought you >> would be a possible contributor to this thread... ?More furhter >> down... >> >> >> -----Jorge Schrauwen skrev: ----- >> Till: Johan Kragsterman >> Fr?n: Jorge Schrauwen >> Datum: 2014-12-18 17:38 >> Kopia: omnios-discuss at lists.omniti.com >> ?rende: Re: [OmniOS-discuss] CoreOS >> >> Something like this will probably work: >> >> >> ??/usr/bin/qemu-system-x86_64 >> ?? -name coreos \ >> ?? -enable-kvm \ >> ?? -no-hpet \ >> ?? -m 4096 >> ?? -cpu Nehalem \ >> ?? -smp sockets=1,cores=4,threads=2 \ >> ?? -rtc base=utc,driftfix=slew \ >> ?? -pidfile /tank/coreo/coreos.pid ?\ >> ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\ >> ?? -vga std ?\ >> ?? -vnc :1 ?\ >> ?? -nographic \ >> ?? -drive >> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \ >> ?? -drive >> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on >> \ >> ?? -boot order=cd,once=d \ >> ?? -device >> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 >> \ >> ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \ >> ?? -chardev >> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \ >> ?? -serial chardev:serial0 \ >> ?? -usb \ >> ?? -usbdevice tablet \ >> ?? -daemonize >> >> You should get vnc at port 5901, seemed to boot for me but I did not >> complete the install. >> >> >> >> At the CoreOS site they say: Start like this: >> >> ./coreos_production_qemu.sh -nographic >> >> and they pass on that string -nographic ?...? >> >> It makes me wonder, because they tell you to connect with the instans >> only over ssh with: ssh -l core -p 2222 localhost ? ... >> >> So I'm not sure if it is possible to connect via VNC...did you >> actually check VNC, to confirm you had a VNC connection? >> >> It should boot and run from the image r/o, so perhaps you just need >> one "disk"? I can see you got two configured, or at least the iso >> file, and then a disk. Don't you think it would be enough with just >> the image file? >> >> Perhaps I just try... > I used the install iso to see if it booted. > -nographic just mean don't spawn a graphical console AKA SDL or simular > window. It does not prevent '-vnc :1' from working. > >> >> >> >> Regards >> >> Jorge >> >> >> >> >> >> >> >> >> >> >> >> >> On 2014-12-18 16:57, Johan Kragsterman wrote: >>> Hi! >>> >>> >>> ?I've been looking at CoreOS and finds it interesting! Since I'd like >>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest. >>> Haven't tested yet, but I downloaded the startscript for qemu, and it >>> looks a little bit "too much" for Illumos KVM... >>> >>> ?It would be nice to get some views on people that have been >>> considering this as well, perhaps some already tested or already >>> running...? >>> >>> ?I've seen that Frederic Alix on this list been blogging about it, >>> but >>> haven't seen if he managed to run it as a KVM guest on OmniOS. >>> >>> ?For me it seems to be some complications at first startup, mainly. >>> It >>> doesn't seem to be reachable by VNC... >>> >>> ?Hope to get some input from you guys... >>> >>> >>> Best regards from/Med v?nliga h?lsningar fr?n >>> >>> Johan Kragsterman >>> >>> Capvert >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss From rt at steait.net Mon Dec 22 18:59:19 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 22 Dec 2014 18:59:19 +0000 Subject: [OmniOS-discuss] mount/create volume lu from snapshot Message-ID: <1419274759953.32966@steait.net> hi all, I have two omnios boxes and zfs replication going between the two every 30 min. I am replicating a volume lu pool01/vol01 from hostA to hostB how can I mount this or create a volume lu out of it on my destination box? br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Mon Dec 22 19:07:06 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 22 Dec 2014 14:07:06 -0500 Subject: [OmniOS-discuss] mount/create volume lu from snapshot In-Reply-To: <1419274759953.32966@steait.net> References: <1419274759953.32966@steait.net> Message-ID: <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> > On Dec 22, 2014, at 1:59 PM, Rune Tipsmark wrote: > > hi all, > > I have two omnios boxes and zfs replication going between the two every 30 min. > > I am replicating a volume lu pool01/vol01 from hostA to hostB > > how can I mount this or create a volume lu out of it on my destination box? > "stmfadm create-lu" does this for a given zvol. Read the stmfadm(1M) man page for more details. Dan From rt at steait.net Mon Dec 22 19:24:46 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 22 Dec 2014 19:24:46 +0000 Subject: [OmniOS-discuss] mount/create volume lu from snapshot In-Reply-To: <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> References: <1419274759953.32966@steait.net>, <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> Message-ID: <1419276286622.95214@steait.net> hi Dan, tried that, get data file error. I tried zfs list and found my snapshot pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 doesn't work... also tried pool01/PHTVOL01, no luck either. which file do I need to use? br, Rune ________________________________________ From: Dan McDonald Sent: Monday, December 22, 2014 8:07 PM To: Rune Tipsmark Cc: omnios-discuss at lists.omniti.com Subject: Re: [OmniOS-discuss] mount/create volume lu from snapshot > On Dec 22, 2014, at 1:59 PM, Rune Tipsmark wrote: > > hi all, > > I have two omnios boxes and zfs replication going between the two every 30 min. > > I am replicating a volume lu pool01/vol01 from hostA to hostB > > how can I mount this or create a volume lu out of it on my destination box? > "stmfadm create-lu" does this for a given zvol. Read the stmfadm(1M) man page for more details. Dan From danmcd at omniti.com Mon Dec 22 19:26:52 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 22 Dec 2014 14:26:52 -0500 Subject: [OmniOS-discuss] mount/create volume lu from snapshot In-Reply-To: <1419276286622.95214@steait.net> References: <1419274759953.32966@steait.net> <, <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> <>> <1419276286622.95214@steait.net> Message-ID: <6E1EAF64-BD33-48E3-A32E-37AE6DC1C57E@omniti.com> > On Dec 22, 2014, at 2:24 PM, Rune Tipsmark wrote: > > hi Dan, > > tried that, get data file error. > > I tried zfs list and found my snapshot > > pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 > > then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 > > doesn't work... also tried pool01/PHTVOL01, no luck either. > > which file do I need to use? /dev/zvol/dsk/pool01/PHTVOL01 If you utter "stmf list-lu" on your original machine, you'll see pathnames. Dan From geoffn at gnaa.net Mon Dec 22 20:41:00 2014 From: geoffn at gnaa.net (Geoff Nordli) Date: Mon, 22 Dec 2014 12:41:00 -0800 Subject: [OmniOS-discuss] mount/create volume lu from snapshot In-Reply-To: <1419276286622.95214@steait.net> References: <1419274759953.32966@steait.net>, <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> <1419276286622.95214@steait.net> Message-ID: <549881DC.9050905@gnaa.net> On 14-12-22 11:24 AM, Rune Tipsmark wrote: > hi Dan, > > tried that, get data file error. > > I tried zfs list and found my snapshot > > pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 > > then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4 > > doesn't work... also tried pool01/PHTVOL01, no luck either. > > which file do I need to use? > > br, > Rune > > ________________________________________ > From: Dan McDonald > Sent: Monday, December 22, 2014 8:07 PM > To: Rune Tipsmark > Cc: omnios-discuss at lists.omniti.com > Subject: Re: [OmniOS-discuss] mount/create volume lu from snapshot > >> On Dec 22, 2014, at 1:59 PM, Rune Tipsmark wrote: >> >> hi all, >> >> I have two omnios boxes and zfs replication going between the two every 30 min. >> >> I am replicating a volume lu pool01/vol01 from hostA to hostB >> >> how can I mount this or create a volume lu out of it on my destination box? >> > "stmfadm create-lu" does this for a given zvol. > > Read the stmfadm(1M) man page for more details. > > Dan > > ____________ Rune, you are going to need to create a zfs clone of the snapshot before you create the lun. Geoff From gate03 at landcroft.co.uk Tue Dec 23 00:51:53 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 10:51:53 +1000 Subject: [OmniOS-discuss] KVM within a child zone Message-ID: <20141223105153.5d50749a@emeritus> Has anyone done this? The main obstacle in my own investigation is that the zonecfg command "add device / set match=/dev/net/vnick0 ; end" is ignored so I can't network the VM. Michael. From gate03 at landcroft.co.uk Tue Dec 23 02:04:02 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 12:04:02 +1000 Subject: [OmniOS-discuss] sudden loss of networking Message-ID: <20141223120402.6714c763@emeritus> Just now, r151012 lost networking. I was able to get in via a serial console (very relieved I took the trouble to set that up) but otherwise the machine wasn't even pingable. Thanks to the wonders of NFS, a reboot preserved everything --- I wasn't running any VMs at the time. svcs -a showed that network/physical:default was online. I didn't really know what else to do. Any suggestions for next time ? assuming I can get in on the serial console. Michael. From danmcd at omniti.com Tue Dec 23 02:27:00 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 22 Dec 2014 21:27:00 -0500 Subject: [OmniOS-discuss] sudden loss of networking In-Reply-To: <20141223120402.6714c763@emeritus> References: <20141223120402.6714c763@emeritus> Message-ID: > On Dec 22, 2014, at 9:04 PM, Michael Mounteney wrote: > > Just now, r151012 lost networking. I was able to get in via a serial > console (very relieved I took the trouble to set that up) but otherwise > the machine wasn't even pingable. Thanks to the wonders of NFS, a > reboot preserved everything --- I wasn't running any VMs at the time. > > svcs -a showed that network/physical:default was online. I didn't > really know what else to do. Any suggestions for next time ? assuming > I can get in on the serial console. Lots of possibilities when you "lose networking". Lots of information helps tell what happened. Output of these commands: - ifconfig -a - dladm show-link - dladm show-phys - dladm show-ether - netstat -rnv Also, make sure you didn't lose routing, as opposed to losing the link. See if a node on the same subnet can ping it (you said it wasn't pingable, but where did you ping from?). Someone recently here mentioned having two default routes installed because in.routed got fed bad information, for example. Dan From gate03 at landcroft.co.uk Tue Dec 23 02:29:10 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 12:29:10 +1000 Subject: [OmniOS-discuss] NTP server needs restarting to be usable Message-ID: <20141223122910.7457532c@emeritus> After the reboot referred-to in my earlier message, the NTP server wasn't really working properly. At first ntpclient on a Linux box was reporting: 41994 07614.540 rejected packet: LI==3 41994 07629.556 rejected packet: LI==3 41994 07644.571 rejected packet: LI==3 41994 07659.586 rejected packet: LI==3 then after "svcadm disable ntp ; svcadm enable ntp" on the server: 41994 07860.947 rejected packet: abs(DISP)>65536 41994 07875.963 rejected packet: abs(DISP)>65536 41994 07890.978 rejected packet: abs(DISP)>65536 41994 07905.993 rejected packet: abs(DISP)>65536 then after "svcadm disable ntp ; sleep 30 ; svcadm enable ntp": 41994 07996.069 rejected packet: abs(DISP)>65536 41994 08011.084 397.0 122.7 -5266.1 215988.2 -3528976 41994 08026.099 334.0 113.8 -5872.0 216217.0 -3528976 41994 08041.115 345.0 122.2 -6198.3 216445.9 -3528976 so it took two restarts to get it going properly. Any ideas why ? /etc/inet/ntp.conf is: driftfile /var/ntp/ntp.drift statsdir /var/ntp/ntpstats/ filegen peerstats file peerstats type day enable filegen loopstats file loopstats type day enable restrict default ignore # Deny everyone by default restrict 127.0.0.1 # Allow localhost full access restrict 192.168.1.0 mask 255.255.255.0 kod nomodify notrap nopeer pool 0.pool.ntp.org iburst pool 1.pool.ntp.org iburst pool 2.pool.ntp.org iburst pool 3.pool.ntp.org iburst restrict 0.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery restrict 1.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery restrict 2.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery restrict 3.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery Michael. From protonwrangler at gmail.com Tue Dec 23 02:34:52 2014 From: protonwrangler at gmail.com (Warren Marts) Date: Mon, 22 Dec 2014 19:34:52 -0700 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223105153.5d50749a@emeritus> References: <20141223105153.5d50749a@emeritus> Message-ID: KVM virtual machines on illumos each already run in a zone - with a specific minimal brand, and qemu-kvm as the zone's primary process. So you can apply global zone resources limits to arbitrate each vm. On Mon, Dec 22, 2014 at 5:51 PM, Michael Mounteney wrote: > > Has anyone done this? The main obstacle in my own investigation is > that the zonecfg command "add device / set match=/dev/net/vnick0 ; end" > is ignored so I can't network the VM. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Dec 23 02:40:30 2014 From: danmcd at omniti.com (Dan McDonald) Date: Mon, 22 Dec 2014 21:40:30 -0500 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: <20141223122910.7457532c@emeritus> References: <20141223122910.7457532c@emeritus> Message-ID: > On Dec 22, 2014, at 9:29 PM, Michael Mounteney wrote: > > After the reboot referred-to in my earlier message, the NTP server > wasn't really working properly. At first ntpclient on a Linux box was > reporting: Did you "pkg update" to the very latest NTP server? There's a security vulnerability attached to it, you know. And you must "svcadm restart ntp" after the update. Dan From gate03 at landcroft.co.uk Tue Dec 23 02:42:15 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 12:42:15 +1000 Subject: [OmniOS-discuss] sudden loss of networking In-Reply-To: References: <20141223120402.6714c763@emeritus> Message-ID: <20141223124215.534116b3@emeritus> On Mon, 22 Dec 2014 21:27:00 -0500 Dan McDonald wrote: > Lots of possibilities when you "lose networking". Lots of > information helps tell what happened. > > Output of these commands: > - ifconfig -a > - dladm show-link > - dladm show-phys > - dladm show-ether > - netstat -rnv Thanks Dan; I've saved those commands in a script which hopefully I'll be able to run if it happens again, via the serial port. > Also, make sure you didn't lose routing, as opposed to losing the > link. See if a node on the same subnet can ping it (you said it > wasn't pingable, but where did you ping from?). I was pinging from a machine on the same subnet; it's just a simple home setup here with two interfaces: one connected to the ISP's cable modem and the other to a hub/router and all the other machines, on the 192.168.1.0/24 subnet. > Someone recently here mentioned having two default routes installed > because in.routed got fed bad information, for example. Maybe the diagnostics will reveal this to be the cause. Michael. From gate03 at landcroft.co.uk Tue Dec 23 02:46:16 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 12:46:16 +1000 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: References: <20141223122910.7457532c@emeritus> Message-ID: <20141223124616.6298d7b9@emeritus> On Mon, 22 Dec 2014 21:40:30 -0500 Dan McDonald wrote: > Did you "pkg update" to the very latest NTP server? There's a > security vulnerability attached to it, you know. Oh yes. root at world:/root# pkg list ntp NAME (PUBLISHER) VERSION IFO service/network/ntp 4.2.8-0.151012 i-- > And you must "svcadm restart ntp" after the update. This only happened after the machine rebooted, as referred-to in my earlier message. Maybe something to do with the server coming up before networking is ready? Just a thought. Michael. From bdha at mirrorshades.net Tue Dec 23 02:50:58 2014 From: bdha at mirrorshades.net (Bryan Horstmann-Allen) Date: Mon, 22 Dec 2014 21:50:58 -0500 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: References: <20141223105153.5d50749a@emeritus> Message-ID: <20141223025058.GB98666@lab.int.icgroup.com> +------------------------------------------------------------------------------ | On 2014-12-22 19:34:52, Warren Marts wrote: | | KVM virtual machines on illumos each already run in a zone - with a | specific minimal brand, and qemu-kvm as the zone's primary process. So you | can apply global zone resources limits to arbitrate each vm. This is only true on SmartOS, not OmniOS. -- bdha From jesus at omniti.com Tue Dec 23 03:07:57 2014 From: jesus at omniti.com (Theo Schlossnagle) Date: Mon, 22 Dec 2014 22:07:57 -0500 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: References: <20141223105153.5d50749a@emeritus> Message-ID: This doesn't happen unless you explicit do that on "illumos." You are describing the default behavior on SmartOS, qemu-kvm is just a process on illumos, you'd have to setup the minimal zone, etc. On Mon, Dec 22, 2014 at 9:34 PM, Warren Marts wrote: > > KVM virtual machines on illumos each already run in a zone - with a > specific minimal brand, and qemu-kvm as the zone's primary process. So you > can apply global zone resources limits to arbitrate each vm. > > On Mon, Dec 22, 2014 at 5:51 PM, Michael Mounteney > wrote: >> >> Has anyone done this? The main obstacle in my own investigation is >> that the zonecfg command "add device / set match=/dev/net/vnick0 ; end" >> is ignored so I can't network the VM. >> >> Michael. >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Theo Schlossnagle http://omniti.com/is/theo-schlossnagle -------------- next part -------------- An HTML attachment was scrubbed... URL: From takashiary at gmail.com Tue Dec 23 03:50:24 2014 From: takashiary at gmail.com (takashi ary) Date: Tue, 23 Dec 2014 12:50:24 +0900 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: <20141223124616.6298d7b9@emeritus> References: <20141223122910.7457532c@emeritus> <20141223124616.6298d7b9@emeritus> Message-ID: Hello, please try using the following setup ------------------------------------------------------------ svccfg -s svc:/network/ntp addpg name-services dependency svccfg -s svc:/network/ntp setprop name-services/entities = fmri: svc:/milestone/name-services:default svccfg -s svc:/network/ntp setprop name-services/grouping = astring: optional_all svccfg -s svc:/network/ntp setprop name-services/restart_on = astring: none svccfg -s svc:/network/ntp setprop name-services/type= astring: service svccfg -s svc:/network/ntp addpg routing-setup dependency svccfg -s svc:/network/ntp setprop routing-setup/entities = fmri: svc:/network/routing-setup:default svccfg -s svc:/network/ntp setprop routing-setup/grouping = astring: optional_all svccfg -s svc:/network/ntp setprop routing-setup/restart_on = astring: none svccfg -s svc:/network/ntp setprop routing-setup/type= astring: service ------------------------------------------------------------ Thanks From gate03 at landcroft.co.uk Tue Dec 23 05:16:28 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 15:16:28 +1000 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: References: <20141223122910.7457532c@emeritus> <20141223124616.6298d7b9@emeritus> Message-ID: <20141223151628.597a12b0@emeritus> On Tue, 23 Dec 2014 12:50:24 +0900 takashi ary wrote: > Hello, > > please try using the following setup > [...] Thank you Takashi: that does appear to fix the problem, although the skew hovers around 5 ms. Michael. From sjorge+ml at blackdot.be Tue Dec 23 08:01:16 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Tue, 23 Dec 2014 09:01:16 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223105153.5d50749a@emeritus> References: <20141223105153.5d50749a@emeritus> Message-ID: <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> IIRC setting the zone to exclusive netstack works to include the /dev/net/vnicX device: --- set ip-type=exclusive add net set physical=zleonov0 end --- -(~)-[.]-{ ls -l /dev/net }-(root at leonov)- total 0 crw-rw-rw- 1 root sys 263, 1010 Dec 20 18:59 zleonov0 --- Regards Jorge On 2014-12-23 01:51, Michael Mounteney wrote: > Has anyone done this? The main obstacle in my own investigation is > that the zonecfg command "add device / set match=/dev/net/vnick0 ; end" > is ignored so I can't network the VM. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From gate03 at landcroft.co.uk Tue Dec 23 09:28:47 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 19:28:47 +1000 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> Message-ID: <20141223192847.74648386@emeritus> On Tue, 23 Dec 2014 09:01:16 +0100 Jorge Schrauwen wrote: > IIRC setting the zone to exclusive netstack works to include the > /dev/net/vnicX device: That gets it a bit closer but I still need the real interface (e1000g1) for vnc and sshing in etc. and I can't give that to the zone exclusively. Michael. From sjorge+ml at blackdot.be Tue Dec 23 09:42:19 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Tue, 23 Dec 2014 10:42:19 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223192847.74648386@emeritus> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> Message-ID: Well just use a socket for vnc, then you can spawn socat in the gz to explose the socket over tcp :) This deals with a serial port as a socket... https://blackdot.be/2013/07/qemu-kvm-monitor-and-serial-console-over-sockets-with-minicom/ But: socat unix-listen:/vms/leonov/run/vnc.sock tcp4:0.0.0.0:5901 & works just as well :) For qemu the option is -vnc unix:/run/vnc.sock (assuming /vms/leonov is your zone root) I tend do drop vnc, serial0 and monitoring interface as unix sockets for all my qemu processes. Then I use a small wrapper I wrote (kvmcon - https://github.com/sjorge/omnios-build-blackdot/blob/master/build/system-kvmcon/files/kvmcon) to connect to them. I use this in combination with kvmadm, works reasonably well. This should not be to hard to adapt to a qemu process running in a zone. Regards Jorge On 2014-12-23 10:28, Michael Mounteney wrote: > On Tue, 23 Dec 2014 09:01:16 +0100 > Jorge Schrauwen wrote: > >> IIRC setting the zone to exclusive netstack works to include the >> /dev/net/vnicX device: > > That gets it a bit closer but I still need the real interface (e1000g1) > for vnc and sshing in etc. and I can't give that to the zone > exclusively. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From lotheac at iki.fi Tue Dec 23 09:44:01 2014 From: lotheac at iki.fi (Lauri Tirkkonen) Date: Tue, 23 Dec 2014 11:44:01 +0200 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223192847.74648386@emeritus> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> Message-ID: <20141223094401.GA18650@gutsman.lotheac.fi> On Tue, Dec 23 2014 19:28:47 +1000, Michael Mounteney wrote: > That gets it a bit closer but I still need the real interface (e1000g1) > for vnc and sshing in etc. and I can't give that to the zone > exclusively. You can create a vnic over your physical interface (with dladm(1M)) and then give that to the zone. -- Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet From natxo.asenjo at gmail.com Tue Dec 23 09:56:30 2014 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Tue, 23 Dec 2014 10:56:30 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223094401.GA18650@gutsman.lotheac.fi> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> Message-ID: On Tue, Dec 23, 2014 at 10:44 AM, Lauri Tirkkonen wrote: > On Tue, Dec 23 2014 19:28:47 +1000, Michael Mounteney wrote: > > That gets it a bit closer but I still need the real interface (e1000g1) > > for vnc and sshing in etc. and I can't give that to the zone > > exclusively. > > You can create a vnic over your physical interface (with dladm(1M)) and > then give that to the zone. > my thoghts exactly. This is what I do in such cases: First let's see the real network hardware: # dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE bge0 Ethernet up 1000 full bge0 then create a vnic on top of that # dladm create-vnic -l bge0 vnic1 check it's there: # dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE VID vnic1 bge0 1000 2:8:20:ae:f6:c0 random 0 create a zone and assign it the new vnic: global # zonecfg -z sapacc sapacc: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:sapacc> create zonecfg:sapacc> set zonepath=/export/zones/sapacc zonecfg:sapacc> set autoboot=false zonecfg:sapacc> set ip-type=exclusive zonecfg:sapacc> add net zonecfg:sapacc:net> set physical=vnic1 zonecfg:sapacc:net> end zonecfg:sapacc> verify zonecfg:sapacc> commit zonecfg:sapacc> exit And then configure the vnic inside the zone like you would normally -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Tue Dec 23 10:48:15 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Tue, 23 Dec 2014 20:48:15 +1000 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> Message-ID: <20141223204815.63e06058@emeritus> On Tue, 23 Dec 2014 10:56:30 +0100 Natxo Asenjo wrote: Thanks to you and everyone else for their help. I hope I'm not polluting the list with newbie noise but hopefully this will stay on record and help out someone else later. My knowledge of vnics, sockets etc. is thin, to say the least. It is now working in a fashion. Here's the KVM zone: root at world:/root# zonecfg -z KVM export create -b set zonepath=/zone/KVM set brand=ipkg set autoboot=true set ip-type=exclusive add net set physical=vnick0 end add device set match=/dev/kvm end add device set match=/dev/dld end add device set match=/dev/zvol/dsk/rpool/vol/* end add dataset set name=rpool/vol end I don't need that dataset but anyway. Within the zone, the kvm launch command is: /usr/bin/qemu-system-x86_64 \ -name "Gentoo XFCE" \ -cpu host \ -boot order=d \ -enable-kvm \ -chardev socket,id=monitor,path=/var/run/monitor.sock,server,nowait \ -monitor chardev:monitor \ -chardev socket,id=serial0,path=/var/run/console.sock,server,nowait \ -serial chardev:serial0 \ -vnc unix:/var/run/XFCE.sock \ -smp 1,maxcpus=1,cores=2 \ -m 1024 \ -no-hpet \ -localtime \ -kernel ./bzImage \ -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \ -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-XFCE,cache=none,if=virtio,index=0 \ -drive file=/dev/zvol/dsk/rpool/vol/Linuswap-XFCE,cache=none,if=virtio,index=1 \ -net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \ -net vnic,vlan=0,name=net0,ifname=5905,macaddr=$mac \ -vga std \ -daemonize and that works inasmuch as the guest assigns an IP of 192.168.1.47 to its interface, and I can ssh and xdmcp into it from elsewhere. Now the three sockets are visible in the global zone: root at world:/root# ls -ltr /zone/KVM/root/var/run/*.sock srwxr-xr-x 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/monitor.sock srwxr-xr-x 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/console.sock srwxr-x--- 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/XFCE.sock so how would I connect a VNC viewer on another machine on the same subnet to the VM ? I need this for when the guest fails to start up for whatever reason. Michael. From vab at bb-c.de Tue Dec 23 15:20:36 2014 From: vab at bb-c.de (Volker A. Brandt) Date: Tue, 23 Dec 2014 16:20:36 +0100 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: References: <20141223122910.7457532c@emeritus> Message-ID: <21657.34884.272541.710083@glaurung.bb-c.de> Hi Dan! Thanks for all of your efforts. OmniOS is a solid part of our IT infrastucture here. You wrote (to Michael Mounteney): > Did you "pkg update" to the very latest NTP server? There's a > security vulnerability attached to it, you know. As an aside, I just updated from 10 to 12 and am now running: # cat /etc/release OmniOS v11 r151012 Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved. Use is subject to license terms. # uname -a SunOS nfs2 5.11 omnios-10b9c79 i86pc i386 i86pc Update was straightforward and worked just fine. But I noticed a strange path in the /usr/share/doc/ntp4 subdirectory: # pkg list -Hv ntp at latest pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z i-- # pkg contents -H pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z etc/inet etc/inet/ntp.client [...] usr/share/doc/ntp4/ntpsweep.html usr/share/doc/ntp4/ntptrace.html usr/share/doc/ntp4/{} usr/share/doc/ntp4/{}/9400n.jpg usr/share/doc/ntp4/{}/a-ux usr/share/doc/ntp4/{}/access.html usr/share/doc/ntp4/{}/accopt.html usr/share/doc/ntp4/{}/accopt.txt usr/share/doc/ntp4/{}/aix usr/share/doc/ntp4/{}/alice11.gif usr/share/doc/ntp4/{}/alice13.gif usr/share/doc/ntp4/{}/alice15.gif usr/share/doc/ntp4/{}/alice23.gif usr/share/doc/ntp4/{}/alice31.gif usr/share/doc/ntp4/{}/alice32.gif usr/share/doc/ntp4/{}/alice35.gif usr/share/doc/ntp4/{}/alice38.gif usr/share/doc/ntp4/{}/alice44.gif usr/share/doc/ntp4/{}/alice47.gif usr/share/doc/ntp4/{}/alice51.gif usr/share/doc/ntp4/{}/alice61.gif usr/share/doc/ntp4/{}/assoc.html usr/share/doc/ntp4/{}/audio.html usr/share/doc/ntp4/{}/audio.txt [...] I guess these braces come from a failed variable expansion... :-) > And you must "svcadm restart ntp" after the update. Couldn't this be automated by tacking a restart_fmri attribute on to some file that is guaranteed to be different, such as the ntpd binary? (It needs to be different from the existing binary for the restart_fmri to fire.) E.g. file 8b55f1839681e3b537177922f31821a19d5f5ba4 path=usr/lib/inet/ntpd \ owner=root group=bin mode=0555 \ chash=4da9e4ef17575df37441057da515a15e57ab4938 elfarch=i386 elfbits=32 \ elfhash=991e353f618ea696ebe95403e70519d71cb3d0b2 pkg.csize=403734 \ pkg.size=1013996 restart_fmri=svc:/network/ntp:default Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Oracle Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim, GERMANY Email: vab at bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgr??e: 46 Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt "When logic and proportion have fallen sloppy dead" From danmcd at omniti.com Tue Dec 23 15:40:47 2014 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 23 Dec 2014 10:40:47 -0500 Subject: [OmniOS-discuss] NTP server needs restarting to be usable In-Reply-To: <21657.34884.272541.710083@glaurung.bb-c.de> References: <20141223122910.7457532c@emeritus> <21657.34884.272541.710083@glaurung.bb-c.de> Message-ID: > On Dec 23, 2014, at 10:20 AM, Volker A. Brandt wrote: > Update was straightforward and worked just fine. But I noticed a strange > path in the /usr/share/doc/ntp4 subdirectory: > > # pkg list -Hv ntp at latest > pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z i-- > > # pkg contents -H pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z > etc/inet > etc/inet/ntp.client > [...] > usr/share/doc/ntp4/ntpsweep.html > usr/share/doc/ntp4/ntptrace.html > usr/share/doc/ntp4/{} > usr/share/doc/ntp4/{}/9400n.jpg > usr/share/doc/ntp4/{}/a-ux > usr/share/doc/ntp4/{}/access.html > usr/share/doc/ntp4/{}/accopt.html > usr/share/doc/ntp4/{}/accopt.txt > usr/share/doc/ntp4/{}/aix > usr/share/doc/ntp4/{}/alice11.gif > usr/share/doc/ntp4/{}/alice13.gif > usr/share/doc/ntp4/{}/alice15.gif > usr/share/doc/ntp4/{}/alice23.gif > usr/share/doc/ntp4/{}/alice31.gif > usr/share/doc/ntp4/{}/alice32.gif > usr/share/doc/ntp4/{}/alice35.gif > usr/share/doc/ntp4/{}/alice38.gif > usr/share/doc/ntp4/{}/alice44.gif > usr/share/doc/ntp4/{}/alice47.gif > usr/share/doc/ntp4/{}/alice51.gif > usr/share/doc/ntp4/{}/alice61.gif > usr/share/doc/ntp4/{}/assoc.html > usr/share/doc/ntp4/{}/audio.html > usr/share/doc/ntp4/{}/audio.txt > [...] > > I guess these braces come from a failed variable expansion... :-) It's possible they came in with the 4.2.7 update (which was introduced in 012). >> And you must "svcadm restart ntp" after the update. > > Couldn't this be automated by tacking a restart_fmri attribute on to > some file that is guaranteed to be different, such as the ntpd binary? > (It needs to be different from the existing binary for the > restart_fmri to fire.) > > E.g. > > file 8b55f1839681e3b537177922f31821a19d5f5ba4 path=usr/lib/inet/ntpd \ > owner=root group=bin mode=0555 \ > chash=4da9e4ef17575df37441057da515a15e57ab4938 elfarch=i386 elfbits=32 \ > elfhash=991e353f618ea696ebe95403e70519d71cb3d0b2 pkg.csize=403734 \ > pkg.size=1013996 restart_fmri=svc:/network/ntp:default If that change happens, it should probably be placed somewhere in a .mog file in the ntp build directory under omnios-build. Dan From gate03 at landcroft.co.uk Tue Dec 23 20:14:46 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 24 Dec 2014 06:14:46 +1000 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223204815.63e06058@emeritus> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> <20141223204815.63e06058@emeritus> Message-ID: <20141224061446.078bf13d@emeritus> Brilliant, with a bit more fiddling I've got it so that the KVM instances can be started within the child zone, AND shut down cleanly via the command "echo system_powerdown | nc -U /var/run/KDE.monsock" so the next step is to put them into a service so that they can be brought up and down within SMF. Thanks again to all for their help. Michael. From hasslerd at gmx.li Tue Dec 23 23:00:03 2014 From: hasslerd at gmx.li (Dominik Hassler) Date: Wed, 24 Dec 2014 00:00:03 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141224061446.078bf13d@emeritus> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> <20141223204815.63e06058@emeritus> <20141224061446.078bf13d@emeritus> Message-ID: <5499F3F3.7010408@gmx.li> Michael, you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ). that'll help you putting your kvms under smf control. at least you should be able to extract all the infos from there... On 12/23/2014 09:14 PM, Michael Mounteney wrote: > Brilliant, with a bit more fiddling I've got it so that the KVM > instances can be started within the child zone, AND shut down cleanly > via the command "echo system_powerdown | nc -U /var/run/KDE.monsock" so > the next step is to put them into a service so that they can be brought > up and down within SMF. > > Thanks again to all for their help. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0xF9ECC6A5.asc Type: application/pgp-keys Size: 2686 bytes Desc: not available URL: From gate03 at landcroft.co.uk Tue Dec 23 23:56:22 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 24 Dec 2014 09:56:22 +1000 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <5499F3F3.7010408@gmx.li> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> <20141223204815.63e06058@emeritus> <20141224061446.078bf13d@emeritus> <5499F3F3.7010408@gmx.li> Message-ID: <20141224095622.3867fe34@emeritus> On Wed, 24 Dec 2014 00:00:03 +0100 Dominik Hassler wrote: > Michael, > > you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ). > > that'll help you putting your kvms under smf control. > > at least you should be able to extract all the infos from there... Thanks Dominic, yes, very similar to Jorges suggestion. I have to work out how to do instances with SMF, because I have two VMs, which differ in (1) into which image they boot and (2) the port number of the VNC connection. It'll be very nice to have everything under proper control. Michael. From sjorge+ml at blackdot.be Wed Dec 24 08:42:39 2014 From: sjorge+ml at blackdot.be (Jorge Schrauwen) Date: Wed, 24 Dec 2014 09:42:39 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141224095622.3867fe34@emeritus> References: <20141223105153.5d50749a@emeritus> <93bb2197e9cbb0d20a5968402954dd05@blackdot.be> <20141223192847.74648386@emeritus> <20141223094401.GA18650@gutsman.lotheac.fi> <20141223204815.63e06058@emeritus> <20141224061446.078bf13d@emeritus> <5499F3F3.7010408@gmx.li> <20141224095622.3867fe34@emeritus> Message-ID: <032d07135554c37dbcebe78b144e9da1@blackdot.be> Well I actually suggested Dominik's kvmadm so they are identical not similar :D kvmadm can certainly do that, lots of features have been added over the last few months and It's pretty solid now. Dominik also has an amazing response time if you open a feature request. Regards Jorge On 2014-12-24 00:56, Michael Mounteney wrote: > On Wed, 24 Dec 2014 00:00:03 +0100 > Dominik Hassler wrote: > >> Michael, >> >> you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ). >> >> that'll help you putting your kvms under smf control. >> >> at least you should be able to extract all the infos from there... > > Thanks Dominic, yes, very similar to Jorges suggestion. I have to work > out how to do instances with SMF, because I have two VMs, which differ > in (1) into which image they boot and (2) the port number of the VNC > connection. It'll be very nice to have everything under proper > control. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From jimklimov at cos.ru Thu Dec 25 10:18:01 2014 From: jimklimov at cos.ru (Jim Klimov) Date: Thu, 25 Dec 2014 11:18:01 +0100 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: <20141223105153.5d50749a@emeritus> References: <20141223105153.5d50749a@emeritus> Message-ID: 23 ??????? 2014??. 1:51:53 CET, Michael Mounteney ?????: >Has anyone done this? The main obstacle in my own investigation is >that the zonecfg command "add device / set match=/dev/net/vnick0 ; end" >is ignored so I can't network the VM. > >Michael. >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Are ypu sure tuere is no typo? i.e. vnick0 vs. vnic0 ? -- Typos courtesy of K-9 Mail on my Samsung Android From gate03 at landcroft.co.uk Thu Dec 25 10:26:39 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Thu, 25 Dec 2014 20:26:39 +1000 Subject: [OmniOS-discuss] KVM within a child zone In-Reply-To: References: <20141223105153.5d50749a@emeritus> Message-ID: <20141225202639.48716756@emeritus> On Thu, 25 Dec 2014 11:18:01 +0100 Jim Klimov wrote: > Are ypu sure tuere is no typo? > i.e. vnick0 vs. vnic0 ? Quite sure, Jim. My choice of name, whimsical as it might be. Michael. From sergei25 at gmail.com Fri Dec 26 22:36:25 2014 From: sergei25 at gmail.com (sergei) Date: Fri, 26 Dec 2014 14:36:25 -0800 Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under installer ? Message-ID: Hi The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which scsi_vhci won't take over without proper conf file listing this model under "scsi-vhci-failover-override" line. Right now those disk device path starts with /pci instead of /scsi_vhci. Yet they are showing up in format output ok. What is the trick to fix this without rebuilding install ISO image ? I could install into one of (larger) Seagates and then mirror/remove mirror to move boot OS to the proper disks. Is there any easier way ? I think it would benefit Omni if you could keep scsi_vhci with at least some updates. I see Nexenta does include bunch of models into it's default scsi_vhci.conf. S -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.elling at richardelling.com Sat Dec 27 18:54:01 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Sat, 27 Dec 2014 10:54:01 -0800 Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under installer ? In-Reply-To: References: Message-ID: <2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com> > On Dec 26, 2014, at 2:36 PM, sergei wrote: > > Hi > > The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which scsi_vhci won't take over without proper conf file listing this model under "scsi-vhci-failover-override" line. Right now those disk device path starts with /pci instead of /scsi_vhci. Yet they are showing up in format output ok. What is the trick to fix this without rebuilding install ISO image ? easy -- don't rebuild the install ISO image :-) > > I could install into one of (larger) Seagates and then mirror/remove mirror to move boot OS to the proper disks. Is there any easier way ? ugh, too much work. Try this (I'm sure I blogged this a few times, or maybe in the Nexenta knowledge base?) I'm sure the procedure is in the email archives... 1. go ahead and do the installation. 2. boot into newly installed OS 3. edit scsi_vhci.conf 4. shutdown 5. boot from install media, go to shell 6. import rpool 7. export rpool 8. reboot into newly installed OS ZFS is tolerant of path changes, but you have to trick the boot process. > > I think it would benefit Omni if you could keep scsi_vhci with at least some updates. I see Nexenta does include bunch of models into it's default scsi_vhci.conf. The root cause is a deficiency in detecting multiple ports. The workaround is to override in scsi_vhci.conf. The fix is known, just need to find the time... -- richard From sergei25 at gmail.com Sat Dec 27 19:26:48 2014 From: sergei25 at gmail.com (sergei) Date: Sat, 27 Dec 2014 11:26:48 -0800 Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under installer ? In-Reply-To: <2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com> References: <2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com> Message-ID: Richard The thing is - installer does not show those two disks in the list. So I can't install into the disks that are supposed to be boot disks. There only larger Seagate drives offered as install target. My guess was that installed filters disks by their device path, not letting /pci ones through ? That is why I wanted to have proper scsi_vhci before the installer. Was hoping this can be done with mdb. Re: scsi_vhci.conf - systems with 3+ years uptime now are very picky about disk replacements (model/vendor). Almost have to feed them Seagate exclusively. If I told anyone that this server won't take TOSHIBA SAS drive but will be more happy with Seagate - I doubt many people would take it seriously. And these days I see lots of TOSHIBA disks that come as replacements. Almost makes one wish for a *some* tool to add entries to vhci and make them active at runtime. On Sat, Dec 27, 2014 at 10:54 AM, Richard Elling < richard.elling at richardelling.com> wrote: > > > On Dec 26, 2014, at 2:36 PM, sergei wrote: > > > > Hi > > > > The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which > scsi_vhci won't take over without proper conf file listing this model under > "scsi-vhci-failover-override" line. Right now those disk device path starts > with /pci instead of /scsi_vhci. Yet they are showing up in format output > ok. What is the trick to fix this without rebuilding install ISO image ? > > easy -- don't rebuild the install ISO image :-) > > > > > I could install into one of (larger) Seagates and then mirror/remove > mirror to move boot OS to the proper disks. Is there any easier way ? > > ugh, too much work. > Try this (I'm sure I blogged this a few times, or maybe in the Nexenta > knowledge base?) > I'm sure the procedure is in the email archives... > > 1. go ahead and do the installation. > 2. boot into newly installed OS > 3. edit scsi_vhci.conf > 4. shutdown > 5. boot from install media, go to shell > 6. import rpool > 7. export rpool > 8. reboot into newly installed OS > > ZFS is tolerant of path changes, but you have to trick the boot process. > > > > > I think it would benefit Omni if you could keep scsi_vhci with at least > some updates. I see Nexenta does include bunch of models into it's default > scsi_vhci.conf. > > The root cause is a deficiency in detecting multiple ports. The workaround > is to override > in scsi_vhci.conf. The fix is known, just need to find the time... > -- richard > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gate03 at landcroft.co.uk Mon Dec 29 06:40:29 2014 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Mon, 29 Dec 2014 16:40:29 +1000 Subject: [OmniOS-discuss] Postgres on Stable In-Reply-To: References: <20141128191821.43382469@punda-mlia> <20141129071835.1daa0799@punda-mlia> Message-ID: <20141229164029.0b4c386c@emeritus> On Fri, 28 Nov 2014 18:37:09 -0500 Zach Malone wrote: > I built postgresql-935 for you on a r151012 system, and published it > to the omniti-ms repo. Want to give it a shot? Past installing it, I > have not tried it on a production system (I'm planning on moving some > systems to r151012 this month), and I have not rolled any of the other > postgres libraries or modules that we publish for 9.2. Zac, can I be really cheeky and ask for this for bloody, r151013 ? I've just switched over to that distro, and there are no versions of Postgres available from the standard repositories. Much thanks in anticipation. Michael. From chip at innovates.com Mon Dec 29 12:52:04 2014 From: chip at innovates.com (Schweiss, Chip) Date: Mon, 29 Dec 2014 06:52:04 -0600 Subject: [OmniOS-discuss] Postgres on Stable In-Reply-To: <20141229164029.0b4c386c@emeritus> References: <20141128191821.43382469@punda-mlia> <20141129071835.1daa0799@punda-mlia> <20141229164029.0b4c386c@emeritus> Message-ID: I use OpenCSW for Postgres and many other packages on OmniOS. While targeted at Solaris, OpenCSW packages have worked flawlessly for me on OmniOS. http://www.opencsw.org/ -Chip On Mon, Dec 29, 2014 at 12:40 AM, Michael Mounteney wrote: > On Fri, 28 Nov 2014 18:37:09 -0500 > Zach Malone wrote: > > > I built postgresql-935 for you on a r151012 system, and published it > > to the omniti-ms repo. Want to give it a shot? Past installing it, I > > have not tried it on a production system (I'm planning on moving some > > systems to r151012 this month), and I have not rolled any of the other > > postgres libraries or modules that we publish for 9.2. > > Zac, can I be really cheeky and ask for this for bloody, r151013 ? > I've just switched over to that distro, and there are no versions of > Postgres available from the standard repositories. > > Much thanks in anticipation. > > Michael. > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Mon Dec 29 21:59:22 2014 From: rt at steait.net (Rune Tipsmark) Date: Mon, 29 Dec 2014 21:59:22 +0000 Subject: [OmniOS-discuss] zfs destroy takes forever, how do I set async_destroy? Message-ID: <1419890360886.59904@steait.net> hi all, as stated above, I have a server where I am syncing some 70 or so TB from and it has some very large snapshots and a destroy takes forever... has run for hours and hours now, 100% disk busy...I read there was a feature flag async_destroy but I don't seem to be able to find it. Any ideas? the box is more or less useless and I have more rather large volumes I need to sync to a remote device. br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From rt at steait.net Wed Dec 31 03:21:44 2014 From: rt at steait.net (Rune Tipsmark) Date: Wed, 31 Dec 2014 03:21:44 +0000 Subject: [OmniOS-discuss] zfs destroy takes forever, how do I set async_destroy? In-Reply-To: <1419890360886.59904@steait.net> References: <1419890360886.59904@steait.net> Message-ID: <1419996103098.13209@steait.net> I found out the feature is already enabled, I guess destroying very large snapshots just takes a very long time regardless... br, Rune ________________________________ From: OmniOS-discuss on behalf of Rune Tipsmark Sent: Monday, December 29, 2014 10:59 PM To: omnios-discuss Subject: [OmniOS-discuss] zfs destroy takes forever, how do I set async_destroy? hi all, as stated above, I have a server where I am syncing some 70 or so TB from and it has some very large snapshots and a destroy takes forever... has run for hours and hours now, 100% disk busy...I read there was a feature flag async_destroy but I don't seem to be able to find it. Any ideas? the box is more or less useless and I have more rather large volumes I need to sync to a remote device. br, Rune -------------- next part -------------- An HTML attachment was scrubbed... URL: From Kevin.Swab at ColoState.EDU Wed Dec 31 19:25:00 2014 From: Kevin.Swab at ColoState.EDU (Kevin Swab) Date: Wed, 31 Dec 2014 12:25:00 -0700 Subject: [OmniOS-discuss] slow drive response times Message-ID: <54A44D8C.5090302@ColoState.EDU> Hello Everyone, We've been running OmniOS on a number of SuperMicro 36bay chassis, with Supermicro motherboards, LSI SAS controllers (9211-8i & 9207-8i) and various SAS HDD's. These systems are serving block storage via Comstar and Qlogic FC HBA's, and have been running well for several years. The problem we've got is that as the drives age, some of them start to perform slowly (intermittently) without failing - no zpool or iostat errors, and nothing logged in /var/adm/messages. The slow performance can be seen as high average service times in iostat or sar. When these service times get above 500ms, they start to cause IO timeouts on the downstream storage consumers, which is bad... I'm wondering - is there a way to tune OmniOS' behavior so that it doesn't try so hard to complete IOs to these slow disks, and instead just gives up and fails them? I found an old post from 2011 which states that some tunables exist, but are ignored by the mpt_sas driver: http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ Does anyone know the current status of these tunables, or have any other suggestions that might help? Thanks, Kevin -- ------------------------------------------------------------------- Kevin Swab UNIX Systems Administrator ACNS Colorado State University Phone: (970)491-6572 Email: Kevin.Swab at ColoState.EDU GPG Fingerprint: 7026 3F66 A970 67BD 6F17 8EB8 8A7D 142F 2392 791C From groups at tierarzt-mueller.de Wed Dec 31 20:40:48 2014 From: groups at tierarzt-mueller.de (Alexander Lesle) Date: Wed, 31 Dec 2014 21:40:48 +0100 Subject: [OmniOS-discuss] Kernel panic - I cant find the problem Message-ID: <1531165597.20141231214048@tierarzt-mueller.de> Hello All, today it was Update-time for my servers. After typing reboot the system hang on reboot. Ok, looking by iKVM what happend and there I see on the screen that the server was restarted after a kernel panic. I cant solve the problem because I dont understand where the problem is. Please help. ,-----[ ]----- | | root at aio:/root# fmdump -Vp -u da148174-fd91-4572-b639-b4fef8253f70 | TIME UUID | SUNW-MSG-ID | | Dec 16 2014 15:30:39.077759000 da148174-fd91-4572-b639-b4fef8253f70 | SUNOS-8000-KL | | | nvlist version: 0 | version = 0x0 | class = list.suspect | uuid = da148174-fd91-4572-b639-b4fef8253f70 | code = SUNOS-8000-KL | diag-time = 1418740239 75809 | de = fmd:///module/software-diagnosis | fault-list-sz = 0x1 | fault-list = (array of embedded nvlists) | (start fault-list[0]) | nvlist version: 0 | version = 0x0 | class = defect.sunos.kernel.panic | certainty = 0x64 | asru = | sw:///:path=/var/crash/unknown/.da148174-fd91-4572-b639-b4fef8253f70 | | resource = | sw:///:path=/var/crash/unknown/.da148174-fd91-4572-b639-b4fef8253f70 | | savecore-succcess = 1 | dump-dir = /var/crash/unknown | dump-files = vmdump.2 | os-instance-uuid = | da148174-fd91-4572-b639-b4fef8253f70 | | panicstr = BAD TRAP: type=e (#pf Page fault) | rp=ffffff00103bd790 addr=fffffffff7afe4f0 | | panicstack = unix:real_mode_stop_cpu_stage2_end+9e23 | () | unix:trap+db3 () | unix:cmntrap+e6 () | cmlb:cmlb_validate+0 () | | sd:sdopen+35f () | genunix:dev_open+30 () | specfs:spec_open+206 () | | genunix:fop_open+89 () | genunix:vn_openat+234 () | genunix:copen+20c | () | genunix:openat32+27 () | genunix:open32+25 () | | unix:brand_sys_sysenter+1c9 () | | | crashtime = 1418740182 | panic-time = Tue Dec 16 15:29:42 2014 CET | (end fault-list[0]) | | fault-status = 0x1 | severity = Major | __ttl = 0x1 | __tod = 0x5490420f 0x4a28218 | | root at aio:/root# | `------------------- Attachments / Dateianlagen: ---------------------------- [1] messages.1 Happy new year to you all. -- Best Regards Alexander Dezember, 31 2014 -------------- next part -------------- A non-text attachment was scrubbed... Name: messages.1 Type: application/octet-stream Size: 40334 bytes Desc: not available URL: From richard.elling at richardelling.com Wed Dec 31 22:22:13 2014 From: richard.elling at richardelling.com (Richard Elling) Date: Wed, 31 Dec 2014 14:22:13 -0800 Subject: [OmniOS-discuss] slow drive response times In-Reply-To: <54A44D8C.5090302@ColoState.EDU> References: <54A44D8C.5090302@ColoState.EDU> Message-ID: > On Dec 31, 2014, at 11:25 AM, Kevin Swab wrote: > > Hello Everyone, > > We've been running OmniOS on a number of SuperMicro 36bay chassis, with > Supermicro motherboards, LSI SAS controllers (9211-8i & 9207-8i) and > various SAS HDD's. These systems are serving block storage via Comstar > and Qlogic FC HBA's, and have been running well for several years. > > The problem we've got is that as the drives age, some of them start to > perform slowly (intermittently) without failing - no zpool or iostat > errors, and nothing logged in /var/adm/messages. The slow performance > can be seen as high average service times in iostat or sar. Look at the drive's error logs using sg_logs (-a for all) > > When these service times get above 500ms, they start to cause IO > timeouts on the downstream storage consumers, which is bad... 500 milliseconds is not unusual for a busy HDD with SCSI TCQ or SATA NCQ > > I'm wondering - is there a way to tune OmniOS' behavior so that it > doesn't try so hard to complete IOs to these slow disks, and instead > just gives up and fails them? Yes, the tuning in Alasdair's blog should work as he describes. More below... > > I found an old post from 2011 which states that some tunables exist, > but are ignored by the mpt_sas driver: > > http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ > > Does anyone know the current status of these tunables, or have any other > suggestions that might help? These tunables are on the order of seconds. The default, 60, is obviously too big unless you have old, slow, SCSI CD-ROMs. But setting it below the manufacturer's internal limit (default or tuned) can lead to an unstable system. Some vendors are better than others at documenting these, but in any case you'll need to see their spec. Expect values on the order of 6 to 15 seconds for modern HDDs and SSDs. There are a lot of tunables in this area at all levels of the architecture. OOB, the OmniOS settings ensure stable behaviour. Tuning any layer without understanding the others can lead to unstable systems, as demonstrated by your current downstream consumers. -- richard > > Thanks, > Kevin > > > -- > ------------------------------------------------------------------- > Kevin Swab UNIX Systems Administrator > ACNS Colorado State University > Phone: (970)491-6572 Email: Kevin.Swab at ColoState.EDU > GPG Fingerprint: 7026 3F66 A970 67BD 6F17 8EB8 8A7D 142F 2392 791C > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From henson at acm.org Wed Dec 31 22:50:28 2014 From: henson at acm.org (Paul B. Henson) Date: Wed, 31 Dec 2014 14:50:28 -0800 Subject: [OmniOS-discuss] adding cua/a as a second login In-Reply-To: <20141205085057.5e000d9e@punda-mlia> References: <20141204153051.3e17ac8f@punda-mlia> <20141205073111.0762e2b1@punda-mlia> <4cff01d01010$dc508510$94f18f30$@acm.org> <20141205085057.5e000d9e@punda-mlia> Message-ID: <20141231225028.GH29549@bender.unx.csupomona.edu> On Fri, Dec 05, 2014 at 08:50:57AM +1000, Michael Mounteney wrote: > Paul: https://www.illumos.org/issues/5391 I finished the code change to make this work. If you'd like to test it out, you can find an omnios-stable compatible binary at: https://www.cpp.edu/~henson/tmp/login You can either boot into a new BE to test it, or just make a backup copy of /usr/bin/login and copy in the test one. I tested it myself and believe it functions correctly, but be sure to have an active root login up in another window to either boot back into the original BE or copy the original login binary back into place if you have any problems. I left the console on the frame buffer and set up a secondary serial login as described at https://docu.blackdot.be/snipets/solaris/misc-serial-console Which is what I believe you did; with a setting of: CONSOLE=/dev/console,/dev/term/a I can log in as root on both the framebuffer console and the serial login. Note that setting CONSOLE to just /dev/console also implicitly allows the virtual terminals access as root too, adding the second device removes that implicit access so they would need to be enumerated individually to maintain it. They're not even enabled by default so that's probably not an issue for you. Now I just need to go update the documentation, put up a webrev, and see what the dev community thinks about it ;). Let me know if you have any questions or problems.