From andykinney at advantagecom.net Sat Oct 1 00:51:19 2016 From: andykinney at advantagecom.net (Andrew Kinney) Date: Fri, 30 Sep 2016 17:51:19 -0700 Subject: [OmniOS-discuss] smartctl and SCT ERC on omnios In-Reply-To: <57EEECB2.5030701@advantagecom.net> References: <57EEECB2.5030701@advantagecom.net> Message-ID: <57EF0887.6080501@advantagecom.net> To add to this, it *appears* I can *set* scterc on omnios, but not *read* scterc. # smartctl -l scterc,70,70 -d sat,12 /dev/rdsk/c4t0d0s0 smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org SCT Error Recovery Control set to: Read: 70 (7.0 seconds) Write: 70 (7.0 seconds) # smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0 smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org SMART WRITE LOG returns COUNT and LBA_LOW register unchanged SCT (Get) Error Recovery Control command failed Without being able to read the values, I'm taking it on faith that it is actually set since the set command didn't error out. Regarding the error message: "The meat of this patch extends the WRITE_LOG command to return the ATA registers back to the higher level code. As it was, smartmontools had some provision for returning 512 byte sectors as results of ioctl()s, but not the contents of the ATA registers. Before ERC appeared, no other ATA commands return a result in this way. When a read or write ERC time is requested, the current ERC value are returned in SectorCount and LBA_Low, as a low and high byte. The number is in deciseconds." This was from http://abatis.org.uk/projects/erc/ where he discusses how smartmontools was patched back in 2009 to get smartctl to read these values properly on Linux. It would seem smartctl is getting the values back, but on Solaris it doesn't know how to interpret and display the data. I hooked up one of these SSDs to a Windows machine and was able to read and set scterc. From the factory, it comes as "disabled", which explains the trouble I had with the long error recovery (2 minutes). -- Andrew Andrew Kinney wrote: > Hello, > > I have a Samsung 845DC Pro SSD that I need to respond more timely during > problems with flash cells. We had an incident where it took 2 minutes to > fail a cell erase and bring in a spare cell, which is too long for a > redundant ZFS log device on a pool with a dataset with property > sync=always serving iSCSI volumes. > > smartctl shows it supports SCT ERC: > > # smartctl -x -d sat,12 /dev/rdsk/c4t0d0s0 > > smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) > Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org > > === START OF INFORMATION SECTION === > Device Model: Samsung SSD 845DC PRO 400GB > > ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c > SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) > > SCT capabilities: (0x003d) SCT Status supported. > SCT Error Recovery Control supported. > SCT Feature Control supported. > SCT Data Table supported. > > > > I should be able to read its current SCT ERC settings, but get an error: > > # smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0 > smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build) > Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org > > SMART WRITE LOG returns COUNT and LBA_LOW register unchanged > SCT (Get) Error Recovery Control command failed > > > I would go read/write the scterc values on a different OS as a > workaround, but I've read that Samsung devices do not persist CCTL > values across reboots. At present, I don't know what it's set to or if > it's being honored. > > It appears that this is a limitation of smartmontools on Solaris and its > derivatives. > > Any ideas how I can read and set the scterc values on Omnios? > > -- > Andrew > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From danmcd at omniti.com Sat Oct 1 01:56:32 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 30 Sep 2016 21:56:32 -0400 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> Message-ID: <1B0C1A14-8979-4DBD-ACD0-B52D18E23643@omniti.com> > On Sep 30, 2016, at 6:57 PM, Michael Talbott wrote: > > Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html > > I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :( > > I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to. > > Any ideas on getting zvols as raw devices in an lx zone? You MIGHT be able to use zonecfg to bring one in. E.g. add device set match=/dev/zvol/dsk/data/oi-data-disk end I haven't tested this yet. That example three extra lines for zonecfg comes from a standard lipkg zone. ALSO, it may be the case that for an LX zone, such a zvol device entry would appear in /native. Dan From daleg at omniti.com Sat Oct 1 03:52:52 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 30 Sep 2016 23:52:52 -0400 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> Message-ID: > On Sep 30, 2016, at 6:57 PM, Michael Talbott wrote: > > Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html > > I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :( > > I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to. Negative, won't work because the illumos kernel doesn't grok ext4. In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs. This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure. For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs. /dale From skiselkov.ml at gmail.com Sat Oct 1 19:36:23 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sat, 1 Oct 2016 21:36:23 +0200 Subject: [OmniOS-discuss] rge_intr troubles Message-ID: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> So I'm playing around with a box that has an on-board Realtek NIC and periodically, about once every 2-5 minutes, the network just goes out to lunch and stops responding to ping or attempts to send anything from the box. I noticed that while doing so, the box is getting floored by interrupts from the NIC, so I see tons of rge_intr activity and one CPU core receiving about 160000 interrupts per second (other cores are idle). I've tried a number of things I've found on the internet, but to no avail. I tried setting msi_enable=0 and msix_enable=0 in /kernel/drv/rge.conf, but that didn't help. Disabling flow control via # ndd -set /dev/rge0 adv_pause_cap 0 # ndd -set /dev/rge0 adv_asym_pause_cap 0 didn't help either. I tried OmniOS r151018, OpenIndiana hipster (2016.04) and even the old OpenIndiana 151a8 (all 64-bit). Not being a driver person myself, I have no idea where to look next. I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config info. Also attached are screenshots of mpstat and dtrace profile at 997hz running on the CPU that's bogged down with interrupts. -- Saso -------------- next part -------------- A non-text attachment was scrubbed... Name: lspci.gz Type: application/x-gzip Size: 5507 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: prtconf.gz Type: application/x-gzip Size: 14324 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dtrace.jpg Type: image/jpeg Size: 55022 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mpstat.jpg Type: image/jpeg Size: 74331 bytes Desc: not available URL: From mir at miras.org Sat Oct 1 21:13:35 2016 From: mir at miras.org (Michael Rasmussen) Date: Sat, 1 Oct 2016 23:13:35 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> Message-ID: <20161001231335.03eca0ee@sleipner.datanom.net> On Sat, 1 Oct 2016 21:36:23 +0200 Saso Kiselkov wrote: > > Not being a driver person myself, I have no idea where to look next. > I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config > info. Also attached are screenshots of mpstat and dtrace profile at > 997hz running on the CPU that's bogged down with interrupts. > Have you tried to disable csum and hardware offloading? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: rugged, adj.: Too heavy to lift. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From skiselkov.ml at gmail.com Sat Oct 1 21:21:40 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sat, 1 Oct 2016 23:21:40 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <20161001231335.03eca0ee@sleipner.datanom.net> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> Message-ID: <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com> On 10/1/16 11:13 PM, Michael Rasmussen wrote: > hardware offloading Just found the article that mentions dohwcksum. Set it to 0 and testing now. Any other tunables I should be aware of? Cheers, -- Saso -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Sat Oct 1 21:32:06 2016 From: mir at miras.org (Michael Rasmussen) Date: Sat, 1 Oct 2016 23:32:06 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com> Message-ID: <20161001233206.154cb73a@sleipner.datanom.net> On Sat, 1 Oct 2016 23:21:40 +0200 Saso Kiselkov wrote: > On 10/1/16 11:13 PM, Michael Rasmussen wrote: > > hardware offloading > > Just found the article that mentions dohwcksum. Set it to 0 and testing > now. Any other tunables I should be aware of? > LRO: https://docs.oracle.com/cd/E53394_01/html/E54790/goryb.html -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: If it doesn't smell yet, it's pretty fresh. -- Dave Johnson, on dead seagulls -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From skiselkov.ml at gmail.com Sat Oct 1 21:32:32 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sat, 1 Oct 2016 23:32:32 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <20161001231335.03eca0ee@sleipner.datanom.net> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> Message-ID: <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com> On 10/1/16 11:13 PM, Michael Rasmussen wrote: > On Sat, 1 Oct 2016 21:36:23 +0200 > Saso Kiselkov wrote: > >> >> Not being a driver person myself, I have no idea where to look next. >> I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config >> info. Also attached are screenshots of mpstat and dtrace profile at >> 997hz running on the CPU that's bogged down with interrupts. >> > Have you tried to disable csum and hardware offloading? Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help. -- Saso -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From skiselkov.ml at gmail.com Sat Oct 1 21:42:34 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sat, 1 Oct 2016 23:42:34 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <20161001233206.154cb73a@sleipner.datanom.net> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com> <20161001233206.154cb73a@sleipner.datanom.net> Message-ID: On 10/1/16 11:32 PM, Michael Rasmussen wrote: > On Sat, 1 Oct 2016 23:21:40 +0200 > Saso Kiselkov wrote: > >> On 10/1/16 11:13 PM, Michael Rasmussen wrote: >>> hardware offloading >> >> Just found the article that mentions dohwcksum. Set it to 0 and testing >> now. Any other tunables I should be aware of? >> > LRO: https://docs.oracle.com/cd/E53394_01/html/E54790/goryb.html > LRO isn't in Illumos' rge driver, at least the option doesn't appear to be in rge's dladm show-linkprop. -- Saso -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From mir at miras.org Sat Oct 1 21:45:56 2016 From: mir at miras.org (Michael Rasmussen) Date: Sat, 1 Oct 2016 23:45:56 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com> Message-ID: <20161001234556.6f701e1f@sleipner.datanom.net> On Sat, 1 Oct 2016 23:32:32 +0200 Saso Kiselkov wrote: > > Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help. > You don't have a left-over intel nic to plug in? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Man 1: Ask me the what the most important thing about telling a good joke is. Man 2: OK, what is the most impo -- Man 1: ______TIMING! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From skiselkov.ml at gmail.com Sat Oct 1 22:00:43 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 00:00:43 +0200 Subject: [OmniOS-discuss] rge_intr troubles In-Reply-To: <20161001234556.6f701e1f@sleipner.datanom.net> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <20161001231335.03eca0ee@sleipner.datanom.net> <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com> <20161001234556.6f701e1f@sleipner.datanom.net> Message-ID: <8f370a86-71f7-bb3b-21da-047a51b889ba@gmail.com> On 10/1/16 11:45 PM, Michael Rasmussen wrote: > On Sat, 1 Oct 2016 23:32:32 +0200 > Saso Kiselkov wrote: > >> >> Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help. >> > You don't have a left-over intel nic to plug in? Sadly no. I might end up buying one, but first I'd like to try getting this to work. -- Saso -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From skiselkov.ml at gmail.com Sat Oct 1 22:01:26 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 00:01:26 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> Message-ID: On 10/1/16 11:45 PM, Dale Ghent wrote: > >> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov wrote: >> >> So I'm playing around with a box that has an on-board Realtek NIC and >> periodically, about once every 2-5 minutes, the network just goes out to >> lunch and stops responding to ping or attempts to send anything from >> the box. I noticed that while doing so, the box is getting floored by >> interrupts from the NIC, so I see tons of rge_intr activity and one CPU >> core receiving about 160000 interrupts per second (other cores are idle). > > One core getting all the interrupts is expected, as both these chips and the driver do not support RSS. > > The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI. > > Knowing this info would be a good first step in figuring out what's going on. Every time, we're returning through the bottom of the function, returning DDI_INTR_CLAIMED. So the interrupt is meant for us. -- Saso From skiselkov.ml at gmail.com Sat Oct 1 22:05:49 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 00:05:49 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> Message-ID: On 10/1/16 11:45 PM, Dale Ghent wrote: > >> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov wrote: >> >> So I'm playing around with a box that has an on-board Realtek NIC and >> periodically, about once every 2-5 minutes, the network just goes out to >> lunch and stops responding to ping or attempts to send anything from >> the box. I noticed that while doing so, the box is getting floored by >> interrupts from the NIC, so I see tons of rge_intr activity and one CPU >> core receiving about 160000 interrupts per second (other cores are idle). > > One core getting all the interrupts is expected, as both these chips and the driver do not support RSS. > > The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI. > > Knowing this info would be a good first step in figuring out what's going on. Btw: current working on modifying the driver to give us int_status through dtrace, so I know what kind of interrupt we're dealing with. -- Saso From skiselkov.ml at gmail.com Sat Oct 1 22:13:06 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 00:13:06 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> Message-ID: On 10/1/16 11:45 PM, Dale Ghent wrote: > >> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov wrote: >> >> So I'm playing around with a box that has an on-board Realtek NIC and >> periodically, about once every 2-5 minutes, the network just goes out to >> lunch and stops responding to ping or attempts to send anything from >> the box. I noticed that while doing so, the box is getting floored by >> interrupts from the NIC, so I see tons of rge_intr activity and one CPU >> core receiving about 160000 interrupts per second (other cores are idle). > > One core getting all the interrupts is expected, as both these chips and the driver do not support RSS. > > The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI. > > Knowing this info would be a good first step in figuring out what's going on. It's a whole buttload of RT_93c46_COMMOND_REG (0x50) interrupts. -- Saso From skiselkov.ml at gmail.com Sat Oct 1 22:15:53 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 00:15:53 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> Message-ID: On 10/1/16 11:45 PM, Dale Ghent wrote: > >> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov wrote: >> >> So I'm playing around with a box that has an on-board Realtek NIC and >> periodically, about once every 2-5 minutes, the network just goes out to >> lunch and stops responding to ping or attempts to send anything from >> the box. I noticed that while doing so, the box is getting floored by >> interrupts from the NIC, so I see tons of rge_intr activity and one CPU >> core receiving about 160000 interrupts per second (other cores are idle). > > One core getting all the interrupts is expected, as both these chips and the driver do not support RSS. > > The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI. > > Knowing this info would be a good first step in figuring out what's going on. Gah, I'm an idiot, it's actually a bitmask of two things: RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT Apparently, we don't give it enough rx descriptors. Trying to now figure out where to change that... -- Saso From skiselkov.ml at gmail.com Sun Oct 2 12:23:05 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 14:23:05 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> Message-ID: <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Well what I know so far is that after a fixed number of packets (probably after filling up the RX ring), we get a storm of RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the adapter is telling us that it filled up the RX ring and we didn't let it know that we dequeued the old packets. Unfortunately, for the life of me, I can't figure out how we're supposed to let it know that. I've been staring at the drivers (both our and FreeBSD's) for hours and to me it's all just a jumble of "DMA sync this, write reg that". -- Saso On 10/2/16 3:40 AM, Garrett D'Amore wrote: > probably we should do something. like reap the descriptors. i am afk but the usual strategy is to treat these kinds of interrupts just like normal rx. after that you should ack the interrupt of course. > > Sent from my iPhone > >> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov wrote: >> >>> On 10/2/16 12:23 AM, Robert Mustacchi wrote: >>>> On 10/1/16 15:15 , Saso Kiselkov wrote: >>>>> On 10/1/16 11:45 PM, Dale Ghent wrote: >>>>> >>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov wrote: >>>>>> >>>>>> So I'm playing around with a box that has an on-board Realtek NIC and >>>>>> periodically, about once every 2-5 minutes, the network just goes out to >>>>>> lunch and stops responding to ping or attempts to send anything from >>>>>> the box. I noticed that while doing so, the box is getting floored by >>>>>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU >>>>>> core receiving about 160000 interrupts per second (other cores are idle). >>>>> >>>>> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS. >>>>> >>>>> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI. >>>>> >>>>> Knowing this info would be a good first step in figuring out what's going on. >>>> >>>> Gah, I'm an idiot, it's actually a bitmask of two things: >>>> >>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT >>>> >>>> Apparently, we don't give it enough rx descriptors. Trying to now figure >>>> out where to change that... >>> >>> There'll always be cases where we don't have enough rx descriptors for >>> devices. Presumably we shouldn't actually care about receiving that >>> interrupt. Do you happen to have a specification for the device handy? >>> >>> Given that we're not doing anything with the NO_RXDESC_INT, we probably >>> should just mask it on the device if possible. >> >> Just as a general FYI, I'm dealing with 8168G version of the MAC. >> FreeBSD does have a driver that supports it, but since the driver there >> appears home-grown (similar to ours), trying to transplant it would be a >> major undertaking. I'll try to identify the major differences between >> the versions we support and the 8168G, but of course, this being >> hardware, they are many and few of them make any logical sense. >> >> -- >> Saso >> > > > ------------------------------------------- > illumos-networking > Archives: https://www.listbox.com/member/archive/182193/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182193/22721964-fe287663 > Modify Your Subscription: https://www.listbox.com/member/?member_id=22721964&id_secret=22721964-d1c6dd60 > Powered by Listbox: http://www.listbox.com > From garrett at damore.org Sun Oct 2 16:59:06 2016 From: garrett at damore.org (Garrett D'Amore) Date: Sun, 2 Oct 2016 09:59:06 -0700 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: Let me take a look. On Sun, Oct 2, 2016 at 5:23 AM, Saso Kiselkov wrote: > Well what I know so far is that after a fixed number of packets > (probably after filling up the RX ring), we get a storm of > RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the > adapter is telling us that it filled up the RX ring and we didn't let it > know that we dequeued the old packets. Unfortunately, for the life of > me, I can't figure out how we're supposed to let it know that. I've been > staring at the drivers (both our and FreeBSD's) for hours and to me it's > all just a jumble of "DMA sync this, write reg that". > > -- > Saso > > On 10/2/16 3:40 AM, Garrett D'Amore wrote: > > probably we should do something. like reap the descriptors. i am afk > but the usual strategy is to treat these kinds of interrupts just like > normal rx. after that you should ack the interrupt of course. > > > > Sent from my iPhone > > > >> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov > wrote: > >> > >>> On 10/2/16 12:23 AM, Robert Mustacchi wrote: > >>>> On 10/1/16 15:15 , Saso Kiselkov wrote: > >>>>> On 10/1/16 11:45 PM, Dale Ghent wrote: > >>>>> > >>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov > wrote: > >>>>>> > >>>>>> So I'm playing around with a box that has an on-board Realtek NIC > and > >>>>>> periodically, about once every 2-5 minutes, the network just goes > out to > >>>>>> lunch and stops responding to ping or attempts to send anything from > >>>>>> the box. I noticed that while doing so, the box is getting floored > by > >>>>>> interrupts from the NIC, so I see tons of rge_intr activity and one > CPU > >>>>>> core receiving about 160000 interrupts per second (other cores are > idle). > >>>>> > >>>>> One core getting all the interrupts is expected, as both these chips > and the driver do not support RSS. > >>>>> > >>>>> The key thing here is to see what rge_intr() is actually doing. It > has 2 outcomes: It identifies the interrupt type, processes it, then > returns to the DDI that it was claimed. IF it doesn't identify the > interrupt, rge_intr() returns and reports unclaimed to the DDI. > >>>>> > >>>>> Knowing this info would be a good first step in figuring out what's > going on. > >>>> > >>>> Gah, I'm an idiot, it's actually a bitmask of two things: > >>>> > >>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT > >>>> > >>>> Apparently, we don't give it enough rx descriptors. Trying to now > figure > >>>> out where to change that... > >>> > >>> There'll always be cases where we don't have enough rx descriptors for > >>> devices. Presumably we shouldn't actually care about receiving that > >>> interrupt. Do you happen to have a specification for the device handy? > >>> > >>> Given that we're not doing anything with the NO_RXDESC_INT, we probably > >>> should just mask it on the device if possible. > >> > >> Just as a general FYI, I'm dealing with 8168G version of the MAC. > >> FreeBSD does have a driver that supports it, but since the driver there > >> appears home-grown (similar to ours), trying to transplant it would be a > >> major undertaking. I'll try to identify the major differences between > >> the versions we support and the 8168G, but of course, this being > >> hardware, they are many and few of them make any logical sense. > >> > >> -- > >> Saso > >> > > > > > > > ------------------------------------------- > illumos-networking > Archives: https://www.listbox.com/member/archive/182193/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182193/ > 22035935-aad458cf > Modify Your Subscription: https://www.listbox.com/ > member/?member_id=22035935&id_secret=22035935-4809d4da > Powered by Listbox: http://www.listbox.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garrett at damore.org Sun Oct 2 17:20:11 2016 From: garrett at damore.org (Garrett D'Amore) Date: Sun, 2 Oct 2016 10:20:11 -0700 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: Yuck. This driver does the loan up thing ? which is ugly and pointless on ?modern? gigE hardware, bcopy is faster. Especially as this driver is probably only ever used with 1500 byte frames (it cannot do conventional jumbo frames, but can do frames up to 7k? which is just bizarre!) It looks like this was copy-paste developed from nge, which explains some of the sadness here. But as a first test, you can try calling rge_receive(). The simplest way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are suboptimal, as the rx interrupt should be the *hot* code path. One thing you might also try doing is changing the value in rge.h for RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be larger than the largest possible rge frame. I have a theory that part of the problem you are encountering may be due to being out of buffers to loan up, and the screwy handling for that case. If I were doing maintenance on this driver, I would gut all the rx loan up code, and just bcopy. On CPUs produced in the past decade or so, its cheaper to bcopy than to do the loan up. Especially if you rip out all the checks you have to do to make the decision about which to do. (And the driver becomes much leaner and simpler.) On Sun, Oct 2, 2016 at 9:59 AM, Garrett D'Amore wrote: > Let me take a look. > > On Sun, Oct 2, 2016 at 5:23 AM, Saso Kiselkov > wrote: > >> Well what I know so far is that after a fixed number of packets >> (probably after filling up the RX ring), we get a storm of >> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the >> adapter is telling us that it filled up the RX ring and we didn't let it >> know that we dequeued the old packets. Unfortunately, for the life of >> me, I can't figure out how we're supposed to let it know that. I've been >> staring at the drivers (both our and FreeBSD's) for hours and to me it's >> all just a jumble of "DMA sync this, write reg that". >> >> -- >> Saso >> >> On 10/2/16 3:40 AM, Garrett D'Amore wrote: >> > probably we should do something. like reap the descriptors. i am afk >> but the usual strategy is to treat these kinds of interrupts just like >> normal rx. after that you should ack the interrupt of course. >> > >> > Sent from my iPhone >> > >> >> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov >> wrote: >> >> >> >>> On 10/2/16 12:23 AM, Robert Mustacchi wrote: >> >>>> On 10/1/16 15:15 , Saso Kiselkov wrote: >> >>>>> On 10/1/16 11:45 PM, Dale Ghent wrote: >> >>>>> >> >>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov >> wrote: >> >>>>>> >> >>>>>> So I'm playing around with a box that has an on-board Realtek NIC >> and >> >>>>>> periodically, about once every 2-5 minutes, the network just goes >> out to >> >>>>>> lunch and stops responding to ping or attempts to send anything >> from >> >>>>>> the box. I noticed that while doing so, the box is getting floored >> by >> >>>>>> interrupts from the NIC, so I see tons of rge_intr activity and >> one CPU >> >>>>>> core receiving about 160000 interrupts per second (other cores are >> idle). >> >>>>> >> >>>>> One core getting all the interrupts is expected, as both these >> chips and the driver do not support RSS. >> >>>>> >> >>>>> The key thing here is to see what rge_intr() is actually doing. It >> has 2 outcomes: It identifies the interrupt type, processes it, then >> returns to the DDI that it was claimed. IF it doesn't identify the >> interrupt, rge_intr() returns and reports unclaimed to the DDI. >> >>>>> >> >>>>> Knowing this info would be a good first step in figuring out what's >> going on. >> >>>> >> >>>> Gah, I'm an idiot, it's actually a bitmask of two things: >> >>>> >> >>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT >> >>>> >> >>>> Apparently, we don't give it enough rx descriptors. Trying to now >> figure >> >>>> out where to change that... >> >>> >> >>> There'll always be cases where we don't have enough rx descriptors for >> >>> devices. Presumably we shouldn't actually care about receiving that >> >>> interrupt. Do you happen to have a specification for the device handy? >> >>> >> >>> Given that we're not doing anything with the NO_RXDESC_INT, we >> probably >> >>> should just mask it on the device if possible. >> >> >> >> Just as a general FYI, I'm dealing with 8168G version of the MAC. >> >> FreeBSD does have a driver that supports it, but since the driver there >> >> appears home-grown (similar to ours), trying to transplant it would be >> a >> >> major undertaking. I'll try to identify the major differences between >> >> the versions we support and the 8168G, but of course, this being >> >> hardware, they are many and few of them make any logical sense. >> >> >> >> -- >> >> Saso >> >> >> > >> > >> >> >> ------------------------------------------- >> illumos-networking >> Archives: https://www.listbox.com/member/archive/182193/=now >> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935- >> aad458cf >> Modify Your Subscription: https://www.listbox.com/member >> /?member_id=22035935&id_secret=22035935-4809d4da >> Powered by Listbox: http://www.listbox.com >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matej at zunaj.si Sun Oct 2 17:33:55 2016 From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=) Date: Sun, 2 Oct 2016 19:33:55 +0200 Subject: [OmniOS-discuss] zpool Write Bottlenecks In-Reply-To: References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org> <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org> Message-ID: <34C206B4-A5FC-4132-8B4D-EFF7A541C2AC@zunaj.si> You could also try benchmarking with 'fio' tool, where you have some more control over the benchmark. I?m also having some troubles with iSCSI performance, where I can get up to 3GB/s with local tests, but can?t go over 500MB/s via network (10gbit, latency is minimal, since both servers are on the same switch). Matej > On 30 Sep 2016, at 18:46, Michael Talbott wrote: > > Ah. That explains it. At first I figured the single stream couldn't be the case since a dd from /dev/zero to /dev/null was in excess of 24GB/s. But what you said about TX groups explains it all. I ran several dd's in parallel and now see all of the disks getting fully saturated. > > Thanks > > >> On Sep 30, 2016, at 6:36 AM, Bob Friesenhahn wrote: >> >> On Thu, 29 Sep 2016, Michael Talbott wrote: >> >>> I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other >> >> I think that Dale is correct that your benchmark may be only benchmarking 'dd'. The performance of single-threaded 'dd' will be entirely driven by read performance of /dev/zero followed by write latency per operation on writes. In your case, the issue is almost certainly that the write latency is a fixed value (regardless of the number of disks) and so there is a fixed maximum write rate regardless of the performance of the underlying store. >> >> If you were to run two 'dd's in parallel, you may see an increase in the write rate. This is the first thing to try. >> >> With zfs, a "write" goes into a buffer associated with a transaction group (TXG) which is written to disk based on a maximum elapsed time, or the maximum size of the TXG being hit. The maximum size of the TXG is estimated based on the available total RAM and zfs's guess as to the time to write the TXG (based on throughput), as well as tunables. >> >> For a brief time, the writer proces will not be able to write any data at all as the current TXG is finished (resulting in writes) and a new one is started. >> >> At the lowest level, the TXG can not be completed until all involved disks have written their data and completed a cache sync operation. The latency of the cache sync is driven by the latency of the slowest disk drive (+HBA) involved. Getting rid of any disks which exhibit abnormally high latency would help with the transaction times. >> >> Bob >> -- >> Bob Friesenhahn >> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ >> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From skiselkov.ml at gmail.com Sun Oct 2 19:45:15 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 21:45:15 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: Thanks for the suggestions! Results below: On 10/2/16 7:20 PM, Garrett D'Amore wrote: > But as a first test, you can try calling rge_receive(). The simplest > way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in > the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are > suboptimal, as the rx interrupt should be the *hot* code path. Sadly, this won't help. RGE_RX_INT is already a composite of RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT > One thing you might also try doing is changing the value in rge.h for > RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be > larger than the largest possible rge frame. I have a theory that part > of the problem you are encountering may be due to being out of buffers > to loan up, and the screwy handling for that case. Tried this, but to no avail. It still goes into the sad place. Sadly, this stupid box doesn't even have a serial port with which I could provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, so the on-board Realtek POS is my last option. Well that, or using Linux with Illumos as a KVM on top of it, but even saying that out loud leaves a bad taste in my mouth... Cheers, -- Saso From garrett at damore.org Sun Oct 2 20:09:49 2016 From: garrett at damore.org (Garrett D'Amore) Date: Sun, 2 Oct 2016 13:09:49 -0700 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun. Sent from my iPhone > On Oct 2, 2016, at 12:45 PM, Saso Kiselkov wrote: > > Thanks for the suggestions! Results below: > >> On 10/2/16 7:20 PM, Garrett D'Amore wrote: >> But as a first test, you can try calling rge_receive(). The simplest >> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in >> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are >> suboptimal, as the rx interrupt should be the *hot* code path. > > Sadly, this won't help. RGE_RX_INT is already a composite of > RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT > >> One thing you might also try doing is changing the value in rge.h for >> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be >> larger than the largest possible rge frame. I have a theory that part >> of the problem you are encountering may be due to being out of buffers >> to loan up, and the screwy handling for that case. > > Tried this, but to no avail. It still goes into the sad place. Sadly, > this stupid box doesn't even have a serial port with which I could > provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, > so the on-board Realtek POS is my last option. Well that, or using Linux > with Illumos as a KVM on top of it, but even saying that out loud leaves > a bad taste in my mouth... > > Cheers, > -- > Saso > > > ------------------------------------------- > illumos-networking > Archives: https://www.listbox.com/member/archive/182193/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf > Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da > Powered by Listbox: http://www.listbox.com From skiselkov.ml at gmail.com Sun Oct 2 20:32:53 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 22:32:53 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com> I'm willing to test anything you suggest. -- Saso On 10/2/16 10:09 PM, Garrett D'Amore wrote: > ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun. > > Sent from my iPhone > >> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov wrote: >> >> Thanks for the suggestions! Results below: >> >>> On 10/2/16 7:20 PM, Garrett D'Amore wrote: >>> But as a first test, you can try calling rge_receive(). The simplest >>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in >>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are >>> suboptimal, as the rx interrupt should be the *hot* code path. >> >> Sadly, this won't help. RGE_RX_INT is already a composite of >> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT >> >>> One thing you might also try doing is changing the value in rge.h for >>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be >>> larger than the largest possible rge frame. I have a theory that part >>> of the problem you are encountering may be due to being out of buffers >>> to loan up, and the screwy handling for that case. >> >> Tried this, but to no avail. It still goes into the sad place. Sadly, >> this stupid box doesn't even have a serial port with which I could >> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, >> so the on-board Realtek POS is my last option. Well that, or using Linux >> with Illumos as a KVM on top of it, but even saying that out loud leaves >> a bad taste in my mouth... >> >> Cheers, >> -- >> Saso >> > > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182180/22722377-e9306e56 > Modify Your Subscription: https://www.listbox.com/member/?member_id=22722377&id_secret=22722377-08ac87bf > Powered by Listbox: http://www.listbox.com > From garrett at damore.org Sun Oct 2 20:37:39 2016 From: garrett at damore.org (Garrett D'Amore) Date: Sun, 2 Oct 2016 13:37:39 -0700 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com> References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com> Message-ID: so do the rx logic first even if the fifo overflow is set. im not in front of the code but refer to the if-else block I referenced earlier. Sent from my iPhone > On Oct 2, 2016, at 1:32 PM, Saso Kiselkov wrote: > > I'm willing to test anything you suggest. > > -- > Saso > >> On 10/2/16 10:09 PM, Garrett D'Amore wrote: >> ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun. >> >> Sent from my iPhone >> >>> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov wrote: >>> >>> Thanks for the suggestions! Results below: >>> >>>> On 10/2/16 7:20 PM, Garrett D'Amore wrote: >>>> But as a first test, you can try calling rge_receive(). The simplest >>>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in >>>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are >>>> suboptimal, as the rx interrupt should be the *hot* code path. >>> >>> Sadly, this won't help. RGE_RX_INT is already a composite of >>> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT >>> >>>> One thing you might also try doing is changing the value in rge.h for >>>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be >>>> larger than the largest possible rge frame. I have a theory that part >>>> of the problem you are encountering may be due to being out of buffers >>>> to loan up, and the screwy handling for that case. >>> >>> Tried this, but to no avail. It still goes into the sad place. Sadly, >>> this stupid box doesn't even have a serial port with which I could >>> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, >>> so the on-board Realtek POS is my last option. Well that, or using Linux >>> with Illumos as a KVM on top of it, but even saying that out loud leaves >>> a bad taste in my mouth... >>> >>> Cheers, >>> -- >>> Saso > > > ------------------------------------------- > illumos-networking > Archives: https://www.listbox.com/member/archive/182193/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf > Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da > Powered by Listbox: http://www.listbox.com From skiselkov.ml at gmail.com Sun Oct 2 20:51:09 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Sun, 2 Oct 2016 22:51:09 +0200 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com> Message-ID: But we already do an rge_receive on receiving NO_RXDESC_INT. The code sequence is: 1) get interrupt status word (line 1479) 2) if interrupt is RX_FIFO_OVERFLOW_INT, modify int_mask variable to exclude further RX_FIFO_OVERFLOW_INT (line 1488) 3) otherwise if interrupt contains NO_RXDESC_INT (among other things), modify int_mask to include RX_FIFO_OVERFLOW_INT (line 1495) 4) if interrupt contains NO_RXDESC_INT (among other things), do rge_receive (line 1510) ... 5) update interrupt mask according to steps 2 & 3. So after receiving the interrupt, the first thing we pretty much do is rge_receive anyway. I'm sorry I can't be of more help here, but this is all completely foreign to me. I suspect the problem has more to do with the fact that we never really properly dequeue packets from the RX ring, so once the NIC runs out of descriptors, it just keeps on notifying us that it needs more RX descriptors. The reason I think is that this problem appears to be triggered strictly by the number of packets received. -- Saso On 10/2/16 10:37 PM, Garrett D'Amore wrote: > so do the rx logic first even if the fifo overflow is set. im not in front of the code but refer to the if-else block I referenced earlier. > > Sent from my iPhone > >> On Oct 2, 2016, at 1:32 PM, Saso Kiselkov wrote: >> >> I'm willing to test anything you suggest. >> >> -- >> Saso >> >>> On 10/2/16 10:09 PM, Garrett D'Amore wrote: >>> ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun. >>> >>> Sent from my iPhone >>> >>>> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov wrote: >>>> >>>> Thanks for the suggestions! Results below: >>>> >>>>> On 10/2/16 7:20 PM, Garrett D'Amore wrote: >>>>> But as a first test, you can try calling rge_receive(). The simplest >>>>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in >>>>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are >>>>> suboptimal, as the rx interrupt should be the *hot* code path. >>>> >>>> Sadly, this won't help. RGE_RX_INT is already a composite of >>>> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT >>>> >>>>> One thing you might also try doing is changing the value in rge.h for >>>>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be >>>>> larger than the largest possible rge frame. I have a theory that part >>>>> of the problem you are encountering may be due to being out of buffers >>>>> to loan up, and the screwy handling for that case. >>>> >>>> Tried this, but to no avail. It still goes into the sad place. Sadly, >>>> this stupid box doesn't even have a serial port with which I could >>>> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, >>>> so the on-board Realtek POS is my last option. Well that, or using Linux >>>> with Illumos as a KVM on top of it, but even saying that out loud leaves >>>> a bad taste in my mouth... >>>> >>>> Cheers, >>>> -- >>>> Saso >> >> > > > ------------------------------------------- > illumos-discuss > Archives: https://www.listbox.com/member/archive/182180/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182180/22722377-e9306e56 > Modify Your Subscription: https://www.listbox.com/member/?member_id=22722377&id_secret=22722377-08ac87bf > Powered by Listbox: http://www.listbox.com > From garrett at damore.org Mon Oct 3 01:55:37 2016 From: garrett at damore.org (Garrett D'Amore) Date: Sun, 2 Oct 2016 18:55:37 -0700 Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles In-Reply-To: References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com> <18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com> <8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com> <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com> Message-ID: <4555194F-E01D-4ACC-B586-F77403AE416D@damore.org> i will try to have another look at this tomorrow. i am too tired tonight. :) Sent from my iPhone > On Oct 2, 2016, at 12:45 PM, Saso Kiselkov wrote: > > Thanks for the suggestions! Results below: > >> On 10/2/16 7:20 PM, Garrett D'Amore wrote: >> But as a first test, you can try calling rge_receive(). The simplest >> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in >> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are >> suboptimal, as the rx interrupt should be the *hot* code path. > > Sadly, this won't help. RGE_RX_INT is already a composite of > RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT > >> One thing you might also try doing is changing the value in rge.h for >> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be >> larger than the largest possible rge frame. I have a theory that part >> of the problem you are encountering may be due to being out of buffers >> to loan up, and the screwy handling for that case. > > Tried this, but to no avail. It still goes into the sad place. Sadly, > this stupid box doesn't even have a serial port with which I could > provide you access. And I'm out of PCI-e slots to shove in an Intel NIC, > so the on-board Realtek POS is my last option. Well that, or using Linux > with Illumos as a KVM on top of it, but even saying that out loud leaves > a bad taste in my mouth... > > Cheers, > -- > Saso > > > ------------------------------------------- > illumos-networking > Archives: https://www.listbox.com/member/archive/182193/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf > Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da > Powered by Listbox: http://www.listbox.com From mir at miras.org Mon Oct 3 20:39:40 2016 From: mir at miras.org (Michael Rasmussen) Date: Mon, 3 Oct 2016 22:39:40 +0200 Subject: [OmniOS-discuss] Supermicro X10SLL-F Message-ID: <20161003223940.7e954c22@sleipner.datanom.net> Hi all, I recently swapped motherboard in an Omnios server to Supermicro X10SLL-F and system came up again without any problems so this is all fine and dandy. But one thing that literally took hours to solve was getting the board to even post. So here is the story and solution to anybody hitting the same problem as I. Symptom: Board gets power since power led lights and IPMI heartbeat led blinks, but no post what so ever. Access to IPMI through web browser works but system is reported to be off and cannot be started. Post snooping shows f0. Solution: 1) Power off the system and pop the CMOS battery. Wait a couple of minuts before going to step 2. 2) Power on system without CMOS battery (Single user mode preferable) 3) When system is powered up fully consult IPMI to ensure everything is OK and the power system off again 4) Reinsert CMOS battery and power system on again. Every thing should work again. Outstanding unresolved issues: Case fans are Arctic F8 PWM (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186072) and Arctic F12 PWM (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186033) Problem: Even though fan speed is controlled by Supermicro fan mode no speed readings are available (display N/A). Maybe this is because the Arctic fans are all revision 1 but to sure to have working speed readings I have ordered new fans from Noctua. A couple of Noctua NF-A8 PWM (http://www.newegg.com/Product/Product.aspx?Item=9SIA9PV3Y64158) A couple of Noctua NF-S12A (http://www.newegg.com/Product/Product.aspx?Item=N82E16835608038) Anybody having experience with these fans on supermicro boards? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Life is a yo-yo, and mankind ties knots in the string. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From groups at tierarzt-mueller.de Tue Oct 4 09:31:05 2016 From: groups at tierarzt-mueller.de (Alexander Lesle) Date: Tue, 4 Oct 2016 11:31:05 +0200 Subject: [OmniOS-discuss] Supermicro X10SLL-F In-Reply-To: <20161003223940.7e954c22@sleipner.datanom.net> References: <20161003223940.7e954c22@sleipner.datanom.net> Message-ID: <294740579.20161004113105@tierarzt-mueller.de> Hello Michael Rasmussen and List, maybe the fans from supermicro are the better way. https://www.supermicro.com/support/resources/Thermal/#FAN On Oktober, 03 2016, 22:39 wrote in [1]: > Outstanding unresolved issues: > Case fans are Arctic F8 PWM > (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186072) > and Arctic F12 PWM > (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186033) > Problem: Even though fan speed is controlled by Supermicro fan mode no > speed readings are available (display N/A). Maybe this is because the > Arctic fans are all revision 1 but to sure to have working speed > readings I have ordered new fans from Noctua. > A couple of Noctua NF-A8 PWM > (http://www.newegg.com/Product/Product.aspx?Item=9SIA9PV3Y64158) > A couple of Noctua NF-S12A > (http://www.newegg.com/Product/Product.aspx?Item=N82E16835608038) > Anybody having experience with these fans on supermicro boards? -- Best Regards Alexander Oktober, 04 2016 ........ [1] mid:20161003223940.7e954c22 at sleipner.datanom.net ........ From mir at miras.org Tue Oct 4 09:59:12 2016 From: mir at miras.org (mir at miras.org) Date: Tue, 04 Oct 2016 11:59:12 +0200 Subject: [OmniOS-discuss] Supermicro X10SLL-F In-Reply-To: <294740579.20161004113105@tierarzt-mueller.de> References: <20161003223940.7e954c22@sleipner.datanom.net> <294740579.20161004113105@tierarzt-mueller.de> Message-ID: On 2016-10-04 11:31, Alexander Lesle wrote: > Hello Michael Rasmussen and List, > > maybe the fans from supermicro are the better way. > https://www.supermicro.com/support/resources/Thermal/#FAN > I have solved the issue with Arctic fans. It was as mentioned the yellow wire which is separated on arctic fans which I had forgotten to attached the extender connector. Everything works now except for the 12 cm fans which seems to spin at a lower RPM than Supermicro accepts for FANA header (means alarm condition activated and all fans running at full speed) so cross connecting with a 8 cm fan have solved the issue. I will however install my new Noctua fans when I get them since I have already made the purchase and because the current fans is near end of life anyway. ---- This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. From timo.raatikainen at sourapple.fi Tue Oct 4 12:20:36 2016 From: timo.raatikainen at sourapple.fi (Timo Raatikainen) Date: Tue, 4 Oct 2016 15:20:36 +0300 Subject: [OmniOS-discuss] Supermicro X10SLL-F In-Reply-To: References: <20161003223940.7e954c22@sleipner.datanom.net> <294740579.20161004113105@tierarzt-mueller.de> Message-ID: Hi, You can change the fan thresholds on a Supermicro board using ipmitool. I also use Noctua fans and have successfully lowered the thresholds. See https://calvin.me/quick-how-to-decrease-ipmi-fan-threshold/ for more specific instructions. Ipmitool should be available in OmniOS repo, but I havent checked lately. You can also use ipmitool from another computer via network. -- Timo Raatikainen > On 4 Oct 2016, at 12.59, mir at miras.org wrote: > > On 2016-10-04 11:31, Alexander Lesle wrote: >> Hello Michael Rasmussen and List, >> maybe the fans from supermicro are the better way. >> https://www.supermicro.com/support/resources/Thermal/#FAN > I have solved the issue with Arctic fans. It was as mentioned the yellow wire which is separated on arctic fans which I had forgotten to attached the extender connector. > > Everything works now except for the 12 cm fans which seems to spin at a lower RPM than Supermicro accepts for FANA header (means alarm condition activated and all fans running at full speed) so cross connecting with a 8 cm fan have solved the issue. > > I will however install my new Noctua fans when I get them since I have already made the purchase and because the current fans is near end of life anyway. > > ---- > > This mail was virus scanned and spam checked before delivery. > This mail is also DKIM signed. See header dkim-signature. > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From mir at miras.org Tue Oct 4 15:53:41 2016 From: mir at miras.org (Michael Rasmussen) Date: Tue, 4 Oct 2016 17:53:41 +0200 Subject: [OmniOS-discuss] Supermicro X10SLL-F In-Reply-To: References: <20161003223940.7e954c22@sleipner.datanom.net> <294740579.20161004113105@tierarzt-mueller.de> Message-ID: <20161004175341.06132902@sleipner.datanom.net> On Tue, 4 Oct 2016 15:20:36 +0300 Timo Raatikainen wrote: > Hi, > > You can change the fan thresholds on a Supermicro board using ipmitool. I also use Noctua fans and have successfully lowered the thresholds. > > See https://calvin.me/quick-how-to-decrease-ipmi-fan-threshold/ for more specific instructions. Ipmitool should be available in OmniOS repo, but I havent checked lately. You can also use ipmitool from another computer via network. > I have tried the solution from the article even did sensor thresh FANA lower 100 100 100 without success. The problem is that the arctic f12 fans RPM is (for 12V) between 450 (0% PWM) and 1350 (100% PWM) which apparently confuses Supermicro IPMI since it interprets this to a RPM span -? to ? resulting in fan read error And ipmitool is available in Omnios -> system/management/ipmitool -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: Parenthesise to avoid ambiguity. - The Elements of Programming Style (Kernighan & Plaugher) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From skeltonr at btconnect.com Tue Oct 4 18:33:26 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Tue, 04 Oct 2016 19:33:26 +0100 Subject: [OmniOS-discuss] omniti/database/mysql-55 keeps on stopping Message-ID: <57F3F5F6.3080703@btconnect.com> Hi, I have installed the pkg://ms.omniti.com/omniti/database/mysql-55 at 5.5.50-0.151014:20160722T162252Z in a zone on my OmniOS v11 r151018 system Which has stopped and I don't see any core file. Looking in the .err log file I can see my restart but no message why it crashed. 160913 22:19:58 InnoDB: Waiting for the background threads to start 160913 22:19:59 InnoDB: 5.5.50 started; log sequence number 0 160913 22:19:59 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 160913 22:19:59 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 160913 22:19:59 [Note] Server socket created on IP: '0.0.0.0'. 160913 22:19:59 [Note] Event Scheduler: Loaded 0 events 160913 22:19:59 [Note] /opt/mysql55/bin/mysqld: ready for connections. Version: '5.5.50' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) 161003 20:48:44 mysqld_safe Starting mysqld daemon with databases from /opt/mysql55/data 161003 20:48:44 [Note] /opt/mysql55/bin/mysqld (mysqld 5.5.50) starting as process 4098 ... 161003 20:48:44 [Note] Plugin 'FEDERATED' is disabled. 161003 20:48:44 InnoDB: The InnoDB memory heap is disabled 161003 20:48:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins 161003 20:48:44 InnoDB: Compressed tables use zlib 1.2.3 161003 20:48:44 InnoDB: Initializing buffer pool, size = 128.0M 161003 20:48:44 InnoDB: Completed initialization of buffer pool 161003 20:48:44 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 161003 20:48:44 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... 161003 20:48:44 InnoDB: Waiting for the background threads to start 161003 20:48:45 InnoDB: 5.5.50 started; log sequence number 1595675 161003 20:48:45 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 161003 20:48:45 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 161003 20:48:45 [Note] Server socket created on IP: '0.0.0.0'. 161003 20:48:45 [Note] Event Scheduler: Loaded 0 events 161003 20:48:45 [Note] /opt/mysql55/bin/mysqld: ready for connections. Can I set more detailed logging ? Cheers Richard. From ghestin at hotmail.com Wed Oct 5 13:50:59 2016 From: ghestin at hotmail.com (ludo zor) Date: Wed, 5 Oct 2016 13:50:59 +0000 Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support Message-ID: Hello I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts . When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" . Can you help me ? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Oct 5 14:35:33 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 5 Oct 2016 10:35:33 -0400 Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support In-Reply-To: References: Message-ID: > On Oct 5, 2016, at 9:50 AM, ludo zor wrote: > > Hello > I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts . > When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" . > Can you help me ? > PLEASE do not use Oracle Solaris drivers on OmniOS. There are likely to be subtle incompatibilities. If you cannot see your controller under the OmniOS LTS media, please see if the more recent OmniOS Stable or OmniOS Bloody release see it. If they do, a backport of the driver may be possible (esp. to a paying customer). Sorry, Dan From eric.sproul at circonus.com Wed Oct 5 14:46:23 2016 From: eric.sproul at circonus.com (Eric Sproul) Date: Wed, 5 Oct 2016 10:46:23 -0400 Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support In-Reply-To: References: Message-ID: On Wed, Oct 5, 2016 at 10:35 AM, Dan McDonald wrote: > >> On Oct 5, 2016, at 9:50 AM, ludo zor wrote: >> >> Hello >> I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts . >> When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" . >> Can you help me ? >> > > PLEASE do not use Oracle Solaris drivers on OmniOS. There are likely to be subtle incompatibilities. I don't think these are drivers-- they sound like admin utilities, though I'm not too familiar with HP hardware. I would first check that the illumos cpqary3 driver is attaching to the hardware: # grep cpqary3 /etc/path_to_inst If this shows nothing, then for some reason the driver did not attach. The kernel log might have some messages related to that. Can you install pkg:/system/pciutils and post the output of `/usr/sbin/lspci -nn` for this machine? The PCI ID for the HBA would need to be one of the IDs that cpqary3 knows about, which are listed in /etc/driver_aliases. Eric From daleg at omniti.com Wed Oct 5 15:10:54 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 5 Oct 2016 11:10:54 -0400 Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support In-Reply-To: References: Message-ID: > On Oct 5, 2016, at 9:50 AM, ludo zor wrote: > > Hello > I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts . > When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" . > Can you help me ? You are using binaries and utilities made for Solaris 10 on OmniOS. There is no guarantee of compatibility, as these are two very different OSes that have grown apart over the years. It is possible that, while your binaries do execute on OmniOS, your hpacucli is looking for and trying to use a facility that has since changed or disappeared. /dale From geoffn at gnaa.net Wed Oct 5 19:57:22 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Wed, 5 Oct 2016 12:57:22 -0700 Subject: [OmniOS-discuss] upgrade from r151008 Message-ID: Hi. Do you think I will be able to upgrade from r151008 to r151014 LTS? What about going to current stable of r151018? thanks, Geoff / / / / -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Oct 5 20:11:48 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 5 Oct 2016 16:11:48 -0400 Subject: [OmniOS-discuss] upgrade from r151008 In-Reply-To: References: Message-ID: Wow! We stopped supporting 008 a long time ago. Follow the upgrade directions on the wiki for 014 (involves a publisher change AND a package policy change) and you should be in good shape. Should be little difference between 018 and 014 with respect to a jump from 018. For both, make sure you've changed the OmniOS publisher policy to require signatures. It should all be on the upgrade pages. Dan Sent from my iPhone (typos, autocorrect, and all) > On Oct 5, 2016, at 3:57 PM, Geoff Nordli wrote: > > Hi. > > Do you think I will be able to upgrade from r151008 to r151014 LTS? > > What about going to current stable of r151018? > > thanks, > Geoff > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoffn at gnaa.net Wed Oct 5 23:03:49 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Wed, 5 Oct 2016 16:03:49 -0700 Subject: [OmniOS-discuss] upgrade from r151008 In-Reply-To: References: Message-ID: It has been a good box. But I should have upgraded it sooner than I did. Always a risk when you wait too long between versions. thanks!! Geoff On 2016-10-05 01:11 PM, Dan McDonald wrote: > Wow! We stopped supporting 008 a long time ago. > > Follow the upgrade directions on the wiki for 014 (involves a > publisher change AND a package policy change) and you should be in > good shape. > > Should be little difference between 018 and 014 with respect to a jump > from 018. For both, make sure you've changed the OmniOS publisher > policy to require signatures. It should all be on the upgrade pages. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Oct 5, 2016, at 3:57 PM, Geoff Nordli > wrote: > >> Hi. >> >> Do you think I will be able to upgrade from r151008 to r151014 LTS? >> >> What about going to current stable of r151018? >> >> thanks, >> Geoff / >> / >> >> / >> / >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Oct 5 23:32:07 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 5 Oct 2016 19:32:07 -0400 Subject: [OmniOS-discuss] upgrade from r151008 In-Reply-To: References: Message-ID: > On Oct 5, 2016, at 7:03 PM, Geoff Nordli wrote: > > Always a risk when you wait too long between versions. > > We tested 006 --> 014 pre-release of 014. 008 --> 014 should be equally workable. Dan From johan.kragsterman at capvert.se Thu Oct 6 09:00:58 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 6 Oct 2016 11:00:58 +0200 Subject: [OmniOS-discuss] stmf trouble, crash and dump Message-ID: Hi! Got a problem here a couple of days ago when I ran a snapshot stream over fibre channel on my home/business/devel server to the clone backup server. Systems: OmniOS 5.11 ? ? omnios-r151018-95eaa7e on both systems, initiator on one, and and target on the other. Also same hardware: Dell precision workstation with dual xeon 6-cores and 96 GB registred ram, intel quad port gb nic, and qlogic QLE2462 HBA's. Configured with one lun provisioned from the target/backup system to the initiator system as a backup lun, and that backup lun configured as a zpool in the initiator system. I should also say, that I run this Fc connection point-to-point, no switch is involved, and it's a single fibre pair, 10 m. I did a zfs send/recv of a snapshot, and I thought it took a long time. It was around 67 GB. Then the initiator system crashed and dumped. It rebooted, and I got into it again without any trouble. What I immediately saw was that the zpool "backpool" that was backed by the Fc lun was not present any longer. I made a zpool import, and it was back there again. I did another test, sent a much smaller snap, this was around 450 MB, and that worked fine, although I thought it took a lot of time. I once again tried with the bigger snap, and same thing happened, system crashed and dumped. I got those two dump files, but I don't know wether this might be a problem on the target system or the initiator side. I can provide access to dump files. Here is some information from the two systems that I find interesting: The initiator system, omni: omni: root at omni:/var/log# dmesg | grep qlc Oct 2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE root at omni:/# dmesg | grep scsi Oct 2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0): Oct 2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline Oct 2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline root at omni:/# dmesg | grep multipath Oct 2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline As you can see, the loop is marked offline at the occasion for the crash. But notably strange, there is also an info of a failed multipath...? Why this? There is no multipath here... The target system, omni2: root at omni2:/root# grep stmf /var/adm/messages Oct 2 09:56:37 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf_sbd0 Oct 2 09:56:37 omni2 genunix: [ID 936769 kern.info] stmf_sbd0 is /pseudo/stmf_sbd at 0 Oct 2 09:56:46 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0 Oct 2 09:56:46 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0 Oct 2 09:57:31 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0 Oct 2 09:57:31 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0 Oct 2 09:57:31 omni2 stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25 There is this warning, ioctl(DKCINFO) failed 25, that I tried to find out what it is about, but not succeeded. Perhaps it is just so simple that the Fc connection isn't good enough. The cable shouldn't be a problem, since it is brand new, but it could of coarse be something with the HBA's. I could get another cable for doing multipath, and see how that would work, but let's start with this first. Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert From danmcd at omniti.com Thu Oct 6 14:35:45 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 6 Oct 2016 10:35:45 -0400 Subject: [OmniOS-discuss] stmf trouble, crash and dump In-Reply-To: References: Message-ID: <531FE7BF-1D78-4AB2-BA7B-9C214A21E5F3@omniti.com> > On Oct 6, 2016, at 5:00 AM, Johan Kragsterman wrote: > > I can provide access to dump files. Please share those, or at least show the "$c" output from mdb when you post a "kernel panic" email. Thank you, Dan From johan.kragsterman at capvert.se Thu Oct 6 16:00:39 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 6 Oct 2016 18:00:39 +0200 Subject: [OmniOS-discuss] Ang: stmf trouble, crash and dump In-Reply-To: References: Message-ID: Hi! Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why... -----"OmniOS-discuss" skrev: ----- Till: omnios-discuss at lists.omniti.com Fr?n: Johan Kragsterman S?nt av: "OmniOS-discuss" Datum: 2016-10-06 11:03 ?rende: [OmniOS-discuss] stmf trouble, crash and dump Hi! Got a problem here a couple of days ago when I ran a snapshot stream over fibre channel on my home/business/devel server to the clone backup server. Systems: OmniOS 5.11 ? ? omnios-r151018-95eaa7e on both systems, initiator on one, and and target on the other. Also same hardware: Dell precision workstation with dual xeon 6-cores and 96 GB registred ram, intel quad port gb nic, and qlogic QLE2462 HBA's. Configured with one lun provisioned from the target/backup system to the initiator system as a backup lun, and that backup lun configured as a zpool in the initiator system. I should also say, that I run this Fc connection point-to-point, no switch is involved, and it's a single fibre pair, 10 m. I did a zfs send/recv of a snapshot, and I thought it took a long time. It was around 67 GB. Then the initiator system crashed and dumped. It rebooted, and I got into it again without any trouble. What I immediately saw was that the zpool "backpool" that was backed by the Fc lun was not present any longer. I made a zpool import, and it was back there again. I did another test, sent a much smaller snap, this was around 450 MB, and that worked fine, although I thought it took a lot of time. I once again tried with the bigger snap, and same thing happened, system crashed and dumped. I got those two dump files, but I don't know wether this might be a problem on the target system or the initiator side. I can provide access to dump files. Here is some information from the two systems that I find interesting: The initiator system, omni: omni: root at omni:/var/log# dmesg | grep qlc Oct ?2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE root at omni:/# dmesg | grep scsi Oct ?2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0): Oct ?2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline Oct ?2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline root at omni:/# dmesg | grep multipath Oct ?2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline As you can see, the loop is marked offline at the occasion for the crash. But notably strange, there is also an info of a failed multipath...? Why this? There is no multipath here... The target system, omni2: root at omni2:/root# grep stmf /var/adm/messages Oct ?2 09:56:37 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf_sbd0 Oct ?2 09:56:37 omni2 genunix: [ID 936769 kern.info] stmf_sbd0 is /pseudo/stmf_sbd at 0 Oct ?2 09:56:46 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0 Oct ?2 09:56:46 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0 Oct ?2 09:57:31 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0 Oct ?2 09:57:31 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0 Oct ?2 09:57:31 omni2 stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25 There is this warning, ioctl(DKCINFO) failed 25, that I tried to find out what it is about, but not succeeded. Perhaps it is just so simple that the Fc connection isn't good enough. The cable shouldn't be a problem, since it is brand new, but it could of coarse be something with the HBA's. I could get another cable for doing multipath, and see how that would work, but let's start with this first. mdb output from the initiator system here. Exactly the same from both crash dumps. So no questions there, the question is why...? In the target system I only found that warning "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25" root at omni:/var/crash/unknown# mdb unix.0 vmcore.0 Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd ip hook neti sockfs arp usba uhci mm stmf stmf_sbd md lofs mpt_sas sata random idm cpc crypto kvm ufs logindmux nsmb ptm smbsrv nfs ] > ::status debugging crash dump vmcore.0 (64-bit) from omni operating system: 5.11 omnios-r151018-95eaa7e (i86pc) image uuid: 0fdcc7e7-aaf9-4d9d-cc75-ac766a3c3b5a panic message: I/O to pool 'backpool' appears to be hung. dump content: kernel pages only > C debugging crash dump vmcore.0 (64-bit) from omni operating system: 5.11 omnios-r151018-95eaa7e (i86pc) image uuid: 0fdcc7e7-aaf9-4d9d-cc75-ac766a3c3b5a panic message: I/O to pool 'backpool' appears to be hung. dump content: kernel pages only > ::panicinfo cpu 20 thread ffffff007ba86c40 message I/O to pool 'backpool' appears to be hung. rdi fffffffff7a72290 rsi ffffff007ba869c0 rdx ffffff007ba86c40 rcx ffffff196891f274 r8 20 r9 a rax ffffff007ba869e0 rbx ffffff25200971c8 rbp ffffff007ba86a20 r10 0 r11 ffffff007ba868c0 r12 ffffff19ae36b000 r13 e984f1e438 r14 ffffff2520096cc0 r15 ffffff753a6df538 fsbase fffffd7fff061a40 gsbase ffffff195ca63a80 ds 38 es 38 fs 0 gs 0 trapno 0 err 0 rip fffffffffb860190 cs 30 rflags 246 rsp ffffff007ba869a8 ss 38 gdt_hi 0 gdt_lo 5000ffff idt_hi 0 idt_lo 4000ffff ldt 0 task 70 cr0 8005003b cr2 7ffcb34d6618 cr3 c000000 cr4 26f8 > root at omni:/var/crash/unknown# mdb unix.1 vmcore.1 Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd ip hook neti sockfs arp usba uhci mm stmf stmf_sbd idm mpt_sas sata cpc crypto md kvm random lofs ufs logindmux nsmb ptm smbsrv nfs ] > ::status debugging crash dump vmcore.1 (64-bit) from omni operating system: 5.11 omnios-r151018-95eaa7e (i86pc) image uuid: 748b6c72-5dec-c92c-a155-f1788f51b3fd panic message: I/O to pool 'backpool' appears to be hung. dump content: kernel pages only > C debugging crash dump vmcore.1 (64-bit) from omni operating system: 5.11 omnios-r151018-95eaa7e (i86pc) image uuid: 748b6c72-5dec-c92c-a155-f1788f51b3fd panic message: I/O to pool 'backpool' appears to be hung. dump content: kernel pages only > ::panicinfo cpu 2 thread ffffff007a4afc40 message I/O to pool 'backpool' appears to be hung. rdi fffffffff7a72290 rsi ffffff007a4af9c0 rdx ffffff007a4afc40 rcx ffffff1964e842ee r8 20 r9 a rax ffffff007a4af9e0 rbx ffffff1966da1188 rbp ffffff007a4afa20 r10 0 r11 ffffff007a4af8c0 r12 ffffff1997848000 r13 e9840bec87 r14 ffffff1966da0c80 r15 ffffff19e954c718 fsbase fffffd7fff072a40 gsbase ffffff195c0e6040 ds 4b es 4b fs 0 gs 0 trapno 0 err 0 rip fffffffffb860190 cs 30 rflags 246 rsp ffffff007a4af9a8 ss 38 gdt_hi 0 gdt_lo d000ffff idt_hi 0 idt_lo c000ffff ldt 0 task 70 cr0 8005003b cr2 7ffd18fc0568 cr3 c400000 cr4 26f8 > Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From eric.sproul at circonus.com Thu Oct 6 16:07:46 2016 From: eric.sproul at circonus.com (Eric Sproul) Date: Thu, 6 Oct 2016 12:07:46 -0400 Subject: [OmniOS-discuss] Ang: stmf trouble, crash and dump In-Reply-To: References: Message-ID: On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman wrote: > > Hi! > > Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why... As it happens, I just wrote a blog post about how to drill down on this type of crash and determine if it's a single disk or multiple that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/ I'm happy to answer questions about that method, if needed. Eric From johan.kragsterman at capvert.se Thu Oct 6 16:14:13 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 6 Oct 2016 18:14:13 +0200 Subject: [OmniOS-discuss] Ang: Re: Ang: stmf trouble, crash and dump In-Reply-To: References: , Message-ID: Hi! Thanks, Eric, I check it out... -----Eric Sproul skrev: ----- Till: Johan Kragsterman Fr?n: Eric Sproul Datum: 2016-10-06 18:07 Kopia: omnios-discuss ?rende: Re: [OmniOS-discuss] Ang: stmf trouble, crash and dump On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman wrote: > > Hi! > > Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why... As it happens, I just wrote a blog post about how to drill down on this type of crash and determine if it's a single disk or multiple that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/ I'm happy to answer questions about that method, if needed. Eric From johan.kragsterman at capvert.se Thu Oct 6 16:36:24 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 6 Oct 2016 18:36:24 +0200 Subject: [OmniOS-discuss] Ang: Ang: Re: Ang: stmf trouble, crash and dump In-Reply-To: References: , , Message-ID: Hi! While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier?? "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25" Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert -----"OmniOS-discuss" skrev: ----- Till: Eric Sproul Fr?n: Johan Kragsterman S?nt av: "OmniOS-discuss" Datum: 2016-10-06 18:15 Kopia: omnios-discuss ?rende: [OmniOS-discuss] Ang: Re: Ang: stmf trouble, crash and dump Hi! Thanks, Eric, I check it out... -----Eric Sproul skrev: ----- Till: Johan Kragsterman Fr?n: Eric Sproul Datum: 2016-10-06 18:07 Kopia: omnios-discuss ?rende: Re: [OmniOS-discuss] Ang: stmf trouble, crash and dump On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman wrote: > > Hi! > > Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why... As it happens, I just wrote a blog post about how to drill down on this type of crash and determine if it's a single disk or multiple that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/ I'm happy to answer questions about that method, if needed. Eric _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From eric.sproul at circonus.com Thu Oct 6 16:46:38 2016 From: eric.sproul at circonus.com (Eric Sproul) Date: Thu, 6 Oct 2016 12:46:38 -0400 Subject: [OmniOS-discuss] Ang: Re: Ang: stmf trouble, crash and dump In-Reply-To: References: Message-ID: On Thu, Oct 6, 2016 at 12:36 PM, Johan Kragsterman wrote: > > Hi! > > > While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier?? > > "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25" DKIOCINFO is the macro for the ioctl that gets info from a disk: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/dkio.h#166 I am not a COMSTAR expert but I believe your warning is coming from here: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1581 It looks like COMSTAR was trying to query one of the zvols, and failed. This is almost assuredly a consequence of the hung pool. Eric From danmcd at omniti.com Thu Oct 6 16:50:20 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 6 Oct 2016 12:50:20 -0400 Subject: [OmniOS-discuss] Ang: Ang: Re: Ang: stmf trouble, crash and dump In-Reply-To: References: Message-ID: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com> > On Oct 6, 2016, at 12:36 PM, Johan Kragsterman wrote: > > > While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier?? > > "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25" > Error 25 is "ENOTTY", aka. "Inappropriate ioctl for device": http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/errno.h#77 The error itself comes because while STMF thinks the underlying storage is a zvol, apparently the DKIOCINFO ioctl failed with the above erro: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1575 As Eric (who beat me to the punch here) said, it was likely caused by a hung pool. Dan From johan.kragsterman at capvert.se Thu Oct 6 17:45:51 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Thu, 6 Oct 2016 19:45:51 +0200 Subject: [OmniOS-discuss] Ang: Re: Ang: Ang: Re: Ang: stmf trouble, crash and dump In-Reply-To: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com> References: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com>, Message-ID: Hi! -----Dan McDonald skrev: ----- Till: Johan Kragsterman , omnios-discuss Fr?n: Dan McDonald Datum: 2016-10-06 18:50 Kopia: Dan McDonald ?rende: Re: [OmniOS-discuss] Ang: Ang: Re: Ang: stmf trouble, crash and dump > On Oct 6, 2016, at 12:36 PM, Johan Kragsterman wrote: > > > While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier?? > > "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25" > Error 25 is "ENOTTY", aka. "Inappropriate ioctl for device": http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/errno.h#77 The error itself comes because while STMF thinks the underlying storage is a zvol, apparently the DKIOCINFO ioctl failed with the above erro: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1575 As Eric (who beat me to the punch here) said, it was likely caused by a hung pool. Dan No, it is the other way around, actually! This error comes from the target system, not the system that crashed, that was the intitiator system. The underlying device IS a zvol on the target system! So the question I ask myself, since I saw this error, is if this could be a/the problem that caused the lu/pool on the initiator system to go off line? Though there were also the loop offline msg on the initiator: root at omni:/var/log# dmesg | grep qlc Oct 2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE And what puzzles me is the multipath messages, because there is no multipath devices on the system: root at omni:/# dmesg | grep scsi Oct 2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0): Oct 2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline Oct 2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline root at omni:/# dmesg | grep multipath Oct 2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline Johan From danmcd at omniti.com Fri Oct 7 01:02:47 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 6 Oct 2016 21:02:47 -0400 Subject: [OmniOS-discuss] Bloody update -- NOW INCLUDES LX Beta SUPPORT Message-ID: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com> It's here! LX Support is now folded into illumos-omnios:master, and therefore is now ready for OmniOS bloody updates... like this one. uname -v says omnios-master-3fd4b56, and that is the commit of illumos-omnios:master which generated this update. omnios-build is at 5f132cf. New with this update: * LX Beta SUPPORT -- If you've been following the LX threads here, you know what's been happening. A new README.OmniOS on the top-level of illumos-omnios covers what was once in the "io-lx" gate. Going forward, this support is considered to be Beta. This will include when r151020 ships. We need to sort out various administrative issues, as well as keep up with Joyent's progress. Feedback will be appreciated. * NSS is now 3.26 * libdbus-glib is now 0.108 * dbus is 1.11.4 * diffutils is now 3.5 * expat is now 2.2.0 * gawk is now 4.1.4 * gettext is now 0.19.8.1 * Git is now 2.10.0 * gnu-make is now 4.2.1 * gnump/gmp is now 6.1.1 The http://pkg.omniti.com/omnios/LX/ repo is now obsolete. For this update, it matches bloody, but going forward it will not receive updates until the next bloody cycle, IF AT ALL. I've also updated release media, as seen here: https://omnios.omniti.com/wiki.php/Installation Happy updating, and happy LX-ing! Dan From bfriesen at simple.dallas.tx.us Fri Oct 7 13:43:46 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Fri, 7 Oct 2016 08:43:46 -0500 (CDT) Subject: [OmniOS-discuss] Bloody update -- NOW INCLUDES LX Beta SUPPORT In-Reply-To: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com> References: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com> Message-ID: On Thu, 6 Oct 2016, Dan McDonald wrote: > > LX Support is now folded into illumos-omnios:master, and therefore is now ready for OmniOS bloody updates... like this one. This is very exciting. Is anyone able to confirm if programs compiled GCC or Clang with Address Sanitizer (ASAN, https://code.google.com/p/address-sanitizer/) enabled are able to function properly in a LX zone? Programs compiled with ASAN use 20GB of virtual memory. I am not sure if any special OS run-time support is required. I ask this because my main reason to use a Linux OS (vs some other OS like OmniOS) is to check my software using ASAN and valgrind for errors. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From garland at bustle.com Fri Oct 7 15:44:38 2016 From: garland at bustle.com (Garland McAlexander) Date: Fri, 7 Oct 2016 11:44:38 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 Message-ID: Hi All, I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling. Any tips on how to get this going? -- Garland McAlexander Technology Manager E: garland at bustle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1758 bytes Desc: not available URL: From daleg at omniti.com Fri Oct 7 17:07:38 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 7 Oct 2016 13:07:38 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: References: Message-ID: On Oct 7, 2016, at 11:44 AM, Garland McAlexander wrote: > > Hi All, > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > Any tips on how to get this going? Are you getting a 1Gb link instead of a 10Gb link? Can you run this: dladm set-linkprop en_10gfdx_cap=1 ixgbeN And check the link speed afterwards (dladm show-phys) /dale From mtalbott at lji.org Fri Oct 7 19:02:29 2016 From: mtalbott at lji.org (Michael Talbott) Date: Fri, 7 Oct 2016 12:02:29 -0700 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> Message-ID: <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org> Bummer. Oh well. Just fyi, I have BeeGFS running successfully on OmniOS in LX zones and is blazing fast. All of the services have been running in LX zones just fine so far except for the metadata server due to EA needs. So I just put the metadata server in a KVM instance until hopefully some day down the road xattrs are made available to LX zones ;) THANK YOU LX ZONES!!! > On Sep 30, 2016, at 8:52 PM, Dale Ghent wrote: > > >> On Sep 30, 2016, at 6:57 PM, Michael Talbott wrote: >> >> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html >> >> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :( >> >> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to. > > Negative, won't work because the illumos kernel doesn't grok ext4. > > In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs. > > This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure. > > For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs. > > /dale From daleg at omniti.com Fri Oct 7 19:46:00 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 7 Oct 2016 15:46:00 -0400 Subject: [OmniOS-discuss] extended attributes in LX zone In-Reply-To: <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org> Message-ID: <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com> Good to hear. With ZFS gaining traction in the Linux world, you might want to talk to the BeeGFS owners/maintainers about making their metadata server more filesystem-agnostic. > On Oct 7, 2016, at 3:02 PM, Michael Talbott wrote: > > Bummer. Oh well. Just fyi, I have BeeGFS running successfully on OmniOS in LX zones and is blazing fast. All of the services have been running in LX zones just fine so far except for the metadata server due to EA needs. So I just put the metadata server in a KVM instance until hopefully some day down the road xattrs are made available to LX zones ;) > > THANK YOU LX ZONES!!! > > >> On Sep 30, 2016, at 8:52 PM, Dale Ghent wrote: >> >> >>> On Sep 30, 2016, at 6:57 PM, Michael Talbott wrote: >>> >>> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html >>> >>> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :( >>> >>> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to. >> >> Negative, won't work because the illumos kernel doesn't grok ext4. >> >> In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs. >> >> This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure. >> >> For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs. >> >> /dale > From robertfantini at gmail.com Sat Oct 8 01:27:05 2016 From: robertfantini at gmail.com (Robert Fantini) Date: Fri, 7 Oct 2016 21:27:05 -0400 Subject: [OmniOS-discuss] using proxmox in a kvm running on napp-it/omnios Message-ID: We use proxmox and napp-it/omnios zfs iscsi storage for our 30 KVM' s and 10 LXC's . I want to cut down the amount of hardware we use. currently that is 2 storage and 3 pve systems. I think it should be possible to run proxmox with in a kvm running on omnios . then using local storage shared with iscsi to proxmox. I've a couple of quesitons: Is there anything wrong with going down that path? Is there documentation for setting up a kvm and using an iso file to install operating system using omnios ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From alka at hfg-gmuend.de Sat Oct 8 08:35:47 2016 From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=) Date: Sat, 8 Oct 2016 10:35:47 +0200 Subject: [OmniOS-discuss] Plex mediaserver in an LX zone In-Reply-To: <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org> <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com> Message-ID: <98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de> about LX I have not yet worked with this concept. Main problem seems the lack of documentation, blogs and howtos or a simple ready to use example for OmniOS. A good start would be a documented example like a Plex LX mediaserver. Has anyone used this in SmartOS and can give some informations about doing the same with OmniOS? I will add a support menu for handling and management of LX containers in napp-it then as I suppose that many non storage related features and add-ons are LX containers in future. see http://lightsandshapes.com/plex-on-smartos.html Gea/ napp-it.org From mir at miras.org Sat Oct 8 10:40:31 2016 From: mir at miras.org (Michael Rasmussen) Date: Sat, 8 Oct 2016 12:40:31 +0200 Subject: [OmniOS-discuss] using proxmox in a kvm running on napp-it/omnios In-Reply-To: References: Message-ID: <20161008124031.55d3aa9f@sleipner.datanom.net> On Fri, 7 Oct 2016 21:27:05 -0400 Robert Fantini wrote: > > I think it should be possible to run proxmox with in a kvm running on > omnios . then using local storage shared with iscsi to proxmox. > Since Omnios does not support nested KVM you will only be able to run containers. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -------------------------------------------------------------- /usr/games/fortune -es says: The future is a race between education and catastrophe. -- H. G. Wells -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 163 bytes Desc: OpenPGP digital signature URL: From denis at denisandyuki.net Sat Oct 8 12:28:19 2016 From: denis at denisandyuki.net (Denis Cheong) Date: Sat, 8 Oct 2016 23:28:19 +1100 Subject: [OmniOS-discuss] Plex mediaserver in an LX zone In-Reply-To: <98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de> References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org> <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org> <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org> <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com> <98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de> Message-ID: I am running Plex within an LX zone, there is very little that is particularly special about it. The article you referred to is a good start. Publish any required file systems over lofs, configure your file ownership and permissions correctly, and just run it as if it's Ubuntu. On 8 October 2016 at 19:35, G?nther Alka wrote: > about LX > I have not yet worked with this concept. > Main problem seems the lack of documentation, blogs and howtos or a simple > ready to use example for OmniOS. > > A good start would be a documented example like a Plex LX mediaserver. > Has anyone used this in SmartOS and can give some informations about doing > the same with OmniOS? I will add a support menu for handling and management > of LX containers in napp-it then as I suppose that many non storage related > features and add-ons are LX containers in future. > > see http://lightsandshapes.com/plex-on-smartos.html > > Gea/ napp-it.org > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skeltonr at btconnect.com Tue Oct 11 07:47:31 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Tue, 11 Oct 2016 08:47:31 +0100 Subject: [OmniOS-discuss] LX zone console broken Message-ID: <57FC9913.3080609@btconnect.com> Hi, I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news I am using the bloody ISO image I downloaded at the weekend. The only problem I have is the console login prompt fails. If i do :- # zlogin -C lx0 I see a no prompt. I can access the zone with :- # zlogin lx0 Looking in /var/log/messages I can see :- Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1 Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed What am I missing? Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Oct 11 14:13:03 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 11 Oct 2016 10:13:03 -0400 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <57FC9913.3080609@btconnect.com> References: <57FC9913.3080609@btconnect.com> Message-ID: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> > On Oct 11, 2016, at 3:47 AM, Richard Skelton wrote: > > Hi, > I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news > I am using the bloody ISO image I downloaded at the weekend. > The only problem I have is the console login prompt fails. > If i do :- > # zlogin -C lx0 > I see a no prompt. > I can access the zone with :- > # zlogin lx0 > Looking in /var/log/messages I can see :- > Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed > Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1 > Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning > Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed Which image did you install? I've seen lots of console problems, even on SmartOS. Dan From skeltonr at btconnect.com Tue Oct 11 16:03:15 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Tue, 11 Oct 2016 17:03:15 +0100 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> Message-ID: <57FD0D43.6020705@btconnect.com> Hi Dan, I used :- curl -o centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file zoneadm -z lx0 install -s /root/centos6.zss.gz zoneadm -z lx0 boot;zlogin -C lx0 Dan McDonald wrote: >> On Oct 11, 2016, at 3:47 AM, Richard Skelton wrote: >> >> Hi, >> I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news >> I am using the bloody ISO image I downloaded at the weekend. >> The only problem I have is the console login prompt fails. >> If i do :- >> # zlogin -C lx0 >> I see a no prompt. >> I can access the zone with :- >> # zlogin lx0 >> Looking in /var/log/messages I can see :- >> Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed >> Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1 >> Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning >> Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed >> > > Which image did you install? > > I've seen lots of console problems, even on SmartOS. > > Dan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skeltonr at btconnect.com Tue Oct 11 16:27:35 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Tue, 11 Oct 2016 17:27:35 +0100 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> Message-ID: <57FD12F7.1010704@btconnect.com> Hi Dan, I used :- curl -o centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file zoneadm -z lx0 install -s /root/centos6.zss.gz zoneadm -z lx0 boot;zlogin -C lx0 Dan McDonald wrote: >> On Oct 11, 2016, at 3:47 AM, Richard Skelton wrote: >> >> Hi, >> I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news >> I am using the bloody ISO image I downloaded at the weekend. >> The only problem I have is the console login prompt fails. >> If i do :- >> # zlogin -C lx0 >> I see a no prompt. >> I can access the zone with :- >> # zlogin lx0 >> Looking in /var/log/messages I can see :- >> Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed >> Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1 >> Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning >> Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed >> > > Which image did you install? > > I've seen lots of console problems, even on SmartOS. > > Dan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Oct 11 16:52:37 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 11 Oct 2016 12:52:37 -0400 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <57FD12F7.1010704@btconnect.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> Message-ID: <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> > On Oct 11, 2016, at 12:27 PM, Richard Skelton wrote: > > Hi Dan, > I used :- > curl -o centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file > zoneadm -z lx0 install -s /root/centos6.zss.gz > zoneadm -z lx0 boot;zlogin -C lx0 Yeah, I'm seeing this too on my CentOS 6 zone as well. I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too. I suspect I mismerged something. It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days. Dan From danmcd at omniti.com Tue Oct 11 17:01:19 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 11 Oct 2016 13:01:19 -0400 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> Message-ID: <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> > On Oct 11, 2016, at 12:52 PM, Dan McDonald wrote: > > I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too. I suspect I mismerged something. It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days. Not mismerge, but missing bugfix: OS-5685 LTP vhangup02 fails and passes Could you do me a favor and test a replacement lx_brand binary? Please respond unicast, and we'll report back to the list once your post-fix findings match mine. :) Thanks, Dan From waldenvik at gmx.com Tue Oct 11 17:53:23 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Tue, 11 Oct 2016 19:53:23 +0200 Subject: [OmniOS-discuss] Slow ssh transfers Message-ID: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> Hi I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow. I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware. I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios? Regards Martin From gearboxes at outlook.com Tue Oct 11 17:53:35 2016 From: gearboxes at outlook.com (Machine Man) Date: Tue, 11 Oct 2016 17:53:35 +0000 Subject: [OmniOS-discuss] NVMe/kvm Message-ID: I'm looking to deploy an all NVMe system, 2x 400GB for rpool and 4x 800GB for the data pool. I have had poor performance for virtual machines using either 10k or 15k drives and also adding mirrored ZIL and L2 cache SSD. All SAS of course. This got expensive fast and only striped mirror vdevs got to acceptable performance and ended up with far more TB than needed by tossing spindles at it. It also did not matter how I played around with blocksizes for the VM zvols. The IO requirements for some of the machines is too high. I'm looking to build a system that will be running 7 virtual machines. 4x Windows standard 2012r2 1x domain controller, 2x IIS systems (low client access) and 1x running an FTP commercial product. 1x SQL 2014 (only 300GB max size) 2x Windows 10 pro workstations The chassis will be a Supermicro SYS-1028U-TN10RT+ 192GB memory My thought is to use stripped mirrors on the data pool and enable dedup on all the vm disks except the SQL server. The reason for not using a commercial product like VMware is snapshots and offsite replication. We don?t have an environment with more than 5 VMs and no Microsoft SQL at the moment. We would prefer to spend on hardware rather than licensing and maintenance cost with added admin of constant software updates for hypervisor backup software and replication pieces etc. The 2 questions I have are: 1) Are all issues with NVMe resolved? 2) Is this beyond of what OmniOS was designed for and should I rather look at a commercial products for this type of deployment? Thanks, Sent from my Windows 10 phone -------------- next part -------------- An HTML attachment was scrubbed... URL: From skeltonr at btconnect.com Tue Oct 11 18:20:42 2016 From: skeltonr at btconnect.com (Richard Skelton) Date: Tue, 11 Oct 2016 19:20:42 +0100 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> Message-ID: <57FD2D7A.1020308@btconnect.com> Hi All, Dan provided an updated lx_brand file which I installed in /usr/kernel/brand/amd64/lx_brand After a reboot of the global zone all is now working :-) CentOS release 6.8 (Final) Kernel 2.6.32 on an x86_64 lx0 login: Dan McDonald wrote: >> On Oct 11, 2016, at 12:52 PM, Dan McDonald wrote: >> >> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too. I suspect I mismerged something. It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days. >> > > Not mismerge, but missing bugfix: > > OS-5685 LTP vhangup02 fails and passes > > Could you do me a favor and test a replacement lx_brand binary? Please respond unicast, and we'll report back to the list once your post-fix findings match mine. :) > > Thanks, > Dan > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Oct 11 19:14:46 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 11 Oct 2016 15:14:46 -0400 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> Message-ID: <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> Dumb question - is your FreeBSD frame size jumbo? Also, try this on OmniOS: ipadm set-prop -p send_buf=1048576 tcp ipadm set-prop -p recv_buf=1048576 tcp And see if that helps if the frame size isn't smaller on OmniOS. Dan Sent from my iPhone (typos, autocorrect, and all) > On Oct 11, 2016, at 1:53 PM, Martin Waldenvik wrote: > > Hi > > I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow. I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware. > > I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios? > > Regards > Martin > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From waldenvik at gmx.com Tue Oct 11 19:58:48 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Tue, 11 Oct 2016 21:58:48 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> Message-ID: On 10/11/2016 09:14 PM, Dan McDonald wrote: > Dumb question - is your FreeBSD frame size jumbo? > > Also, try this on OmniOS: > > ipadm set-prop -p send_buf=1048576 tcp > ipadm set-prop -p recv_buf=1048576 tcp > > And see if that helps if the frame size isn't smaller on OmniOS. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > >> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik wrote: >> >> Hi >> >> I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow. I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware. >> >> I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios? >> >> Regards >> Martin >> >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss Hi Dan Thanks for your input. I have had this problem for a long time, then i thought is was the old SunSSH but the problem seems to persist with OpenSSH. I have not touched the frame size, the only thing i have is LACP with L3,L4. Intel NIC on a supermicro board. Tried to set send and recv buffer but there was no difference. It's about the same speed with both sending and recieving ~40-45M/s. I have found that zrep is simple enough for me that manually replicates my system on an irregular basis. Maybe there is something that simple around that works with netcat. Regards Martin From jimklimov at cos.ru Tue Oct 11 20:50:34 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Tue, 11 Oct 2016 22:50:34 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> Message-ID: 11 ??????? 2016??. 21:58:48 CEST, Martin Waldenvik ?????: >On 10/11/2016 09:14 PM, Dan McDonald wrote: >> Dumb question - is your FreeBSD frame size jumbo? >> >> Also, try this on OmniOS: >> >> ipadm set-prop -p send_buf=1048576 tcp >> ipadm set-prop -p recv_buf=1048576 tcp >> >> And see if that helps if the frame size isn't smaller on OmniOS. >> >> Dan >> >> Sent from my iPhone (typos, autocorrect, and all) >> >>> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik >wrote: >>> >>> Hi >>> >>> I'm wondering if anyone can shed some light on a problem I have. I'm >using lftp sftp a lot and it is very slow. I can work around that >sometimes by using another server. But I am also using zrep (using ssh) >to replicate my pool to a backup server. This is also slow. Transfer is >about 40 megabyte (omnios to omnios). When doing this on freebsd it is >about 100 megabyte and it saturates my gigabit link (freebsd to >freebsd). Same hardware. >>> >>> I'm using the latest Omios LTS release and openssh. Is there any way >to speed this up on Omnios? >>> >>> Regards >>> Martin >>> >>> >>> _______________________________________________ >>> OmniOS-discuss mailing list >>> OmniOS-discuss at lists.omniti.com >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss > >Hi Dan > >Thanks for your input. I have had this problem for a long time, then i >thought is was the old SunSSH but the problem seems to persist with >OpenSSH. > >I have not touched the frame size, the only thing i have is LACP with >L3,L4. Intel NIC on a supermicro board. > >Tried to set send and recv buffer but there was no difference. It's >about the same speed with both sending and recieving ~40-45M/s. > >I have found that zrep is simple enough for me that manually replicates > >my system on an irregular basis. Maybe there is something that simple >around that works with netcat. > >Regards >Martin >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Does this problem hit any ssh transfers, or in particular zfs send/recv? The latter is known to be bursty and benefits a lot from caching by mbuffer on at least one side of the link. Also for curiosity, did you try the hpc ssh patches (build your own), and/or reduced-security protocols (arcfour), and/or inline compression (ssh -z or explicit gzip/gzcat in the pipe)? Jim -- Typos courtesy of K-9 Mail on my Samsung Android From jstockett at molalla.com Tue Oct 11 22:03:06 2016 From: jstockett at molalla.com (Jeff Stockett) Date: Tue, 11 Oct 2016 22:03:06 +0000 Subject: [OmniOS-discuss] problem with boot-archive after upgrade Message-ID: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com> I upgraded a system from 012 to 018 just now and everything went totally smoothly - system rebooted fine. However, the boot environment names weren't what I wanted and since beadm can't rename the active one, I create a new one, activated it, and rebooted again. On reboot this second (and subsequent times), I'm getting this in /var/src/log/system-boot-archive:default.log: [ Mar 20 14:25:01 Executing start method ("/lib/svc/method/boot-archive"). ] cannot find: /etc/cluster/nodeid: No such file or directory cannot find: /etc/devices/mdi_ib_cache: No such file or directory cannot find: /etc/devices/retire_store: No such file or directory [ Mar 20 14:25:50 Method "start" exited with status 0. ] [ May 18 05:20:09 Enabled. ] [ May 18 05:20:14 Executing start method ("/lib/svc/method/boot-archive"). ] [ May 18 05:21:15 Method or service exit timed out. Killing contract 16. ] [ May 18 05:21:15 Method "start" failed due to signal KILL. ] The system eventually comes up to a single user mode login, and if I login as root, and enter "svcadm clear system/boot-archive" and/or maybe just wait a while longer, then logout the system eventually comes up and works fine in multi-user mode, but I was wondering if there is an easy way to fix whatever I've done to shoot myself in the foot here? I tried "bootadm update-archive" but that doesn't appear to fix it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guorong.koh at gmail.com Wed Oct 12 00:02:25 2016 From: guorong.koh at gmail.com (Guo-Rong Koh) Date: Wed, 12 Oct 2016 10:32:25 +1030 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> Message-ID: <1476230545.603.5.camel@gmail.com> On Tue, 2016-10-11 at 21:58 +0200, Martin Waldenvik wrote: > On 10/11/2016 09:14 PM, Dan McDonald wrote: > > Dumb question - is your FreeBSD frame size jumbo? > > > > Also, try this on OmniOS: > > > > ipadm set-prop -p send_buf=1048576 tcp > > ipadm set-prop -p recv_buf=1048576 tcp > > > > And see if that helps if the frame size isn't smaller on OmniOS. > > > > Dan > > > > Sent from my iPhone (typos, autocorrect, and all) > > > > > > > > > > > On Oct 11, 2016, at 1:53 PM, Martin Waldenvik wrote: > > > > > > Hi > > > > > > > > > > > > > > > > > > > > > > > > I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow.??I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios??to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware. > > > > > > > > > I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios? > > > > > > Regards > > > Martin > > > > > > > > > _______________________________________________ > > > OmniOS-discuss mailing list > > > OmniOS-discuss at lists.omniti.com > > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > Hi Dan > > > Thanks for your input. I have had this problem for a long time, then i? > > thought is was the old SunSSH but the problem seems to persist with OpenSSH. > > > I have not touched the frame size, the only thing i have is LACP with? > L3,L4. Intel NIC on a supermicro board. > > Tried to set send and recv buffer but there was no difference. It's? > about the same speed with both sending and recieving ~40-45M/s. > > > I have found that zrep is simple enough for me that manually replicates? > > my system on an irregular basis. Maybe there is something that simple? > around that works with netcat. Hi Martin, Have you considered changing/choosing the cipher used in the sftp transfer? I found something like this: https://turecki.net/content/getting-most-out-ssh-hardware-acceleration- tuning-aes-ni enabled scp to saturate my gigabit link whilst saving CPU. Prior to that, the default negotiated cipher seemed to peak at ~50MB/s. regards, Guo-Rong -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Oct 12 02:39:41 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 11 Oct 2016 22:39:41 -0400 Subject: [OmniOS-discuss] LX zone console broken In-Reply-To: <57FD2D7A.1020308@btconnect.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> Message-ID: <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> The bloody repo server has an updated LX brand package with this fix. Dan Sent from my iPhone (typos, autocorrect, and all) > On Oct 11, 2016, at 2:20 PM, Richard Skelton wrote: > > Hi All, > Dan provided an updated lx_brand file which I installed in /usr/kernel/brand/amd64/lx_brand > After a reboot of the global zone all is now working :-) > > CentOS release 6.8 (Final) > Kernel 2.6.32 on an x86_64 > > lx0 login: > > > > Dan McDonald wrote: >> >>> On Oct 11, 2016, at 12:52 PM, Dan McDonald wrote: >>> >>> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too. I suspect I mismerged something. It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days. >>> >> >> Not mismerge, but missing bugfix: >> >> OS-5685 LTP vhangup02 fails and passes >> >> Could you do me a favor and test a replacement lx_brand binary? Please respond unicast, and we'll report back to the list once your post-fix findings match mine. :) >> >> Thanks, >> Dan >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From natxo.asenjo at gmail.com Wed Oct 12 12:54:09 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 12 Oct 2016 14:54:09 +0200 Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup tool Message-ID: hi, There is a w10 system at the home and it's a bit critical for the wife ;-), so I want to have this up and running asap if some kind of trouble happens with it. Anyway, this windows 10 OS has a backup tool that allows you to dump an image to a network share using cifs, obviously. So I have a zfs file system, shared it for cifs, set the permissions for the share. Start the backup, and it will fail consistently when it has reached 57%. After much searching, I found this: https://social.technet.microsoft.com/Forums/windows/en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup-win7-file-recovery-system-image-creation-to-nas-device-fails-with-error?forum=w8itprogeneral And yes, after: # sharectl set -p max_protocol=1 smb and restarting the cifs service, now the backup is running correctly (97% and counting). Apparently it has to do with 'resilency support', whatever that may be: https://support.microsoft.com/en-us/kb/2920193 Which is a shame, because now the cifs host is limited to smb 1.5 instead of 2.1 (but at least it works). Hopefully I spare somebody some unnecessary pain with this info. And it would be awesome if this 'resiliciency support' made it to the cifs server in a next version ;-) Thanks! -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From waldenvik at gmx.com Wed Oct 12 14:21:43 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Wed, 12 Oct 2016 16:21:43 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <1476230545.603.5.camel@gmail.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> Message-ID: <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> On 10/12/2016 02:02 AM, Guo-Rong Koh wrote: > On Tue, 2016-10-11 at 21:58 +0200, Martin Waldenvik wrote: >> On 10/11/2016 09:14 PM, Dan McDonald wrote: >>> Dumb question - is your FreeBSD frame size jumbo? Also, try this on >>> OmniOS: ipadm set-prop -p send_buf=1048576 tcp ipadm set-prop -p >>> recv_buf=1048576 tcp And see if that helps if the frame size isn't >>> smaller on OmniOS. Dan Sent from my iPhone (typos, autocorrect, and all) >>>> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik >>> > wrote: Hi I'm wondering if anyone can >>>> shed some light on a problem I have. I'm using lftp sftp a lot and >>>> it is very slow. I can work around that sometimes by using another >>>> server. But I am also using zrep (using ssh) to replicate my pool to >>>> a backup server. This is also slow. Transfer is about 40 megabyte >>>> (omnios to omnios). When doing this on freebsd it is about 100 >>>> megabyte and it saturates my gigabit link (freebsd to freebsd). Same >>>> hardware. I'm using the latest Omios LTS release and openssh. Is >>>> there any way to speed this up on Omnios? Regards Martin >>>> _______________________________________________ OmniOS-discuss >>>> mailing list OmniOS-discuss at lists.omniti.com >>>> >>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> >> Hi Dan >> >> Thanks for your input. I have had this problem for a long time, then i >> thought is was the old SunSSH but the problem seems to persist with OpenSSH. >> >> I have not touched the frame size, the only thing i have is LACP with >> L3,L4. Intel NIC on a supermicro board. >> >> Tried to set send and recv buffer but there was no difference. It's >> about the same speed with both sending and recieving ~40-45M/s. >> >> I have found that zrep is simple enough for me that manually replicates >> my system on an irregular basis. Maybe there is something that simple >> around that works with netcat. > > Hi Martin, > > Have you considered changing/choosing the cipher used in the sftp transfer? > I found something like this: > https://turecki.net/content/getting-most-out-ssh-hardware-acceleration-tuning-aes-ni > enabled scp to saturate my gigabit link whilst saving CPU. > > Prior to that, the default negotiated cipher seemed to peak at ~50MB/s. > > regards, > Guo-Rong > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > Thank you so much Guo-Rong for your help. When i changed from: Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr I get full speed. I am so happy for this. Warm regards Martin From danmcd at omniti.com Wed Oct 12 14:37:11 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 12 Oct 2016 10:37:11 -0400 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> Message-ID: <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> > On Oct 12, 2016, at 10:21 AM, Martin Waldenvik wrote: >> > Thank you so much Guo-Rong for your help. When i changed from: > Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr > > to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr > > I get full speed. I am so happy for this. I'm curious about one thing. Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default? Or does chacha20 run faster on FreeBSD? Thanks, Dan From daleg at omniti.com Wed Oct 12 14:41:07 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 10:41:07 -0400 Subject: [OmniOS-discuss] problem with boot-archive after upgrade In-Reply-To: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com> References: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com> Message-ID: <333BA485-2796-4744-A22F-F30355EE4B36@omniti.com> > On Oct 11, 2016, at 6:03 PM, Jeff Stockett wrote: > > I upgraded a system from 012 to 018 just now and everything went totally smoothly ? system rebooted fine. However, the boot environment names weren?t what I wanted and since beadm can?t rename the active one, I create a new one, activated it, and rebooted again. On reboot this second (and subsequent times), I?m getting this in /var/src/log/system-boot-archive:default.log: > > [ Mar 20 14:25:01 Executing start method ("/lib/svc/method/boot-archive"). ] > cannot find: /etc/cluster/nodeid: No such file or directory > cannot find: /etc/devices/mdi_ib_cache: No such file or directory > cannot find: /etc/devices/retire_store: No such file or directory > [ Mar 20 14:25:50 Method "start" exited with status 0. ] > [ May 18 05:20:09 Enabled. ] > [ May 18 05:20:14 Executing start method ("/lib/svc/method/boot-archive"). ] > [ May 18 05:21:15 Method or service exit timed out. Killing contract 16. ] > [ May 18 05:21:15 Method "start" failed due to signal KILL. ] > > The system eventually comes up to a single user mode login, and if I login as root, and enter ?svcadm clear system/boot-archive? and/or maybe just wait a while longer, then logout the system eventually comes up and works fine in multi-user mode, but I was wondering if there is an easy way to fix whatever I?ve done to shoot myself in the foot here? I tried ?bootadm update-archive? but that doesn?t appear to fix it. When you manually ran 'bootadm update-archive' did it give you any output such as "updating..." , or did it silently exit? If the latter, do something like 'touch /etc/system' first, then run it, as bootadm will regenerate the boot archive only if it detects any changes in the files its concerned with. And yes, I've seen this same problem you speak of on rare occasion, with a rebuild of the boot archive being the cure. /dale From waldenvik at gmx.com Wed Oct 12 14:45:20 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Wed, 12 Oct 2016 16:45:20 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> Message-ID: On 10/12/2016 04:37 PM, Dan McDonald wrote: > >> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik wrote: >>> >> Thank you so much Guo-Rong for your help. When i changed from: >> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >> >> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >> >> I get full speed. I am so happy for this. > > I'm curious about one thing. Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default? Or does chacha20 run faster on FreeBSD? > > Thanks, > Dan > Hi Dan I have used those ciphers in that order on all my servers and it works in full speed i both freebsd and linux but not in Omnios. I have configured my ssh-servers as recommended here: https://stribika.github.io/2015/01/04/secure-secure-shell.html Regards Martin From daleg at omniti.com Wed Oct 12 14:53:01 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 10:53:01 -0400 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> Message-ID: <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com> > On Oct 12, 2016, at 10:45 AM, Martin Waldenvik wrote: > > On 10/12/2016 04:37 PM, Dan McDonald wrote: >> >>> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik wrote: >>>> >>> Thank you so much Guo-Rong for your help. When i changed from: >>> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >>> >>> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >>> >>> I get full speed. I am so happy for this. >> >> I'm curious about one thing. Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default? Or does chacha20 run faster on FreeBSD? >> >> Thanks, >> Dan >> > > Hi Dan > > I have used those ciphers in that order on all my servers and it works in full speed i both freebsd and linux but not in Omnios. > > I have configured my ssh-servers as recommended here: > > https://stribika.github.io/2015/01/04/secure-secure-shell.html Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and OmniOS boxes? /dale From jdg117 at elvis.arl.psu.edu Wed Oct 12 15:00:58 2016 From: jdg117 at elvis.arl.psu.edu (John D Groenveld) Date: Wed, 12 Oct 2016 11:00:58 -0400 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: Your message of "Wed, 12 Oct 2016 16:45:20 +0200." References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> Message-ID: <201610121500.u9CF0wQF018308@elvis.arl.psu.edu> In message , Martin Waldenvik wri tes: >I have used those ciphers in that order on all my servers and it works >in full speed i both freebsd and linux but not in Omnios. If your FreeBSD and OmniOS installs are on similar hardware, openssl(1) speed benchmark might be interesting. John groenveld at acm.org From waldenvik at gmx.com Wed Oct 12 15:02:27 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Wed, 12 Oct 2016 17:02:27 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com> Message-ID: <409c3004-6182-5285-a02c-de532002fca9@gmx.com> On 10/12/2016 04:53 PM, Dale Ghent wrote: > >> On Oct 12, 2016, at 10:45 AM, Martin Waldenvik wrote: >> >> On 10/12/2016 04:37 PM, Dan McDonald wrote: >>> >>>> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik wrote: >>>>> >>>> Thank you so much Guo-Rong for your help. When i changed from: >>>> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >>>> >>>> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr >>>> >>>> I get full speed. I am so happy for this. >>> >>> I'm curious about one thing. Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default? Or does chacha20 run faster on FreeBSD? >>> >>> Thanks, >>> Dan >>> >> >> Hi Dan >> >> I have used those ciphers in that order on all my servers and it works in full speed i both freebsd and linux but not in Omnios. >> >> I have configured my ssh-servers as recommended here: >> >> https://stribika.github.io/2015/01/04/secure-secure-shell.html > > Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and OmniOS boxes? > > /dale > Hi Seems to be the same Omnios box - ssh -Q cipher 3des-cbc blowfish-cbc cast128-cbc arcfour arcfour128 arcfour256 aes128-cbc aes192-cbc aes256-cbc rijndael-cbc at lysator.liu.se aes128-ctr aes192-ctr aes256-ctr aes128-gcm at openssh.com aes256-gcm at openssh.com chacha20-poly1305 at openssh.com FreeBSD box - ssh -Q cipher: 3des-cbc blowfish-cbc cast128-cbc arcfour arcfour128 arcfour256 aes128-cbc aes192-cbc aes256-cbc rijndael-cbc at lysator.liu.se aes128-ctr aes192-ctr aes256-ctr aes128-gcm at openssh.com aes256-gcm at openssh.com chacha20-poly1305 at openssh.com From garland at bustle.com Wed Oct 12 15:59:21 2016 From: garland at bustle.com (Garland McAlexander) Date: Wed, 12 Oct 2016 11:59:21 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: References: Message-ID: Yes, I'm getting a 1GbE link. Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE. Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > On Oct 7, 2016, at 11:44 AM, Garland McAlexander > wrote: > > > > Hi All, > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works > perfectly fine at 10GbE if using Windows or CentOS on the same machine. > Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > Any tips on how to get this going? > > Are you getting a 1Gb link instead of a 10Gb link? > > Can you run this: > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > And check the link speed afterwards (dladm show-phys) > > /dale > > -- Garland McAlexander Technology Manager E: garland at bustle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1758 bytes Desc: not available URL: From gearboxes at outlook.com Wed Oct 12 16:00:23 2016 From: gearboxes at outlook.com (Machine Man) Date: Wed, 12 Oct 2016 16:00:23 +0000 Subject: [OmniOS-discuss] KVM +NVMe + dedup Message-ID: I am looking to build an all NVMe system using: SYS-1028U-TN10RT+ OS will be on a mirror pair of 400GB and there will be a data pool of 4x 800GB in striped mirror pair. (there will be one 2x small VMs on rpool) The system will run 7 virtual machines (most will be Windows 2012R2 and one a 300GB SQL 2014 server) configured in KVM on OmniOS. The main reason for not using 10K or 15K drives is performance and dedup. I would like to get down to the a 1U chassis for this build. I have not been that successful with SAS 10/15K configuration and using KVM. I have tried different block sizes and some tuning and it could just be the load of these systems. We have a number of system running for years now, but never considered dedup largely due to fear of all the negative post, but most could also be due incorrect hardware configurations. My main reason for posting is to perhaps get feedback if this route can be considered or to stick to 2U 24x drive chassis and use more spindles without dedup. The system will have 192GB of memory and the VMs will not even consume close to 100GB. CPUs will be 2x 2.6Ghz 6core E5-2600 v3. The systems are fairly critical and the reason for not using VMware or commercial products is for snapshots and to cut down on how many products we need to update (hypervisor, backup software, replication components). Also, when using dedup, is there a higher disk of data corruption in the event of power outage? Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From waldenvik at gmx.com Wed Oct 12 16:05:14 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Wed, 12 Oct 2016 18:05:14 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <201610121500.u9CF0wQF018308@elvis.arl.psu.edu> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> <201610121500.u9CF0wQF018308@elvis.arl.psu.edu> Message-ID: <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com> On 10/12/2016 05:00 PM, John D Groenveld wrote: > In message , Martin Waldenvik wri > tes: >> I have used those ciphers in that order on all my servers and it works >> in full speed i both freebsd and linux but not in Omnios. > > If your FreeBSD and OmniOS installs are on similar hardware, > openssl(1) speed benchmark might be interesting. > > John > groenveld at acm.org > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > Hi Not really similar. Freebsd is a xeon 1260L (sandy bridge, 2.4GHz) and Omnios a xeon 1230L (haswell, 1.8GHz). Ran some tests. https://calomel.org/aesni_ssl_performance.html omnios - ./openssl speed -elapsed -evp chacha The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes chacha 111089.26k 148893.74k 155299.16k 159068.16k 159257.94k freebsd - ./openssl speed -elapsed -evp chacha The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes chacha 160908.79k 258242.61k 274012.32k 277625.70k 278313.86k omnios - ./openssl speed -elapsed -evp aes-256-gcm The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-gcm 36546.42k 39239.49k 39289.51k 39519.91k 39613.78k freebsd - ./openssl speed -elapsed -evp aes-256-gcm The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-gcm 222688.71k 638351.04k 889610.92k 993149.57k 1017445.72k Martin From gordon.w.ross at gmail.com Wed Oct 12 16:07:49 2016 From: gordon.w.ross at gmail.com (Gordon Ross) Date: Wed, 12 Oct 2016 12:07:49 -0400 Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup tool In-Reply-To: References: Message-ID: Interesting finding re. VHD support needing resiliency support in SMB 2.1. Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193 We have resiliency support in NexentaStor 5.0 (shameless plug:) https://nexenta.com/products/nexentastor We should work on upstreaming that code soon. On Wed, Oct 12, 2016 at 8:54 AM, Natxo Asenjo wrote: > hi, > > There is a w10 system at the home and it's a bit critical for the wife ;-), > so I want to have this up and running asap if some kind of trouble happens > with it. > > Anyway, this windows 10 OS has a backup tool that allows you to dump an > image to a network share using cifs, obviously. > > So I have a zfs file system, shared it for cifs, set the permissions for the > share. Start the backup, and it will fail consistently when it has reached > 57%. > > After much searching, I found this: > https://social.technet.microsoft.com/Forums/windows/en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup-win7-file-recovery-system-image-creation-to-nas-device-fails-with-error?forum=w8itprogeneral > > And yes, after: > > # sharectl set -p max_protocol=1 smb > > and restarting the cifs service, now the backup is running correctly (97% > and counting). > > Apparently it has to do with 'resilency support', whatever that may be: > > https://support.microsoft.com/en-us/kb/2920193 > > Which is a shame, because now the cifs host is limited to smb 1.5 instead of > 2.1 (but at least it works). > > Hopefully I spare somebody some unnecessary pain with this info. And it > would be awesome if this 'resiliciency support' made it to the cifs server > in a next version ;-) > > Thanks! > > -- > Groeten, > natxo > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > From daleg at omniti.com Wed Oct 12 16:19:14 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 12:19:14 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: References: Message-ID: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) how about setting that to 0, then back to 1? /dale > On Oct 12, 2016, at 11:59 AM, Garland McAlexander wrote: > > Yes, I'm getting a 1GbE link. > > Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE. > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > On Oct 7, 2016, at 11:44 AM, Garland McAlexander wrote: > > > > Hi All, > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > Any tips on how to get this going? > > Are you getting a 1Gb link instead of a 10Gb link? > > Can you run this: > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > And check the link speed afterwards (dladm show-phys) > > /dale > > > > > -- > Garland McAlexander > > Technology Manager > > E: garland at bustle.com > > > From garland at bustle.com Wed Oct 12 16:23:50 2016 From: garland at bustle.com (Garland McAlexander) Date: Wed, 12 Oct 2016 12:23:50 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> References: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> Message-ID: Hah, yes, I tried with both interface 0 and 1. Changing that back and forth has no change. Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are set to auto, and will show as STATE - Down and SPEED - 0 when I set my switch ports to 10GbE. On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent wrote: > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) > > how about setting that to 0, then back to 1? > > /dale > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander > wrote: > > > > Yes, I'm getting a 1GbE link. > > > > Randomly it'll work as 10GbE after a reboot, but another reboot will > cause it to go back to 1GbE. > > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander > wrote: > > > > > > Hi All, > > > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works > perfectly fine at 10GbE if using Windows or CentOS on the same machine. > Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > > > Any tips on how to get this going? > > > > Are you getting a 1Gb link instead of a 10Gb link? > > > > Can you run this: > > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > > > And check the link speed afterwards (dladm show-phys) > > > > /dale > > > > > > > > > > -- > > Garland McAlexander > > > > Technology Manager > > > > E: garland at bustle.com > > > > > > > > -- Garland McAlexander Technology Manager E: garland at bustle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1758 bytes Desc: not available URL: From daleg at omniti.com Wed Oct 12 16:25:53 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 12:25:53 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: References: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> Message-ID: <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com> Oof, I realized I may not have been clear in how I worded that. What I meant to ask is: What happens when you set en_10gfdx_cap to 0, then back to 1 ? > On Oct 12, 2016, at 12:23 PM, Garland McAlexander wrote: > > Hah, yes, I tried with both interface 0 and 1. > > Changing that back and forth has no change. > > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are set to auto, and will show as STATE - Down and SPEED - 0 when I set my switch ports to 10GbE. > > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent wrote: > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) > > how about setting that to 0, then back to 1? > > /dale > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander wrote: > > > > Yes, I'm getting a 1GbE link. > > > > Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE. > > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander wrote: > > > > > > Hi All, > > > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > > > Any tips on how to get this going? > > > > Are you getting a 1Gb link instead of a 10Gb link? > > > > Can you run this: > > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > > > And check the link speed afterwards (dladm show-phys) > > > > /dale > > > > > > > > > > -- > > Garland McAlexander > > > > Technology Manager > > > > E: garland at bustle.com > > > > > > > > > > > -- > Garland McAlexander > > Technology Manager > > E: garland at bustle.com > > > From garland at bustle.com Wed Oct 12 16:29:14 2016 From: garland at bustle.com (Garland McAlexander) Date: Wed, 12 Oct 2016 12:29:14 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com> References: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com> Message-ID: No no, I understood. When I set it to 0 and back to 1, the link will lose connection. On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent wrote: > > Oof, I realized I may not have been clear in how I worded that. What I > meant to ask is: > > What happens when you set en_10gfdx_cap to 0, then back to 1 ? > > > On Oct 12, 2016, at 12:23 PM, Garland McAlexander > wrote: > > > > Hah, yes, I tried with both interface 0 and 1. > > > > Changing that back and forth has no change. > > > > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are > set to auto, and will show as STATE - Down and SPEED - 0 when I set my > switch ports to 10GbE. > > > > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent wrote: > > > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) > > > > how about setting that to 0, then back to 1? > > > > /dale > > > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander > wrote: > > > > > > Yes, I'm getting a 1GbE link. > > > > > > Randomly it'll work as 10GbE after a reboot, but another reboot will > cause it to go back to 1GbE. > > > > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > > > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander > wrote: > > > > > > > > Hi All, > > > > > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It > works perfectly fine at 10GbE if using Windows or CentOS on the same > machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > > > > > Any tips on how to get this going? > > > > > > Are you getting a 1Gb link instead of a 10Gb link? > > > > > > Can you run this: > > > > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > > > > > And check the link speed afterwards (dladm show-phys) > > > > > > /dale > > > > > > > > > > > > > > > -- > > > Garland McAlexander > > > > > > Technology Manager > > > > > > E: garland at bustle.com > > > > > > > > > > > > > > > > > > > -- > > Garland McAlexander > > > > Technology Manager > > > > E: garland at bustle.com > > > > > > > > -- Garland McAlexander Technology Manager E: garland at bustle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1758 bytes Desc: not available URL: From daleg at omniti.com Wed Oct 12 16:34:07 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 12:34:07 -0400 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> <201610121500.u9CF0wQF018308@elvis.arl.psu.edu> <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com> Message-ID: <270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com> > On Oct 12, 2016, at 12:05 PM, Martin Waldenvik wrote: > > omnios - ./openssl speed -elapsed -evp aes-256-gcm > > The 'numbers' are in 1000s of bytes per second processed. > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes > aes-256-gcm 36546.42k 39239.49k 39289.51k 39519.91k 39613.78k > > freebsd - ./openssl speed -elapsed -evp aes-256-gcm > > The 'numbers' are in 1000s of bytes per second processed. > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes > aes-256-gcm 222688.71k 638351.04k 889610.92k 993149.57k 1017445.72k > That's still quite slow on the OmniOS box. On a 3.5GHz Haswell-E (E3-1241 v3) I get: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-gcm 394656.68k 1001238.31k 2094818.99k 2407235.93k 2689916.93k Perhaps you may want to check the BIOS settings of your OmniOS box and ensure that "AES-NI Instructions" are turned on. Your hardware vendor might have that termed differently - AES Acceleration and such. /dale From daleg at omniti.com Wed Oct 12 16:34:58 2016 From: daleg at omniti.com (Dale Ghent) Date: Wed, 12 Oct 2016 12:34:58 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: References: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com> Message-ID: <70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com> Interesting. So your switch configuration is set to nail the ports at 10Gbe rather than allow autoneg? /dale > On Oct 12, 2016, at 12:29 PM, Garland McAlexander wrote: > > No no, I understood. > > When I set it to 0 and back to 1, the link will lose connection. > > On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent wrote: > > Oof, I realized I may not have been clear in how I worded that. What I meant to ask is: > > What happens when you set en_10gfdx_cap to 0, then back to 1 ? > > > On Oct 12, 2016, at 12:23 PM, Garland McAlexander wrote: > > > > Hah, yes, I tried with both interface 0 and 1. > > > > Changing that back and forth has no change. > > > > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are set to auto, and will show as STATE - Down and SPEED - 0 when I set my switch ports to 10GbE. > > > > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent wrote: > > > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) > > > > how about setting that to 0, then back to 1? > > > > /dale > > > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander wrote: > > > > > > Yes, I'm getting a 1GbE link. > > > > > > Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE. > > > > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > > > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander wrote: > > > > > > > > Hi All, > > > > > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > > > > > Any tips on how to get this going? > > > > > > Are you getting a 1Gb link instead of a 10Gb link? > > > > > > Can you run this: > > > > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > > > > > And check the link speed afterwards (dladm show-phys) > > > > > > /dale > > > > > > > > > > > > > > > -- > > > Garland McAlexander > > > > > > Technology Manager > > > > > > E: garland at bustle.com > > > > > > > > > > > > > > > > > > > -- > > Garland McAlexander > > > > Technology Manager > > > > E: garland at bustle.com > > > > > > > > > > > -- > Garland McAlexander > > Technology Manager > > E: garland at bustle.com > > > From garland at bustle.com Wed Oct 12 16:35:54 2016 From: garland at bustle.com (Garland McAlexander) Date: Wed, 12 Oct 2016 12:35:54 -0400 Subject: [OmniOS-discuss] Issues with 10GbE - X540 In-Reply-To: <70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com> References: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com> <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com> <70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com> Message-ID: They're autoneg, but I've been trying everything (including forcing to 10GbE). For testing, I currently have one port set to autoneg, and one port to 10GbE to try and find a solution. On Wed, Oct 12, 2016 at 12:34 PM, Dale Ghent wrote: > > Interesting. So your switch configuration is set to nail the ports at > 10Gbe rather than allow autoneg? > > /dale > > > On Oct 12, 2016, at 12:29 PM, Garland McAlexander > wrote: > > > > No no, I understood. > > > > When I set it to 0 and back to 1, the link will lose connection. > > > > On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent wrote: > > > > Oof, I realized I may not have been clear in how I worded that. What I > meant to ask is: > > > > What happens when you set en_10gfdx_cap to 0, then back to 1 ? > > > > > On Oct 12, 2016, at 12:23 PM, Garland McAlexander > wrote: > > > > > > Hah, yes, I tried with both interface 0 and 1. > > > > > > Changing that back and forth has no change. > > > > > > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports > are set to auto, and will show as STATE - Down and SPEED - 0 when I set my > switch ports to 10GbE. > > > > > > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent wrote: > > > > > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0) > > > > > > how about setting that to 0, then back to 1? > > > > > > /dale > > > > > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander < > garland at bustle.com> wrote: > > > > > > > > Yes, I'm getting a 1GbE link. > > > > > > > > Randomly it'll work as 10GbE after a reboot, but another reboot will > cause it to go back to 1GbE. > > > > > > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change. > > > > > > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent wrote: > > > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander > wrote: > > > > > > > > > > Hi All, > > > > > > > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It > works perfectly fine at 10GbE if using Windows or CentOS on the same > machine. Just to be safe, I've tried multiple switches and Cat6a cabling. > > > > > > > > > > Any tips on how to get this going? > > > > > > > > Are you getting a 1Gb link instead of a 10Gb link? > > > > > > > > Can you run this: > > > > > > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN > > > > > > > > And check the link speed afterwards (dladm show-phys) > > > > > > > > /dale > > > > > > > > > > > > > > > > > > > > -- > > > > Garland McAlexander > > > > > > > > Technology Manager > > > > > > > > E: garland at bustle.com > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Garland McAlexander > > > > > > Technology Manager > > > > > > E: garland at bustle.com > > > > > > > > > > > > > > > > > > > -- > > Garland McAlexander > > > > Technology Manager > > > > E: garland at bustle.com > > > > > > > > -- Garland McAlexander Technology Manager E: garland at bustle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1758 bytes Desc: not available URL: From waldenvik at gmx.com Wed Oct 12 16:39:07 2016 From: waldenvik at gmx.com (Martin Waldenvik) Date: Wed, 12 Oct 2016 18:39:07 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> <201610121500.u9CF0wQF018308@elvis.arl.psu.edu> <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com> <270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com> Message-ID: <1afc4341-e212-ec85-c467-543ac201a34a@gmx.com> On 10/12/2016 06:34 PM, Dale Ghent wrote: > >> On Oct 12, 2016, at 12:05 PM, Martin Waldenvik wrote: >> >> omnios - ./openssl speed -elapsed -evp aes-256-gcm >> >> The 'numbers' are in 1000s of bytes per second processed. >> type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes >> aes-256-gcm 36546.42k 39239.49k 39289.51k 39519.91k 39613.78k >> >> freebsd - ./openssl speed -elapsed -evp aes-256-gcm >> >> The 'numbers' are in 1000s of bytes per second processed. >> type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes >> aes-256-gcm 222688.71k 638351.04k 889610.92k 993149.57k 1017445.72k >> > > That's still quite slow on the OmniOS box. On a 3.5GHz Haswell-E (E3-1241 v3) I get: > > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes > aes-256-gcm 394656.68k 1001238.31k 2094818.99k 2407235.93k 2689916.93k > > Perhaps you may want to check the BIOS settings of your OmniOS box and ensure that "AES-NI Instructions" are turned on. Your hardware vendor might have that termed differently - AES Acceleration and such. > > /dale > Must be someting wrong with what i did. Used libressl to test chacha. This is with built in openssl: type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-gcm 309921.38k 784682.82k 1614856.62k 1867160.92k 2117028.52k Looks a lot better. Martin From natxo.asenjo at gmail.com Wed Oct 12 17:11:52 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 12 Oct 2016 19:11:52 +0200 Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup tool In-Reply-To: <136C13E89D22BB468B2A7025993639736E1293D1@EXMCCMB.molalla.com> References: <136C13E89D22BB468B2A7025993639736E1293D1@EXMCCMB.molalla.com> Message-ID: On Wed, Oct 12, 2016 at 5:00 PM, Jeff Stockett wrote: > FWIW, you could create a backup volume and share it via iscsi to your > wife?s PC ? quite a few commands to accomplish on the omnios side but easy > if you have nappit already. Once you attach it, run windows backup, and it > will immediately see it as an available backup location and initialize it > for just that purpose if you let it. Then you could leave your CIFS shares > 2.1. > sure, you can offer an iscsi disk but recovering bare metal from iscsi targets is not as easy as from a share ;-) and well, as I read in the thread, Nexenta already has this and will upstream their work (thanks! awesome work), so for the time being I guess I can either turn smb 1 on/off every now and then for a backup, or just live with a slower cifs protocol. -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From natxo.asenjo at gmail.com Wed Oct 12 17:12:05 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 12 Oct 2016 19:12:05 +0200 Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup tool In-Reply-To: References: Message-ID: On Wed, Oct 12, 2016 at 6:07 PM, Gordon Ross wrote: > Interesting finding re. VHD support needing resiliency support in SMB 2.1. > Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193 > > We have resiliency support in NexentaStor 5.0 (shameless plug:) > https://nexenta.com/products/nexentastor > We should work on upstreaming that code soon. > > > On Wed, Oct 12, 2016 at 8:54 AM, Natxo Asenjo > wrote: > > hi, > > > > There is a w10 system at the home and it's a bit critical for the wife > ;-), > > so I want to have this up and running asap if some kind of trouble > happens > > with it. > > > > Anyway, this windows 10 OS has a backup tool that allows you to dump an > > image to a network share using cifs, obviously. > > > > So I have a zfs file system, shared it for cifs, set the permissions for > the > > share. Start the backup, and it will fail consistently when it has > reached > > 57%. > > > > After much searching, I found this: > > https://social.technet.microsoft.com/Forums/windows/ > en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup- > win7-file-recovery-system-image-creation-to-nas-device- > fails-with-error?forum=w8itprogeneral > > > > And yes, after: > > > > # sharectl set -p max_protocol=1 smb > > > > and restarting the cifs service, now the backup is running correctly (97% > > and counting). > > > > Apparently it has to do with 'resilency support', whatever that may be: > > > > https://support.microsoft.com/en-us/kb/2920193 > > > > Which is a shame, because now the cifs host is limited to smb 1.5 > instead of > > 2.1 (but at least it works). > > > > Hopefully I spare somebody some unnecessary pain with this info. And it > > would be awesome if this 'resiliciency support' made it to the cifs > server > > in a next version ;-) > > > > Thanks! > > > > -- > > Groeten, > > natxo > > > > _______________________________________________ > > OmniOS-discuss mailing list > > OmniOS-discuss at lists.omniti.com > > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > > -- -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From natxo.asenjo at gmail.com Wed Oct 12 17:14:22 2016 From: natxo.asenjo at gmail.com (Natxo Asenjo) Date: Wed, 12 Oct 2016 19:14:22 +0200 Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup tool In-Reply-To: References: Message-ID: hi, > > On Wed, Oct 12, 2016 at 6:07 PM, Gordon Ross > wrote: > >> Interesting finding re. VHD support needing resiliency support in SMB 2.1. >> Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193 >> >> We have resiliency support in NexentaStor 5.0 (shameless plug:) >> https://nexenta.com/products/nexentastor >> We should work on upstreaming that code soon. >> > That would be terrific! Thanks for your efforts for making an awesome cifs service. -- Groeten, natxo -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom-omnios-discuss at tom.bn-ulm.de Wed Oct 12 17:49:16 2016 From: tom-omnios-discuss at tom.bn-ulm.de (Thomas Wagner) Date: Wed, 12 Oct 2016 19:49:16 +0200 Subject: [OmniOS-discuss] Slow ssh transfers In-Reply-To: <409c3004-6182-5285-a02c-de532002fca9@gmx.com> References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com> <398961A7-5F81-4731-87B9-B3A829946245@omniti.com> <1476230545.603.5.camel@gmail.com> <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com> <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com> <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com> <409c3004-6182-5285-a02c-de532002fca9@gmx.com> Message-ID: <20161012174916.GI25497@wagner-net.com> The SSH Daemon is in most cases the limiting party. The ssh client is allowed to support much more older and weaker algorithms and protocols. Therefore I would compare the Ciphers and macs the Server supports. If you run the ssh client with "ssh -vvv" then you see what the SSH server offers and what the client can do. Preferences can be tweaked from the server side if the order inside the Chipers / KexAlgorithms / macs are changed. The same is the case on the client side if e.g. the allowed algorithms are locked down to a secure and fast enough Cipher / mac. I've seen on older CPUs that they limit transfer speed early if newer algorithms are used. The hardware accelleration does not help in all cases, as they can't support all algorithms with accelerated subtasks. Are junks very small? Then HW-accelleration is usually killed by the overhead. So I would experiment with the algorithms / macs and try finding one which is accellerated on both sides. Thomas > > > > Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and > > OmniOS boxes? > > > > /dale > > > > Hi > > Seems to be the same > > Omnios box - ssh -Q cipher > 3des-cbc > blowfish-cbc > cast128-cbc > arcfour > arcfour128 > arcfour256 > aes128-cbc > aes192-cbc > aes256-cbc > rijndael-cbc at lysator.liu.se > aes128-ctr > aes192-ctr > aes256-ctr > aes128-gcm at openssh.com > aes256-gcm at openssh.com > chacha20-poly1305 at openssh.com > > > FreeBSD box - ssh -Q cipher: > 3des-cbc > blowfish-cbc > cast128-cbc > arcfour > arcfour128 > arcfour256 > aes128-cbc > aes192-cbc > aes256-cbc > rijndael-cbc at lysator.liu.se > aes128-ctr > aes192-ctr > aes256-ctr > aes128-gcm at openssh.com > aes256-gcm at openssh.com > chacha20-poly1305 at openssh.com > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > -- -- Thomas Wagner ------------------------------------------------------------------------ Service rund um UNIX(TM), Wagner Network Services, Thomas Wagner Solaris(TM), Linux(TM) Eschenweg 21, 89174 Altheim, Germany Windows(TM) TEL: +49-731-9807799, FAX: +49-731-9807711 Telekommunikation, LAN, MOBILE/CELL: +49-171-6135989 Internet-Service, Elektronik EMAIL: wagner at wagner-net.com From alka at hfg-gmuend.de Wed Oct 12 18:40:58 2016 From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=) Date: Wed, 12 Oct 2016 20:40:58 +0200 Subject: [OmniOS-discuss] LX zones and a small Howto In-Reply-To: <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> Message-ID: <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> Gratulation! I have also tried the CentOS example and it works for me. I have created a small howto setup LX and the VM, see http://www.napp-it.org/doc/downloads/zones.pdf I have not used SmartOS in the past so the concept was at first not clear to me. What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics. What I will add to the current concept in napp-it is deploying containers together with the config file (zone.cfg) like it is done on ESXi where you can simply provide a folder with the VM that you can simply import and run. An option is that one deploy LX images as a folder imagefile with the zone config file within and the container with the OS as a subfolder. Gea Am 12.10.2016 um 04:39 schrieb Dan McDonald: > The bloody repo server has an updated LX brand package with this fix. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > > On Oct 11, 2016, at 2:20 PM, Richard Skelton > wrote: > >> Hi All, >> Dan provided an updated lx_brand file which I installed in >> /usr/kernel/brand/amd64/lx_brand >> After a reboot of the global zone all is now working :-) >> >> CentOS release 6.8 (Final) >> Kernel 2.6.32 on an x86_64 >> >> lx0 login: >> >> >> >> Dan McDonald wrote: >>>> On Oct 11, 2016, at 12:52 PM, Dan McDonald wrote: >>>> >>>> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too. I suspect I mismerged something. It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days. >>>> >>> >>> Not mismerge, but missing bugfix: >>> >>> OS-5685 LTP vhangup02 fails and passes >>> >>> Could you do me a favor and test a replacement lx_brand binary? Please respond unicast, and we'll report back to the list once your post-fix findings match mine. :) >>> >>> Thanks, >>> Dan >>> >>> >>> > > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -- Guenther Alka, Dipl.-Ing. (FH) Leiter des Rechenzentrums head of computer center Tel 07171 602 627 Fax 07171 69259 guenther.alka at hfg-gmuend.de http://rz.hfg-gmuend.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Wed Oct 12 18:52:30 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 12 Oct 2016 14:52:30 -0400 Subject: [OmniOS-discuss] LX zones and a small Howto In-Reply-To: <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> Message-ID: > On Oct 12, 2016, at 2:40 PM, G?nther Alka wrote: > > What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics. Joyent did most of the hard work here with LX -- credit where due, please. :) By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example). Thanks for the napp-it update! Dan From alka at hfg-gmuend.de Wed Oct 12 20:10:55 2016 From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=) Date: Wed, 12 Oct 2016 22:10:55 +0200 Subject: [OmniOS-discuss] LX zones and a small Howto In-Reply-To: References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> Message-ID: Most important is the success of the Illumos or Open-ZFS platform. LX containers are from Joyent (btw have you read this article https://www.infoq.com/news/2016/03/containers-summit-nyc ), SMB2+ and sequential resilvering are or may come from Nexenta and native encryption from ZoL with many features or additions that strengthen the platform from others versus Oracle or Linux. ESXi, SmartOS or Proxmox are very good as a cloud or virtualisation platform where you can run or use ZFS but OmniOS is a perfect NAS/SAN filer or server OS with the option to run a Linux container. This is indeed new. Thanks to all Gea Am 12.10.2016 um 20:52 schrieb Dan McDonald: >> On Oct 12, 2016, at 2:40 PM, G?nther Alka wrote: >> >> What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics. > Joyent did most of the hard work here with LX -- credit where due, please. :) > > By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example). > > Thanks for the napp-it update! > Dan > -- Guenther Alka, Dipl.-Ing. (FH) Leiter des Rechenzentrums head of computer center Tel 07171 602 627 Fax 07171 69259 guenther.alka at hfg-gmuend.de http://rz.hfg-gmuend.de From jstockett at molalla.com Wed Oct 12 21:52:07 2016 From: jstockett at molalla.com (Jeff Stockett) Date: Wed, 12 Oct 2016 21:52:07 +0000 Subject: [OmniOS-discuss] problem with boot-archive after upgrade In-Reply-To: <333BA485-2796-4744-A22F-F30355EE4B36@omniti.com> References: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com> <333BA485-2796-4744-A22F-F30355EE4B36@omniti.com> Message-ID: <136C13E89D22BB468B2A7025993639736E12B5A2@EXMCCMB.molalla.com> > > I upgraded a system from 012 to 018 just now and everything went totally > smoothly ? system rebooted fine. However, the boot environment names > weren?t what I wanted and since beadm can?t rename the active one, I create > a new one, activated it, and rebooted again. On reboot this second (and > subsequent times), I?m getting this in /var/src/log/system-boot- > archive:default.log: > > > > [ Mar 20 14:25:01 Executing start method > > ("/lib/svc/method/boot-archive"). ] cannot find: /etc/cluster/nodeid: > > No such file or directory cannot find: /etc/devices/mdi_ib_cache: No > > such file or directory cannot find: /etc/devices/retire_store: No such > > file or directory [ Mar 20 14:25:50 Method "start" exited with status > > 0. ] [ May 18 05:20:09 Enabled. ] [ May 18 05:20:14 Executing start > > method ("/lib/svc/method/boot-archive"). ] [ May 18 05:21:15 Method or > > service exit timed out. Killing contract 16. ] [ May 18 05:21:15 > > Method "start" failed due to signal KILL. ] > > > > The system eventually comes up to a single user mode login, and if I login > as root, and enter ?svcadm clear system/boot-archive? and/or maybe just > wait a while longer, then logout the system eventually comes up and works > fine in multi-user mode, but I was wondering if there is an easy way to fix > whatever I?ve done to shoot myself in the foot here? I tried ?bootadm > update-archive? but that doesn?t appear to fix it. > > When you manually ran 'bootadm update-archive' did it give you any output > such as "updating..." , or did it silently exit? > > If the latter, do something like 'touch /etc/system' first, then run it, as > bootadm will regenerate the boot archive only if it detects any changes in the > files its concerned with. And yes, I've seen this same problem you speak of > on rare occasion, with a rebuild of the boot archive being the cure. Thanks for your reply - here is what I get when I try your suggestion which looks promising - but when I reboot, still same problem. root at zfs01:/root# touch /etc/system; bootadm update-archive -v cannot find: /etc/cluster/nodeid: No such file or directory cannot find: /etc/devices/mdi_ib_cache: No such file or directory changed /etc/system mkisofs not found, creating UFS archive updating /platform/i86pc/amd64/boot_archive updating /platform/i86pc/boot_archive root at zfs01:/root# If you or anyone else has any other ideas, I'm all ears, otherwise I'll probably just live with it since it doesn't seem to cause problems other than slowing the initial boot. From ergi.thanasko at avsquad.com Thu Oct 13 23:13:59 2016 From: ergi.thanasko at avsquad.com (Ergi Thanasko) Date: Thu, 13 Oct 2016 23:13:59 +0000 Subject: [OmniOS-discuss] Network throughout 1GB/sec In-Reply-To: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com> <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de> Message-ID: Hi Gea, Your property values are a bit off, here is what i got from EMC https://community.emc.com/docs/DOC-39156 * ipadm set-prop ?p send_buf =2097152 tcp * ipadm set-prop ?p recv_buf=2097152 tcp * ipadm set-prop ?p max_buf=16777216 tcp * ipadm set-prop ?p _cwnd_max=8388608 tcp also the power of 2 in twenty is 1048576. Can you confirm that the set-prop -p values should the 2 in the power of 20 and 21st [cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home] On Sep 16, 2016, at 12:57 PM, Guenther Alka > wrote: I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex max_buf=4097152 tcp send_buf=2048576 tcp recv_buf=2048576 tcp, NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576) http://napp-it.org/doc/downloads/performance_smb2.pdf Gea Am 16.09.2016 um 19:43 schrieb Ergi Thanasko: Hi all, We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about 600-800MB/sec between those two servers. Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving and we prefer something in the nature of rsync but to utilize the network hardware. If Anyone has a hardware setup that can see 1GB/sec throughput and does not mind sharing? Any software that use multithreads sessions to move data around zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work. Thank you for your time _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ErgiThanasko.png Type: image/png Size: 7910 bytes Desc: ErgiThanasko.png URL: From terminator2202 at web.de Fri Oct 14 10:37:49 2016 From: terminator2202 at web.de (terminator2202 at web.de) Date: Fri, 14 Oct 2016 12:37:49 +0200 Subject: [OmniOS-discuss] vmware and omnios problem with passthru Message-ID: An HTML attachment was scrubbed... URL: From daleg at omniti.com Fri Oct 14 14:30:05 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 14 Oct 2016 10:30:05 -0400 Subject: [OmniOS-discuss] vmware and omnios problem with passthru In-Reply-To: References: Message-ID: Interesting that it seems to happen upon making the ixgbe driver do work. However, vmware logs don't do us much good here as they don't actually give us any valuable information. Would would need to see the crash info from OmniOS and a the crash dump. /dale > On Oct 14, 2016, at 6:37 AM, terminator2202 at web.de wrote: > > Hello, > i have some problems with omnios and VMware 6U2. I am not sure if it is VMware related or omnios related. > > First, my Hardware: > X10SRL-F with XEON CPU > 64GB ECC RAM > 2x M1015 SAS Controller with IT Mode (Firmware P19) > 1x Intel 540x-T1 > and one or two other PCIe Devices > > I have some issues with Passthru some PCIe devices. > i create a VM with the newest omnios_bloody (bloody_20161006.iso). > After that i passthru the 2x M1015 and import the exisiting pool -> no problem -> run's stable. > (see log WITHOUT problems/crash: http://pastebin.com/Y0G5YHEg ) > > After that i tried to passthru my intel x540-T1 NIC (and later a i210-NIC onboard-Port). About 10 seconds after omibos boot the vm just crashes. > Sometimes it is stable for a bit longer and i can login and execute dladm show-phys or show-link but after that i crashes instantly. > (see log WITH problems/crash: http://pastebin.com/gPNLrdg7 ) > > I also tried to install omnios without vmware (native) and i have no problems at all. Omnios regocnize all discs on M1015 and x540 NIC and i210 NIC. > I guess the issues are msi-x/interups related, but maybe you can help with that. > > Thanks, manuel > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From geoffn at gnaa.net Fri Oct 14 19:52:00 2016 From: geoffn at gnaa.net (Geoff Nordli) Date: Fri, 14 Oct 2016 12:52:00 -0700 Subject: [OmniOS-discuss] upgrade from r151008 In-Reply-To: References: Message-ID: <0585465a-1242-a904-6123-a1edacdf6633@gnaa.net> On 2016-10-05 04:32 PM, Dan McDonald wrote: >> On Oct 5, 2016, at 7:03 PM, Geoff Nordli wrote: >> >> Always a risk when you wait too long between versions. >> >> > We tested 006 --> 014 pre-release of 014. 008 --> 014 should be equally workable. > > Dan > Upgrade worked perfectly. 006 --> 018. Nothing fancy on it though, just a basic install as a storage server. Happy Friday!! Geoff From daleg at omniti.com Fri Oct 14 19:56:01 2016 From: daleg at omniti.com (Dale Ghent) Date: Fri, 14 Oct 2016 15:56:01 -0400 Subject: [OmniOS-discuss] upgrade from r151008 In-Reply-To: <0585465a-1242-a904-6123-a1edacdf6633@gnaa.net> References: <0585465a-1242-a904-6123-a1edacdf6633@gnaa.net> Message-ID: <57213295-D6C4-46F1-AB0C-8C2F090FE1C4@omniti.com> > On Oct 14, 2016, at 3:52 PM, Geoff Nordli wrote: > > > On 2016-10-05 04:32 PM, Dan McDonald wrote: >>> On Oct 5, 2016, at 7:03 PM, Geoff Nordli wrote: >>> >>> Always a risk when you wait too long between versions. >>> >>> >> We tested 006 --> 014 pre-release of 014. 008 --> 014 should be equally workable. >> >> Dan >> > > > Upgrade worked perfectly. 006 --> 018. > > Nothing fancy on it though, just a basic install as a storage server. > Remember to upgrade your zpools. Also, 020 is coming soon enough :) /dale From terminator2202 at web.de Sat Oct 15 14:31:23 2016 From: terminator2202 at web.de (terminator2202 at web.de) Date: Sat, 15 Oct 2016 16:31:23 +0200 Subject: [OmniOS-discuss] vmware and omnios problem with passthru In-Reply-To: References: , Message-ID: An HTML attachment was scrubbed... URL: From danmcd at omniti.com Tue Oct 18 17:40:54 2016 From: danmcd at omniti.com (Dan McDonald) Date: Tue, 18 Oct 2016 13:40:54 -0400 Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior Message-ID: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> VIM8 is in the omnios-build master branch for next-bloody and r151020. VIM8's defaults are more VIM-like and less classic-VI like. Would it kill people if I made "vi" act like "vim -C"? Or would that be MORE surprising? Curious, Dan From gate03 at landcroft.co.uk Tue Oct 18 21:27:22 2016 From: gate03 at landcroft.co.uk (Michael Mounteney) Date: Wed, 19 Oct 2016 07:27:22 +1000 Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> Message-ID: <20161019072722.1b979323@punda-mlia> On Tue, 18 Oct 2016 13:40:54 -0400 Dan McDonald wrote: > VIM8 is in the omnios-build master branch for next-bloody and r151020. > > VIM8's defaults are more VIM-like and less classic-VI like. > > Would it kill people if I made "vi" act like "vim -C"? Or would that > be MORE surprising? No strong opinion here. I use vim on Linux so am used to it, and am slightly caught off guard by the differences of vi on OmniOS. ______________ Michael Mounteney From bfriesen at simple.dallas.tx.us Wed Oct 19 02:45:11 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Tue, 18 Oct 2016 21:45:11 -0500 (CDT) Subject: [OmniOS-discuss] Firewall in OmniOS? Message-ID: While I do have a dedicated low-power box I can use, it came to mind that I could use OmniOS on a more powerful Internet-facing server and use it as the firewall for my network (screening and session-oriented NAT). Assuming that the Internet-facing interface is sufficiently battened down, this should provide optimum performance for other software running on the same system while providing good security. I know that ipfilter is available in OmniOS but have heard that it is a bit dated when it comes to NAT. Has anyone tried (or is it feasible) to install OpenBSD or FreeBSD (perhaps pfSense or OPNsense) in a KVM instance (running in a zone) to provide an effective firewall for LANs? Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From johan.kragsterman at capvert.se Wed Oct 19 07:33:31 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Wed, 19 Oct 2016 09:33:31 +0200 Subject: [OmniOS-discuss] Ang: Firewall in OmniOS? In-Reply-To: References: Message-ID: Hi, Bob! -----"OmniOS-discuss" skrev: ----- Till: omnios-discuss at lists.omniti.com Fr?n: Bob Friesenhahn S?nt av: "OmniOS-discuss" Datum: 2016-10-19 02:47 ?rende: [OmniOS-discuss] Firewall in OmniOS? While I do have a dedicated low-power box I can use, it came to mind that I could use OmniOS on a more powerful Internet-facing server and use it as the firewall for my network (screening and session-oriented NAT). ?Assuming that the Internet-facing interface is sufficiently battened down, this should provide optimum performance for other software running on the same system while providing good security. I know that ipfilter is available in OmniOS but have heard that it is a bit dated when it comes to NAT. Has anyone tried (or is it feasible) to install OpenBSD or FreeBSD (perhaps pfSense or OPNsense) in a KVM instance (running in a zone) to provide an effective firewall for LANs? Yeah, I do that since several yrs with PfSense. I use it a lot, it is of great help, since it is so easy to handle and has a lot of great features. I think I have 5 virtual interfaces on it, both strictly internal and the ones going out of the box. You can say that whatever I handle in my infrastructure, PfSense is involved. I like fixed dhcp addressing, so most interfaces provides that. Rgrds Johan Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, ? ?http://www.GraphicsMagick.org/ _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From omnios at citrus-it.net Wed Oct 19 12:10:41 2016 From: omnios at citrus-it.net (Andy Fiddaman) Date: Wed, 19 Oct 2016 12:10:41 +0000 (UTC) Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> Message-ID: On Tue, 18 Oct 2016, Dan McDonald wrote: ; VIM8 is in the omnios-build master branch for next-bloody and r151020. ; ; VIM8's defaults are more VIM-like and less classic-VI like. ; ; Would it kill people if I made "vi" act like "vim -C"? Or would that be MORE surprising? No preference here. We replace /usr/share/vim/vimrc with our own version anyway. Andy -- Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ Registered in England and Wales | Company number 4899123 From danmcd at omniti.com Wed Oct 19 15:58:25 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 19 Oct 2016 11:58:25 -0400 Subject: [OmniOS-discuss] Firewall in OmniOS? In-Reply-To: References: Message-ID: I still use ipfilter, but only as a NAT. Porting one of the others to OmniOS/illumos would require a lot of work. (Hell, part of me thinks writing a NAT-only hooks module would be easier.) My $.02, Dan From gary at genashor.com Wed Oct 19 16:17:44 2016 From: gary at genashor.com (Gary Gendel) Date: Wed, 19 Oct 2016 12:17:44 -0400 Subject: [OmniOS-discuss] Firewall in OmniOS? In-Reply-To: References: Message-ID: I used OpenIndiana and OmniOS with ipfilter in a router configuration for years without any serious issues. I was also providing DNS, DHCP and IPv6 over v4. However, one of the drawbacks of this is that, if you use this as your WAN/LAN router, you bring your whole network down with the occasional reboot. I ended up picking up a MicroTik RB750 for about $50. This is no longer made but the "hEX" replacement looks sweet. These have Cisco-level functionality, a tiny footprint, and low power consumption. I've never looked back. I'm running a split-horizon DNS with hairpin routing rules so the internal network machines get the internal ip addresses for the services. It reboots in a few seconds so there is minimal downtime when I upgrade the software. Adding my $.02 too. Gary On 10/19/2016 11:58 AM, Dan McDonald wrote: > I still use ipfilter, but only as a NAT. > > Porting one of the others to OmniOS/illumos would require a lot of work. (Hell, part of me thinks writing a NAT-only hooks module would be easier.) > > My $.02, > Dan > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3703 bytes Desc: S/MIME Cryptographic Signature URL: From danmcd at omniti.com Wed Oct 19 18:11:15 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 19 Oct 2016 14:11:15 -0400 Subject: [OmniOS-discuss] OpenSSH Update Message-ID: <2E60B827-AF11-4DD1-958E-4C710E26BD5C@omniti.com> There's a small memory leak in OpenSSH that, if properly attacked, can cause OpenSSH to eat a bunch of memory. If you're using OpenSSH on supported releases (014, 016, 018), please utter "pkg update" now. Your OpenSSH service will restart automatically with the fixed sshd after updating is completed. Happy patching! Dan From skiselkov.ml at gmail.com Wed Oct 19 20:55:07 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Wed, 19 Oct 2016 22:55:07 +0200 Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots Message-ID: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> So I've been trying to get illumos-gate built on a fresh r151014 install in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and console redirected to serial. Other than that, no tuning. The problem is, when I start the gate build process, randomly after about 15-30 minutes of building, the machine just spontaneously reboots. No panic, no log message in libvirt, nothing (even when kmdb was loaded). Out of nothing, the VM just hits a reset and reappears in the bootloader. Has anybody else seen this? Cheers, -- Saso From danmcd at omniti.com Wed Oct 19 21:02:21 2016 From: danmcd at omniti.com (Dan McDonald) Date: Wed, 19 Oct 2016 17:02:21 -0400 Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots In-Reply-To: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> Message-ID: Not sure how hard it'd be, but can you see if 018 has similar problems? Dan Sent from my iPhone (typos, autocorrect, and all) > On Oct 19, 2016, at 4:55 PM, Saso Kiselkov wrote: > > So I've been trying to get illumos-gate built on a fresh r151014 install > in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard > qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and > console redirected to serial. Other than that, no tuning. The problem > is, when I start the gate build process, randomly after about 15-30 > minutes of building, the machine just spontaneously reboots. No panic, > no log message in libvirt, nothing (even when kmdb was loaded). Out of > nothing, the VM just hits a reset and reappears in the bootloader. > > Has anybody else seen this? > > Cheers, > -- > Saso > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss From skiselkov.ml at gmail.com Wed Oct 19 21:07:36 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Wed, 19 Oct 2016 23:07:36 +0200 Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots In-Reply-To: References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> Message-ID: On 10/19/16 11:02 PM, Dan McDonald wrote: > Not sure how hard it'd be, but can you see if 018 has similar problems? I'm gonna try to give it a go. -- Saso >> On Oct 19, 2016, at 4:55 PM, Saso Kiselkov wrote: >> >> So I've been trying to get illumos-gate built on a fresh r151014 install >> in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard >> qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and >> console redirected to serial. Other than that, no tuning. The problem >> is, when I start the gate build process, randomly after about 15-30 >> minutes of building, the machine just spontaneously reboots. No panic, >> no log message in libvirt, nothing (even when kmdb was loaded). Out of >> nothing, the VM just hits a reset and reappears in the bootloader. >> >> Has anybody else seen this? >> >> Cheers, >> -- >> Saso >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From david.ledger at ivdcs.co.uk Wed Oct 19 21:14:11 2016 From: david.ledger at ivdcs.co.uk (David Ledger) Date: Wed, 19 Oct 2016 22:14:11 +0100 Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> Message-ID: <9B403F44-2C9C-413A-AF10-B9B2EF071231@ivdcs.co.uk> > On 18 Oct 2016, at 18:40, Dan McDonald wrote: > > VIM8 is in the omnios-build master branch for next-bloody and r151020. > > VIM8's defaults are more VIM-like and less classic-VI like. > > Would it kill people if I made "vi" act like "vim -C"? Or would that be MORE surprising? > > Curious, > Dan As long as I can configure things so that I have a vi that works like that in Bsd 4.3 it?s ok. The thing I fall over most often is the simple ^U^U to scroll back to the last change. It?s muscle memory and loses me two changes on some versions. David - Unix since ?83 From skiselkov.ml at gmail.com Wed Oct 19 22:00:11 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Thu, 20 Oct 2016 00:00:11 +0200 Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots In-Reply-To: References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> Message-ID: <065cd587-8ec6-f4ea-b282-6d277fbfa94c@gmail.com> On 10/19/16 11:02 PM, Dan McDonald wrote: > Not sure how hard it'd be, but can you see if 018 has similar problems? Tried it, still happens. After about 10 minutes of building, VM spontaneously reboots. -- Saso >> On Oct 19, 2016, at 4:55 PM, Saso Kiselkov wrote: >> >> So I've been trying to get illumos-gate built on a fresh r151014 install >> in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard >> qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and >> console redirected to serial. Other than that, no tuning. The problem >> is, when I start the gate build process, randomly after about 15-30 >> minutes of building, the machine just spontaneously reboots. No panic, >> no log message in libvirt, nothing (even when kmdb was loaded). Out of >> nothing, the VM just hits a reset and reappears in the bootloader. >> >> Has anybody else seen this? >> >> Cheers, >> -- >> Saso >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss From danmcd at omniti.com Thu Oct 20 04:15:51 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 20 Oct 2016 00:15:51 -0400 Subject: [OmniOS-discuss] Likely last bloody update before r151020 Message-ID: I hope this is the bloody that will form the r151020 branch. Please note that because of the sheer volume of upstream illumos-gate activity, we froze pulls from -gate early this time, early enough to not include BSD Loader and Python 2.7 support. A few cherrypicks may happen pre-020, and a couple have already happened: - illumos 4498, update ACPI to version 6.x - A series of nvme and blkdev fixes culminating in illumos 7382, basic NVMe 1.1 support. uname -v says omnios-master-2f9273c, corresponding to the illumos-omnios:master commit, and omnios-build is at commit 8d79e11. This is likely to be where we branch r151020. Staying with Python2.6 means that if you wish to compile illumos-gate on ANY OmniOS, you need to add these two lines to your .env file: +# Starting with illumos bug 5969, upstream defaults to Python2.7. We still +# use Python2.6 as of r151020 +export PYTHON_VERSION="2.6" +export PYTHON_PKGVERS="-26" + The shipping "/opt/onbld/env/omnios-illumos-gate" file has these lines in it now. The rest of the software in omnios-build has been updated as well. Including: * GNU grep is now 2.26 * GNU tar is now 1.29 * gzip is now 1.8 * iperf is now 3.1.3 * isc-dhcp is now 4.3.5 * iso-codes are now 3.70 * libidn is now 1.33 * libxslt is now 1.1.29 * mercurial is now 3.9.2 * mozilla-nss-nspr is up to 3.25 and 4.12, respectively * nghttp2 is new, at version 1.15.0 * p7zip is now at 16.02. * pciutils are now at 3.5.2 * pcre is now at 8.39 * python26-coverage is now at 4.2 * python26-lxml is now at 3.6.4 * python26-numpy is now at 1.11.2 * python26-ply is now at 3.9 * python26-pylint is now at 1.6.4 * readline is now at 7.0, but we will ship the 6.3 libraries from r151018 to maintain linked-application compatibility * screen is now at 4.4.0 * sigcpp is now at 2.10.0 * sqlite3 is now at 3.15.0 * tmux is now at 2.3 * vim is now at 8.0 (and "vi" is, for now, going to behave as if you invoked vim, because of how 8.0 works - see the other thread on omnios-discuss about this) This is an repo-only update. Happy updating! Dan From Josh.Barton at usurf.usu.edu Thu Oct 20 13:30:23 2016 From: Josh.Barton at usurf.usu.edu (Josh Barton) Date: Thu, 20 Oct 2016 13:30:23 +0000 Subject: [OmniOS-discuss] IPS and Updating Message-ID: I have set up an IPS repository for all the software we have compiled and added it as a publisher on one of or boxes. As it turns out, running "pkg update" fails because OmniOs tried to download he updates from my personal repository rather than directly from Omniti. Is there a work around for this? Is it possible that I added my custom publisher incorrectly? Josh Barton USU Research Foundation 1695 Research Park Way North Logan UT, 84321 Office: (435) 713-3089 Cell: (435) 770-4179 From danmcd at omniti.com Thu Oct 20 13:36:36 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 20 Oct 2016 09:36:36 -0400 Subject: [OmniOS-discuss] IPS and Updating In-Reply-To: References: Message-ID: > On Oct 20, 2016, at 9:30 AM, Josh Barton wrote: > > I have set up an IPS repository for all the software we have compiled and added it as a publisher on one of or boxes. As it turns out, running "pkg update" fails because OmniOs tried to download he updates from my personal repository rather than directly from > Omniti. Is there a work around for this? Is it possible that I added my custom publisher incorrectly? You need to share more specific information for folks to help. Please share the output of: pkg publisher and pkg update -v So people can see what package update triggered the failure. There are a lot of ways things can go wrong, so we need to narrow things down. Dan From skiselkov.ml at gmail.com Thu Oct 20 14:47:12 2016 From: skiselkov.ml at gmail.com (Saso Kiselkov) Date: Thu, 20 Oct 2016 16:47:12 +0200 Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots In-Reply-To: References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com> Message-ID: <8dca4287-62b7-f09c-466a-0b440e623a3b@gmail.com> On 10/19/16 11:02 PM, Dan McDonald wrote: > Not sure how hard it'd be, but can you see if 018 has similar problems? Managed to get this resolved by upping amount of vram. My suspicion was roused because I saw a shell command return with "fork: not enough space". After I went from 2G to 4G of vram and swap from 1g to 2g, full build /w lint went through without a hitch, twice. Maybe just increasing swap would have been enough, at the expense of the speed of the build. Cheers, -- Saso From tim at multitalents.net Fri Oct 21 03:19:24 2016 From: tim at multitalents.net (Tim Rice) Date: Thu, 20 Oct 2016 20:19:24 -0700 (PDT) Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com> Message-ID: Hi Dan, On Tue, 18 Oct 2016, Dan McDonald wrote: > VIM8 is in the omnios-build master branch for next-bloody and r151020. > > VIM8's defaults are more VIM-like and less classic-VI like. > > Would it kill people if I made "vi" act like "vim -C"? Or would that be MORE surprising? On the contrary, having "vi" act like vi would be great. On Linux distros I always have to patch the system vimrc so I don't get annoyed all the time. If someone wants vim "features" they can invoke vim. > Curious, > Dan > -- Tim Rice Multitalents tim at multitalents.net From alka at hfg-gmuend.de Fri Oct 21 15:04:35 2016 From: alka at hfg-gmuend.de (Guenther Alka) Date: Fri, 21 Oct 2016 17:04:35 +0200 Subject: [OmniOS-discuss] LX zones and a small Howto In-Reply-To: References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> Message-ID: I have added a menu in napp-it to select, download and preconfigure LX images from Joyent beside the menu to configure/ attach/ start/ stop/ detach/ delete http://www.napp-it.org/doc/downloads/zones.pdf Whats not yet clear to me is the way to configure lx zones with zoneadm/zonecfg according to the settings in the Joyent XML file like network ip/dhcp, router, netmask, gateway, maxram, ssh, mount zfs filesystems to name a few. btw what about kvm branded zones from Joyent? the concept is very new to me.... you may discuss at https://forums.servethehome.com/index.php?threads/omnios-now-includes-lx-support-from-joyent-smartos.11577/ >> On Oct 12, 2016, at 2:40 PM, G?nther Alka wrote: >> >> What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics. > Joyent did most of the hard work here with LX -- credit where due, please. :) > > By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example). > > Thanks for the napp-it update! > Dan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Fri Oct 21 17:20:52 2016 From: danmcd at omniti.com (Dan McDonald) Date: Fri, 21 Oct 2016 13:20:52 -0400 Subject: [OmniOS-discuss] LX zones and a small Howto In-Reply-To: References: <57FC9913.3080609@btconnect.com> <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com> <57FD12F7.1010704@btconnect.com> <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com> <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com> <57FD2D7A.1020308@btconnect.com> <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com> <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de> Message-ID: <12F5F8D5-9676-4B63-B9A8-2B896EFEEE1F@omniti.com> > On Oct 21, 2016, at 11:04 AM, Guenther Alka wrote: > > I have added a menu in napp-it to select, download and preconfigure LX images from Joyent > beside the menu to configure/ attach/ start/ stop/ detach/ delete > > http://www.napp-it.org/doc/downloads/zones.pdf > > Whats not yet clear to me is the way to configure lx zones with zoneadm/zonecfg according to the settings in the Joyent XML file like network ip/dhcp, router, netmask, gateway, maxram, ssh, mount zfs filesystems to name a few. We didn't port all of SmartOS, we ported the LX zones. For now, there are a few specific zonecfg(1M) parameters for kernel-version and network setting. We hope to have /native/sbin/ipadm persistence before r151020 ships. Otherwise, you treat an LX zone like you would any other, using zonecfg(1M) where appropriate. > btw > what about kvm branded zones from Joyent? > the concept is very new to me.... A KVM zone instantiates a zone, but instead of running a child init(1M) process, it runs KVM directly. It wraps a KVM process in it's own zone, whose sole purpose is to service KVM. We don't have these in OmniOS. Like I said, we weren't porting all of SmartOS. > you may discuss at > https://forums.servethehome.com/index.php?threads/omnios-now-includes-lx-support-from-joyent-smartos.11577/ I'm trying to get r151020 out the door, so I can't for the time being. Dan From johan.kragsterman at capvert.se Sat Oct 22 11:51:47 2016 From: johan.kragsterman at capvert.se (Johan Kragsterman) Date: Sat, 22 Oct 2016 13:51:47 +0200 Subject: [OmniOS-discuss] pass H/W device to LX zone Message-ID: Hi! I wonder if it is, or will be, possible to pass a hardware device(pci, usb, whatever) that has no omnios driver support, to an LX zone, where it has driver support? Best regards from/Med v?nliga h?lsningar fr?n Johan Kragsterman Capvert From jimklimov at cos.ru Sat Oct 22 14:31:21 2016 From: jimklimov at cos.ru (Jim Klimov) Date: Sat, 22 Oct 2016 16:31:21 +0200 Subject: [OmniOS-discuss] pass H/W device to LX zone In-Reply-To: References: Message-ID: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru> 22 ??????? 2016??. 13:51:47 CEST, Johan Kragsterman ?????: > >Hi! > > >I wonder if it is, or will be, possible to pass a hardware device(pci, >usb, whatever) that has no omnios driver support, to an LX zone, where >it has driver support? > > >Best regards from/Med v?nliga h?lsningar fr?n > >Johan Kragsterman > >Capvert > > >_______________________________________________ >OmniOS-discuss mailing list >OmniOS-discuss at lists.omniti.com >http://lists.omniti.com/mailman/listinfo/omnios-discuss Your question actually provokes a larger one: will it ever be possible to run a linux HW driver in an LX zone? @Johan: Initially I wanted to bluntly say "of course not" because an LX zone has the illumos kernel and a layer to emulate some syscalls and translate them from linux to solaris - which is ok for some userland software but likely not for linux kernel modules (which is what drivers are, mostly). But then again, what if some drivers use this set of syscalls (or some others not difficult to provide mappings for) and might just run? Jim -- Typos courtesy of K-9 Mail on my Samsung Android From danmcd at omniti.com Sat Oct 22 18:07:06 2016 From: danmcd at omniti.com (Dan McDonald) Date: Sat, 22 Oct 2016 14:07:06 -0400 Subject: [OmniOS-discuss] pass H/W device to LX zone In-Reply-To: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru> References: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru> Message-ID: > On Oct 22, 2016, at 10:31 AM, Jim Klimov wrote: > > > @Johan: Initially I wanted to bluntly say "of course not" because an LX zone has the illumos kernel and a layer to emulate some syscalls and translate them from linux to solaris - which is ok for some userland software but likely not for linux kernel modules (which is what drivers are, mostly). But then again, what if some drivers use this set of syscalls (or some others not difficult to provide mappings for) and might just run? Your blunt answer is generally right. LX zones have the illumos kernel underneath. Unless there's some amazing work going on in the in-kernel-emulation I'm not aware of, you won't be able to run Linux device drivers ever in an LX Zone. There's another part to this tale as well, but I will not commit it to email, especially an archived list. Let's just say it involves GPL and leave it at that. :-P Dan From bfriesen at simple.dallas.tx.us Sat Oct 22 19:33:28 2016 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Sat, 22 Oct 2016 14:33:28 -0500 (CDT) Subject: [OmniOS-discuss] pass H/W device to LX zone In-Reply-To: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru> References: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru> Message-ID: On Sat, 22 Oct 2016, Jim Klimov wrote: > 22 ??????? 2016??. 13:51:47 CEST, Johan Kragsterman ?????: >> >> I wonder if it is, or will be, possible to pass a hardware device(pci, >> usb, whatever) that has no omnios driver support, to an LX zone, where >> it has driver support? In my experience the Linux kernel interface to its drivers is quite unstable and driver code requires tweaks to work with the updated kernel. There is a tight coupling between drivers and the kernel so that they have to be compiled with the kernel, or re-compiled based on specific kernel headers. The driver interfaces are not mostly stable like they are with Solaris/Illumos. The reason why Linux works with so many devices is due to the many interested parties who work to keep it working. If someone was to compile the driver from source code they could use it with the Illumos kernel on their own system without violating GPL. Any small error might result in the OS needing to be re-installed and/or total loss of data. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From info.fmo at gmx.de Sun Oct 23 01:06:28 2016 From: info.fmo at gmx.de (Frank M.) Date: Sun, 23 Oct 2016 03:06:28 +0200 Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes In-Reply-To: <1344112614.20161023025416@gmx.de> References: <1344112614.20161023025416@gmx.de> Message-ID: <1878224113.20161023030628@gmx.de> Perhaps my post is here better, because I use OmniOS of course... ...and the next stable OmniOS is in sight... This is a forwarded message From: Frank M. To: Illumos Discussion Date: Sunday, October 23, 2016, 2:54:16 AM Subject: NVMe in ESXi-Passthrough crashes ===8<==============Original message text=============== Hi, I?ve installed the new bloody as a new test virtual machine. The vm crashes whenever there is a access to the nvme device. Tests with a "special nvme driver" on the last stable had no problems. I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+. Tomorrow I will test shortly a Samsung SM961 on another Server (HP). I hope, you will find a solution until the next stable release... greets from Germany Frank ===8<===========End of original message text=========== -- Best regards, Frank mailto:info.fmo at gmx.de From danmcd at omniti.com Sun Oct 23 01:11:09 2016 From: danmcd at omniti.com (Dan McDonald) Date: Sat, 22 Oct 2016 21:11:09 -0400 Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes In-Reply-To: <1878224113.20161023030628@gmx.de> References: <1344112614.20161023025416@gmx.de> <1878224113.20161023030628@gmx.de> Message-ID: > On Oct 22, 2016, at 9:06 PM, Frank M. wrote: > > > I?ve installed the new bloody as a new test virtual machine. The vm > crashes whenever there is a access to the nvme device. > Tests with a "special nvme driver" on the last stable had no problems. > I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+. > Tomorrow I will test shortly a Samsung SM961 on another Server (HP). > I hope, you will find a solution until the next stable release... Oddly enough: 1.) r151020 (next stable) is coming soon (November). 2.) Due to some interesting circumstances, the NVMe fixes in bloody & 020 may be backported to LTS (014) and last-stable (018). Dan From info.fmo at gmx.de Sun Oct 23 14:09:05 2016 From: info.fmo at gmx.de (Frank M.) Date: Sun, 23 Oct 2016 16:09:05 +0200 Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes In-Reply-To: References: <1344112614.20161023025416@gmx.de> <1878224113.20161023030628@gmx.de> Message-ID: <1987622123.20161023160905@gmx.de> Hi, I?ve tested now on another machine (HP DL380 G6). The new bloody OmniOS crashes here too... Frank Sunday, October 23, 2016, 3:11:09 AM, you wrote: >> On Oct 22, 2016, at 9:06 PM, Frank M. wrote: >> >> >> I?ve installed the new bloody as a new test virtual machine. The vm >> crashes whenever there is a access to the nvme device. >> Tests with a "special nvme driver" on the last stable had no problems. >> I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+. >> Tomorrow I will test shortly a Samsung SM961 on another Server (HP). >> I hope, you will find a solution until the next stable release... DM> Oddly enough: DM> 1.) r151020 (next stable) is coming soon (November). DM> 2.) Due to some interesting circumstances, the NVMe fixes in DM> bloody & 020 may be backported to LTS (014) and last-stable (018). DM> Dan From an3s.annema at gmail.com Mon Oct 24 19:04:07 2016 From: an3s.annema at gmail.com (Andries Annema) Date: Mon, 24 Oct 2016 21:04:07 +0200 Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS Message-ID: Hello there, guys, THE PROBLEM IN SHORT or TL;DR: I have installed the tun/tap device within the global zone of a r151014 machine, but the /dev/tun and /dev/tap devices don't show up. What am I missing here? THE PROBLEM IN MORE DETAIL: I'm trying to get OpenVPN up and running as a server from within a non-global zone. For this the TUN/TAP device driver is needed within the global zone and passed on to the non-global zone. Seems straight forward, and I actually did succeed on one system, but my main production server is playing tricks on me. As I'm not running just one OmniOS server anymore, it seemed obvious that I needed my own package repository and not build software from source on each production server. So, this is what I did - at first, all of it on a virtual OmniOS r151014 machine: 1. Set up a local IPS package repository within another NGZ, 2. set up a build environment within even another NGZ, 3. within the build environment, I downloaded the TUN/TAP source code from https://github.com/kaizawa/tuntap 4. configure --prefix=/proto/install/path/, make, make install 5. Created a manifest, specify package contents, etc etc, "pkglint" to verify before publishing, 6. and published the package to my local repo. 7. Then from the global zone on this VM, with "pkg set-publisher" I added this repo, 8. and installed my brand new tuntap-package with "pkg install network/tuntap". 9. to verify "ls -l /dev/tun /dev/tap" and indeed the devices showed up. And not just these links, but also the actual devices listed under "/devices/pseudo/clone at 0:[tun|tap]". Awesome! My very first built and published package! Now, to get more serious, I repeated all this on my actual production server. But there it goes funny at step 9: the devices don't show up. The files that are included within the IPS package however, are indeed installed. No error/warning during install whatsoever. Reinstalled the tuntap a few times. No effect. Rebooted the system and reinstalled. Still no effect. The system hadn't been updated in over a year (431 days uptime...), so I figured maybe I'm missing a package or something. I uninstalled the tuntap-device, updated the system, tried again: still no luck. Fiddling around with "modinfo" and "modload" didn't help either: "modinfo" doesn't show up a tun or tap device and "modload" is unable to load it. Then I tried step 7-9 on a second and quite similarly build OmniOS r151014 system, a system that is much younger and had all its packages up to date to begin with (not that I'm sure that the cause of the problem lies in there, but it usually is a good/better starting point) and guess what? Two beautiful new devices are listed at /dev/tun and /dev/tap. Here, also "modinfo" shows nicely a tun and a tap device. So my questions are: A. Where do I have to look for feedback/logs on the installation procedure to get more insight, if any, on where it failed? (The whole logging on Unix/Linux and alikes is one of many subjects that I have to master sometime). B. How come it does install nicely on one system, but does not on another (seemingly identical or at least very similar) system? C. The most important question: how can I get these devices actually showing up? Any help would be greatly appreciated! Thanks. We only learn by making mistakes and screwing things up, right! ;) Regards, Andries From mailinglists at qutic.com Tue Oct 25 10:59:50 2016 From: mailinglists at qutic.com (qutic development) Date: Tue, 25 Oct 2016 12:59:50 +0200 Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS In-Reply-To: References: Message-ID: <86177927-04CE-43E4-89ED-A25458641F65@qutic.com> Hi Andries, > Am 24.10.2016 um 21:04 schrieb Andries Annema : > > I have installed the tun/tap device within the global zone of a r151014 machine, but the /dev/tun and /dev/tap devices don't show up. What am I missing here? I use the following build script to get a working module for r151014: https://github.com/jfqd/omnios-build/tree/151006/build/tuntap - Stefan From tom-omnios-discuss at tom.bn-ulm.de Tue Oct 25 12:41:25 2016 From: tom-omnios-discuss at tom.bn-ulm.de (Thomas Wagner) Date: Tue, 25 Oct 2016 14:41:25 +0200 Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS In-Reply-To: References: Message-ID: <20161025124125.GC15500@wagner-net.com> Hi Andries, does your IPS manifest have an "action" to add the driver entries "tun" and "tap"? If you scroll down here to the %action section: https://sourceforge.net/p/pkgbuild/code/HEAD/tree/spec-files-extra/trunk/SFEtun.spec This results if built with "pkgtool"/"pkbbuild" to an entry in the manifest file like this: driver name=tap driver name=tun A ready-made package is available here: http://sfe.opencsw.org/quickrepolinks (fully independent of OpenCSW project, but kindly hosted by them) Btw, it would be cool if you could share your build recipes and / or participate in one of the public repositories for build recipes. Regards, Thomas From dasaint.blair at gmail.com Tue Oct 25 19:11:13 2016 From: dasaint.blair at gmail.com (Matt Mabis) Date: Tue, 25 Oct 2016 13:11:13 -0600 Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf + NFS = NFS Disconnects in ESXI on DC Reboot) Message-ID: Hey All, Was wondering if you could help me out, using b13298F but i have seen this in other versions too where i joined the Omni to the Domain so i could use ACLs and UserID Mappings for SMB. however when i reboot one of the 2 DCs i have it looks like it loses all access to the NFS Shares till that DC Comes back online. >From my troubleshooting i have figured out it has something to do with Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops the connections (Even though i am doing Root @ IP Range) based ESXi Mappings for NFS v3 I believe Kerberos and NFS are intertwined in some sense cause even though i can ping the IP its like i just get inherently denied access when that DC Is down... Trying to make sure if i have to take these DC's down for updates/patches/maintenance they arent going to take down my entire NFS structure... Any Help would be appreciated. KRB5.CONF File See Data *Changed domain name to DOMAIN.LOCAL* but all other information is correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power down DC1 is when i start loosing connection) # # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License (the "License"). # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at usr/src/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # ident ? "%Z%%M% ? %I% ? %E% SMI" # # krb5.conf template # In order to complete this configuration file # you will need to replace the ____ placeholders # with appropriate values for your network and uncomment the # appropriate entries. # [libdefaults] default_realm = DOMAIN.LOCAL [realms] DOMAIN.LOCAL = { kdc = 192.168.30.14 kdc = 192.168.30.15 admin_server = 192.168.30.15 kpasswd_server = 192.168.30.15 kpasswd_protocol = SET_CHANGE } [domain_realm] .domain.local = DOMAIN.LOCAL [logging] default = FILE:/var/krb5/kdc.log kdc = FILE:/var/krb5/kdc.log ? kdc_rotate = { # How often to rotate kdc.log. Logs will get rotated no more # often than the period, and less often if the KDC is not used # frequently. ? ? period = 1d # how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...) ? ? versions = 10 ? } [appdefaults] ? kinit = { ? ? renewable = true ? ? forwardable= true ? } -------------- next part -------------- An HTML attachment was scrubbed... URL: From asc1111 at gmail.com Wed Oct 26 22:08:24 2016 From: asc1111 at gmail.com (Aaron Curry) Date: Wed, 26 Oct 2016 16:08:24 -0600 Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf + NFS = NFS Disconnects in ESXI on DC Reboot) In-Reply-To: References: Message-ID: I had a similar problem. Turned out to be a DNS issue. Our DNS was running on the DC. When the DC was down, NFS wouldn't work. All I had to do was add our router as a nameserver in resolv.conf and it started working again. It seemed to be that as long as OmniOS has someone responding to DNS requests NFS will work, even if the DNS has no local info. Hope that helps. Aaron On Oct 25, 2016 1:13 PM, "Matt Mabis" wrote: > Hey All, > > Was wondering if you could help me out, using b13298F but i have seen this > in other versions too where i joined the Omni to the Domain so i could use > ACLs and UserID Mappings for SMB. however when i reboot one of the 2 DCs i > have it looks like it loses all access to the NFS Shares till that DC Comes > back online. > > From my troubleshooting i have figured out it has something to do with > Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops > the connections (Even though i am doing Root @ IP Range) based ESXi > Mappings for NFS v3 I believe Kerberos and NFS are intertwined in some > sense cause even though i can ping the IP its like i just get inherently > denied access when that DC Is down... > > Trying to make sure if i have to take these DC's down for > updates/patches/maintenance they arent going to take down my entire NFS > structure... > > Any Help would be appreciated. > > > KRB5.CONF File > See Data *Changed domain name to DOMAIN.LOCAL* but all other information > is correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power > down DC1 is when i start loosing connection) > > > # > # CDDL HEADER START > # > # The contents of this file are subject to the terms of the > # Common Development and Distribution License (the "License"). > # You may not use this file except in compliance with the License. > # > # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE > # or http://www.opensolaris.org/os/licensing. > # See the License for the specific language governing permissions > # and limitations under the License. > # > # When distributing Covered Code, include this CDDL HEADER in each > # file and include the License file at usr/src/OPENSOLARIS.LICENSE. > # If applicable, add the following below this CDDL HEADER, with the > # fields enclosed by brackets "[]" replaced with your own identifying > # information: Portions Copyright [yyyy] [name of copyright owner] > # > # CDDL HEADER END > # > # > # Copyright 2007 Sun Microsystems, Inc. All rights reserved. > # Use is subject to license terms. > # > # ident ? "%Z%%M% ? %I% ? %E% SMI" > # > > # krb5.conf template > # In order to complete this configuration file > # you will need to replace the ____ placeholders > # with appropriate values for your network and uncomment the > # appropriate entries. > # > [libdefaults] > default_realm = DOMAIN.LOCAL > > > [realms] > DOMAIN.LOCAL = { > kdc = 192.168.30.14 > kdc = 192.168.30.15 > admin_server = 192.168.30.15 > kpasswd_server = 192.168.30.15 > kpasswd_protocol = SET_CHANGE > } > > [domain_realm] > .domain.local = DOMAIN.LOCAL > > [logging] > default = FILE:/var/krb5/kdc.log > kdc = FILE:/var/krb5/kdc.log > ? kdc_rotate = { > > # How often to rotate kdc.log. Logs will get rotated no more > # often than the period, and less often if the KDC is not used > # frequently. > > ? ? period = 1d > > # how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...) > > ? ? versions = 10 > ? } > > [appdefaults] > ? kinit = { > ? ? renewable = true > ? ? forwardable= true > ? } > > _______________________________________________ > OmniOS-discuss mailing list > OmniOS-discuss at lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dasaint.blair at gmail.com Wed Oct 26 22:13:36 2016 From: dasaint.blair at gmail.com (Matt Mabis) Date: Wed, 26 Oct 2016 16:13:36 -0600 Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf + NFS = NFS Disconnects in ESXI on DC Reboot) In-Reply-To: References: Message-ID: It does, i got another email from another member directly that explained the issue as well, i however did have my router in there and another active DNS controller and yet it still happened... The other guy recommended to get around this eaiser to put the esxi host names in the /etc/hosts file to ensure that dns was ok.... Funny part is i didnt think it would be DNS as my hosts were using IPs to connect not DNS names.. My Resolv.conf had the following info and even tho .15 and .1 were both online as soon as i lost .14 it lost connectivity... so i think theres still something wonky with the NFS protocol as it shouldnt have lost connection consistently... it would go up and down like it was always trying to hit 14 first and lost connection then found 15 then reconnected... then rechecked at 14 then lost it.... 192.168.30.14 - DC1 192.168.30.15 - DC2 192.168.30.1 - Router On Wed, Oct 26, 2016 at 4:08 PM, Aaron Curry wrote: > I had a similar problem. Turned out to be a DNS issue. Our DNS was running > on the DC. When the DC was down, NFS wouldn't work. All I had to do was add > our router as a nameserver in resolv.conf and it started working again. It > seemed to be that as long as OmniOS has someone responding to DNS requests > NFS will work, even if the DNS has no local info. > > Hope that helps. > > Aaron > On Oct 25, 2016 1:13 PM, "Matt Mabis" wrote: > >> Hey All, >> >> Was wondering if you could help me out, using b13298F but i have seen >> this in other versions too where i joined the Omni to the Domain so i could >> use ACLs and UserID Mappings for SMB. however when i reboot one of the 2 >> DCs i have it looks like it loses all access to the NFS Shares till that DC >> Comes back online. >> >> From my troubleshooting i have figured out it has something to do with >> Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops >> the connections (Even though i am doing Root @ IP Range) based ESXi >> Mappings for NFS v3 I believe Kerberos and NFS are intertwined in some >> sense cause even though i can ping the IP its like i just get inherently >> denied access when that DC Is down... >> >> Trying to make sure if i have to take these DC's down for >> updates/patches/maintenance they arent going to take down my entire NFS >> structure... >> >> Any Help would be appreciated. >> >> >> KRB5.CONF File >> See Data *Changed domain name to DOMAIN.LOCAL* but all other information >> is correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power >> down DC1 is when i start loosing connection) >> >> >> # >> # CDDL HEADER START >> # >> # The contents of this file are subject to the terms of the >> # Common Development and Distribution License (the "License"). >> # You may not use this file except in compliance with the License. >> # >> # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE >> # or http://www.opensolaris.org/os/licensing. >> # See the License for the specific language governing permissions >> # and limitations under the License. >> # >> # When distributing Covered Code, include this CDDL HEADER in each >> # file and include the License file at usr/src/OPENSOLARIS.LICENSE. >> # If applicable, add the following below this CDDL HEADER, with the >> # fields enclosed by brackets "[]" replaced with your own identifying >> # information: Portions Copyright [yyyy] [name of copyright owner] >> # >> # CDDL HEADER END >> # >> # >> # Copyright 2007 Sun Microsystems, Inc. All rights reserved. >> # Use is subject to license terms. >> # >> # ident ? "%Z%%M% ? %I% ? %E% SMI" >> # >> >> # krb5.conf template >> # In order to complete this configuration file >> # you will need to replace the ____ placeholders >> # with appropriate values for your network and uncomment the >> # appropriate entries. >> # >> [libdefaults] >> default_realm = DOMAIN.LOCAL >> >> >> [realms] >> DOMAIN.LOCAL = { >> kdc = 192.168.30.14 >> kdc = 192.168.30.15 >> admin_server = 192.168.30.15 >> kpasswd_server = 192.168.30.15 >> kpasswd_protocol = SET_CHANGE >> } >> >> [domain_realm] >> .domain.local = DOMAIN.LOCAL >> >> [logging] >> default = FILE:/var/krb5/kdc.log >> kdc = FILE:/var/krb5/kdc.log >> ? kdc_rotate = { >> >> # How often to rotate kdc.log. Logs will get rotated no more >> # often than the period, and less often if the KDC is not used >> # frequently. >> >> ? ? period = 1d >> >> # how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...) >> >> ? ? versions = 10 >> ? } >> >> [appdefaults] >> ? kinit = { >> ? ? renewable = true >> ? ? forwardable= true >> ? } >> >> _______________________________________________ >> OmniOS-discuss mailing list >> OmniOS-discuss at lists.omniti.com >> http://lists.omniti.com/mailman/listinfo/omnios-discuss >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From apenner.it at gmail.com Thu Oct 27 21:38:53 2016 From: apenner.it at gmail.com (Artem Penner) Date: Thu, 27 Oct 2016 21:38:53 +0000 Subject: [OmniOS-discuss] Only grub after boot Message-ID: Hi, i have the same problem. When I try to install OmniOS on HGST HUC101818CS4204 which connected through LSI SAS 9300-16e HBA, boot hangs on "GRUB _" screen. In same JBOD I successfully install OS on HUSMH8020BSS204 SSD. I also test Solaris 11.3, but same problem appeared. -------------- next part -------------- An HTML attachment was scrubbed... URL: From danmcd at omniti.com Thu Oct 27 22:09:57 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 27 Oct 2016 18:09:57 -0400 Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018 Message-ID: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com> LTS and Stable users, please run "pkg update" for a full illumos update. It will create a new BE, and require a reboot. Both are receiving backports, and some critical kernel bugfixes. Backports include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet driver. The new uname -v for r151014 is omnios-cac2b76, and for r151018 is omnios-r151018-258cc99. Thanks, Dan From henson at acm.org Thu Oct 27 22:23:13 2016 From: henson at acm.org (Paul B. Henson) Date: Thu, 27 Oct 2016 15:23:13 -0700 Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018 In-Reply-To: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com> References: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com> Message-ID: <1fb801d230a0$b5732ad0$20598070$@acm.org> > From: Dan McDonald > Sent: Thursday, October 27, 2016 3:10 PM > > Both are receiving backports, and some critical kernel bugfixes. Backports > include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet driver. Could I trouble you to elaborate on "critical kernel bug fixes"? Are we talking "drop everything else and get these done tonight" fixes, or "schedule for the next standard maintenance window" fixes? From danmcd at omniti.com Thu Oct 27 23:05:21 2016 From: danmcd at omniti.com (Dan McDonald) Date: Thu, 27 Oct 2016 19:05:21 -0400 Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018 In-Reply-To: <1fb801d230a0$b5732ad0$20598070$@acm.org> References: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com> <1fb801d230a0$b5732ad0$20598070$@acm.org> Message-ID: <57D3304A-105A-4976-91AE-E2B9BFF3075D@omniti.com> The exploits are nontrivial, but these do have CVE numbers assigned to them (to be announced). Dan Sent from my iPhone (typos, autocorrect, and all) On Oct 27, 2016, at 6:23 PM, Paul B. Henson wrote: >> From: Dan McDonald >> Sent: Thursday, October 27, 2016 3:10 PM >> >> Both are receiving backports, and some critical kernel bugfixes. > Backports >> include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet > driver. > > Could I trouble you to elaborate on "critical kernel bug fixes"? Are we > talking "drop everything else and get these done tonight" fixes, or > "schedule for the next standard maintenance window" fixes? > > From apenner.it at gmail.com Fri Oct 28 07:40:40 2016 From: apenner.it at gmail.com (Artem Penner) Date: Fri, 28 Oct 2016 07:40:40 +0000 Subject: [OmniOS-discuss] Only grub after boot In-Reply-To: References: Message-ID: I try to install grub manually on disk, but have the same result. After that i try to install grub on usb flash, now stage2 is loading, but can't find rpool. When i try to list disks in grub shell, grub also hangs. my hardware: Supermicro 2U CSE-825TQ-R740LPB motherboard: SuperMicro 2xCPU X10DRI-LN4+ E5-2600v3/v4 UpTo2x12DIMM/ 10xSATA3/ C612 RAID 0/1/5/10/ 4xGE/ 2xPCIx16, 3xPCIx8, 1xPCIx4 (in x8)(13.68" x 13") SAS controller: LSI SAS9300-16E SGL SAS 12G disks: Hitachi HUSMH8020BSS204 - this is ok, bootable, no problem Hitachi SAS 3.0 1800Gb HUC101818CS4204 Ultrastar C10K1800 - this is disk, from which i can't boot. ??, 28 ???. 2016 ?. ? 0:38, Artem Penner : > Hi, i have the same problem. > When I try to install OmniOS on HGST HUC101818CS4204 > which > connected through LSI SAS 9300-16e HBA, boot hangs on "GRUB _" screen. > In same JBOD I successfully install OS on HUSMH8020BSS204 SSD. > > I also test Solaris 11.3, but same problem appeared. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From an3s.annema at gmail.com Mon Oct 31 19:32:12 2016 From: an3s.annema at gmail.com (Andries Annema) Date: Mon, 31 Oct 2016 20:32:12 +0100 Subject: [OmniOS-discuss] FW: Tun/Tap device installed but not showing up on OmniOS In-Reply-To: <003601d233ab$fed56db0$fc804910$@gmail.com> References: <86177927-04CE-43E4-89ED-A25458641F65@qutic.com> <003601d233ab$fed56db0$fc804910$@gmail.com> Message-ID: <003801d233ad$7b091fc0$711b5f40$@gmail.com> (Forgot the include the discuss list. Sorry) -----Original Message----- From: Andries Annema [mailto:an3s.annema at gmail.com] Sent: maandag 31 oktober 2016 20:22 To: 'qutic development' Subject: RE: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS Thanks Stefan, for your suggestion. However, when I tried to build stuff using the OmniOS build scripts that are available on github some weeks ago, I was unable to master the process; it kept throwing errors at me and I was unable to find some decent, to-the-point documentation to get on top of it. That may be due to beginners error, but for me the learning curve was too steep at the moment and I figured I had to go back to the "classic" Solaris build procedure as described here, for example: http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-097-cre ate-pkg-ips-524496.html With that, I managed to build and publish a number of packages, so for the time being I think I'll stick with that - unless someone has a _really_ good starting guide for the github-way. In the meantime, the suggestions as given by Thomas got me to solve my tun/tap-issue - I'll reply to that separately. Regards, Andries -----Original Message----- From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of qutic development Sent: dinsdag 25 oktober 2016 13:00 To: omnios-discuss Subject: Re: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS Hi Andries, > Am 24.10.2016 um 21:04 schrieb Andries Annema : > > I have installed the tun/tap device within the global zone of a r151014 machine, but the /dev/tun and /dev/tap devices don't show up. What am I missing here? I use the following build script to get a working module for r151014: https://github.com/jfqd/omnios-build/tree/151006/build/tuntap - Stefan _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss From an3s.annema at gmail.com Mon Oct 31 19:32:23 2016 From: an3s.annema at gmail.com (Andries Annema) Date: Mon, 31 Oct 2016 20:32:23 +0100 Subject: [OmniOS-discuss] FW: Tun/Tap device installed but not showing up on OmniOS In-Reply-To: <003701d233ad$3c310510$b4930f30$@gmail.com> References: <20161025124125.GC15500@wagner-net.com> <003701d233ad$3c310510$b4930f30$@gmail.com> Message-ID: <003901d233ad$813b9490$83b2bdb0$@gmail.com> (Forgot the include the discuss list. Sorry) -----Original Message----- From: Andries Annema [mailto:an3s.annema at gmail.com] Sent: maandag 31 oktober 2016 20:30 To: 'Thomas Wagner' Subject: RE: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS Thomas, thanks! Your suggestion to include the lines: driver name=tap driver name=tun in the manifest solved it for me! Wow. Although the build process with "pkgtool"/"pkgbuild" and the script as shown by your first link seem old, and even though they probably might still work, I used the build process as documented by Solaris here: http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-097-cre ate-pkg-ips-524496.html Combined this with the two "driver name"-lines and bingo! Your second link - of the OpenCSW project - got me a nice manifest to compare to and with it polish some other settings. Nice. I am not sure however on how to participate in a public repository or share my recipes any other way; I'm still learning and am not sure, security wise, on how to allow access to my personal repository and maintain it on a public level. Suggestions are always welcome though. Thanks again! Greetings, Andries -----Original Message----- From: Thomas Wagner [mailto:tom-omnios-discuss at tom.bn-ulm.de] Sent: dinsdag 25 oktober 2016 14:41 To: Andries Annema Cc: omnios-discuss Subject: Re: [OmniOS-discuss] Tun/Tap device installed but not showing up on OmniOS Hi Andries, does your IPS manifest have an "action" to add the driver entries "tun" and "tap"? If you scroll down here to the %action section: https://sourceforge.net/p/pkgbuild/code/HEAD/tree/spec-files-extra/trunk/SFE tun.spec This results if built with "pkgtool"/"pkbbuild" to an entry in the manifest file like this: driver name=tap driver name=tun A ready-made package is available here: http://sfe.opencsw.org/quickrepolinks (fully independent of OpenCSW project, but kindly hosted by them) Btw, it would be cool if you could share your build recipes and / or participate in one of the public repositories for build recipes. Regards, Thomas