[OmniOS-discuss] beadm activate issue (and pkg update fails to install boot files)

[Tom Robinson]

Hi,

I was discussing this on #omnios IRC so forgive the repeat if you've seen this before. And also
there's a lot of lead up to the beadm issue. It's turned out to be a deep rabbit hole.

# cat /etc/release
  OmniOS v11 r151006
  Copyright 2012-2013 OmniTI Computer Consulting, Inc. All rights reserved.
  Use is subject to license terms.

Basically we're trying to update because we think there's an issue with the infiniband srp targets
where they hang. It's a lot like this (same error messages):

https://www.illumos.org/issues/5078

OmniOS is providing iSCSI boot disk to two ESXi hosts. The guest VM disks are via SRP. Every so
often (once a month) something hangs and I have to reboot the whole shooting match to get things
working again (I've taken to beating it to the draw by rebooting before we get an issue but
sometimes it beates me!). At first I thought it was ESXi and made some queries/changes there but it
didn't fix the issue.

During a maintenance period I was trying to offline the srp target but it wouldn't offline - just
hung in 'going offline' state and I couldn't do anything with it until after reboot of the OmniOS
host. There were other tests, like after fresh reboot I could offline/online the SRP target to my
hearts content without once it hanging. Come back 24 hours later and it hangs in going offline mode.

Me thinks that there are some updates, maybe? OK, so I try pkg update but it fails because the
ca-bundle was out of sync. After some help I jumped that hurdle and decide to update to r151008
following:

http://omnios.omniti.com/wiki.php/Upgrade_r151006_r151008

But that was rejected!

http://pastebin.com/7RiqEmhy

I now think that maybe I'll just apply the r151006 pending updates. After all, pkg update -nv shows
a couple of interesting things to update. Amongst others:

network/iscsi/iser
network/iscsi/target
storage/stmf
system/kernel

So I just try pkg update under r151006 and all goes well at first but then I get an issue:

pkg: unable to activate omnios-2

# beadm list
BE              Active Mountpoint     Space Policy Created
omnios          NR     /              6.87G static 2013-08-08 23:04
omnios-1        -      -              301M  static 2013-10-15 16:32
omnios-2        -      /tmp/tmp2muEqp 450M  static 2014-09-24 07:37
omnios-backup-1 -      -              71.0K static 2013-08-12 21:25
omnios-backup-2 -      -              81.0K static 2013-08-12 21:28
omnios-backup-3 -      -              44.0K static 2014-09-23 11:06
omnios-backup-4 -      -              101K  static 2014-09-23 11:17
omniosvar       -      -              31.0K static 2013-08-08 23:04

Hmmm, no so good. On #omnios IRC a suggestion to leap to r151010 makes some sense and I follow these
steps:

beadm create r151010
beadm mount r151010 /mnt
pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/stable/ -g
http://pkg.omniti.com/omnios/r151010/ omnios
pkg -R /mnt update
beadm activate r151010

But that last step fails again with:

# beadm activate r151010
Unable to activate r151010.
Error installing boot files.

Hmm, much later I thought to pass the -v option and got:

beadm activate -v r151010
be_do_installgrub: installgrub failed for device c4t3d0s0.
  Command: "/sbin/installgrub /mnt/boot/grub/stage1 /mnt/boot/grub/stage2 /dev/rdsk/c4t3d0s0"
H�Z���H�Partition 0 of the disk has an incorrect offset
Unable to gather device information for /dev/rdsk/c4t3d0s0
be_run_cmd: command terminated with error status: 1
Unable to activate r151010.
Error installing boot files.

(also, truss output here if you're interested: http://pastebin.com/fnznXtrC )

# zpool status rpool
  pool: rpool
 state: ONLINE
  scan: resilvered 65.4G in 0h14m with 0 errors on Fri Aug  9 01:48:26 2013
config:

        NAME          STATE     READ WRITE CKSUM
        rpool         ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            c4t2d0s0  ONLINE       0     0     0
            c4t3d0s0  ONLINE       0     0     0

errors: No known data errors

Maybe this is something simple but I'm stumped. Any help is appreciated.

Thanks,
Tom

-- 

Tom Robinson
IT Manager/System Administrator

MoTeC Pty Ltd

121 Merrindale Drive
Croydon South
3136 Victoria
Australia

T: +61 3 9761 5050
F: +61 3 9761 5051   
E: tom.robinson at motec.com.au


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <https://omniosce.org/ml-archive/attachments/20140924/671a8f66/attachment.bin>