[OmniOS-discuss] ipadm hangs on interface creation

[Dan McDonald]

> On Dec 10, 2014, at 11:12 AM, Stephan Budach <stephan.budach at jvm.de> wrote:
> 
> Hi Dan,
> 
> I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running?

I meant what exact command-line arguments you typed for ipadm(1M).  But that's okay, because it's good to know the other things you're about to show me.

> What is really funny, is that running pstack against the ipadm process, somehow brought things back in line:
> 
> root at nfsvmpool01:~# pkg info entire
>          Name: entire
>       Summary: Incorporation to constrain core system packages to same build
>   Description: This package constrains core system package versions to the same
>                build and provides a minimal set of additional packages.
>         State: Installed
>     Publisher: omnios
>       Version: 11
> Build Release: 5.11
>        Branch: 0.151006
> Packaging Date: Mon Oct 27 19:15:00 2014
>          Size: 0.00 B
>          FMRI: pkg://omnios/entire@11,5.11-0.151006:20141027T191500Z
> 
> I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested:
> 
> root at nfsvmpool01:~# pstack `pgrep ipadm`
> 12397:    ipadm create-if ixgbe3
> fef07723 open     (8047190, 2, fec9404c)
> feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
> 0805553e do_create_if (2) + 8f
> 08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df
> 080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83
> 
> ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors.

That's VERY odd, albeit with pleasant results.  I wouldn't think pstack would unblock something like what you showed me.  You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now.

You mentioned a desire to not reboot your system.  Did you plug in these ixgbes while booted (i.e. hotplugging)?  Or were they always there, and you just hadn't configured them until now?

It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free.  I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem.  The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur.

Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug.

Dan