[OmniOS-discuss] ipadm hangs on interface creation

[Stephan Budach]

Am 10.12.14 um 17:20 schrieb Dan McDonald:
>> On Dec 10, 2014, at 11:12 AM, Stephan Budach <stephan.budach at jvm.de> wrote:
>>
>> Hi Dan,
>>
>> I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running?
> I meant what exact command-line arguments you typed for ipadm(1M).  But that's okay, because it's good to know the other things you're about to show me.
>
>> What is really funny, is that running pstack against the ipadm process, somehow brought things back in line:
>>
>> root at nfsvmpool01:~# pkg info entire
>>           Name: entire
>>        Summary: Incorporation to constrain core system packages to same build
>>    Description: This package constrains core system package versions to the same
>>                 build and provides a minimal set of additional packages.
>>          State: Installed
>>      Publisher: omnios
>>        Version: 11
>> Build Release: 5.11
>>         Branch: 0.151006
>> Packaging Date: Mon Oct 27 19:15:00 2014
>>           Size: 0.00 B
>>           FMRI: pkg://omnios/entire@11,5.11-0.151006:20141027T191500Z
>>
>> I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested:
>>
>> root at nfsvmpool01:~# pstack `pgrep ipadm`
>> 12397:    ipadm create-if ixgbe3
>> fef07723 open     (8047190, 2, fec9404c)
>> feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
>> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
>> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
>> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
>> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
>> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
>> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
>> 0805553e do_create_if (2) + 8f
>> 08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df
>> 080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83
>>
>> ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors.
> That's VERY odd, albeit with pleasant results.  I wouldn't think pstack would unblock something like what you showed me.  You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now.
Yes, it is… ;)
>
> You mentioned a desire to not reboot your system.  Did you plug in these ixgbes while booted (i.e. hotplugging)?  Or were they always there, and you just hadn't configured them until now?
The two cards had been in the system right from the start and I already 
configured one port (ixgbe0) to connect to a 10GbE port on our 6509, 
which went without issues. I wanted to resolve this issue without 
rebooting, since rebooting would have killed 53 VMs that are residing on 
my NFS export that is running over ixgbe0.
> It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free.  I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem.  The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur.
I can maybe help with that as well… I tried to set the MTU to the 
maximum, that the driver would accept and I ended up with a mtu of 15500 
(what a odd number anyway) which I set using dladm on ixgbe3. When i 
tried to create the interface using ipadm, that was the moment, when 
things went quirky. Although I trimmed the mtu down to a reasonable 9216 
ipadm wouldn't create any longer and would hang at creating interfaces 
on any of the remaining 3 ports of my T540-X2's.
>
> Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug.
>
> Dan
>
Ha ha, no prob - you already provided a great deal of help!

Thanks a lot,
budy