[OmniOS-discuss] fmd core dump in r150008j

[Lawrence Giam]

Hi All,

I have a new server and today I got an alert that diskspace was almost
fully used up. Checking on the system, I found that under /var/fm/fmd,
there is a core dump file. I checked /var/adm/messages and fmdump, cannot
find anything suspicious.

I am running OmniOS r150008j.

This is an analysis of the core dump.

root at sgbk02:/tankD01/backups/sgsan1n1/fmd# mdb core.fmd.559
Loading modules: [ fmd libumem.so.1 libc.so.1 libnvpair.so.1 libtopo.so.1
libuutil.so.1 libavl.so.1 libsysevent.so.1 eft.so ld.so.1 ]
> $c
libc.so.1`_lwp_kill+0x15(4, 6, 6786, fef56000, fef56000, 95)
libc.so.1`raise+0x2b(6, 0, fda5ea00, feed5a09, 0, 0)
libc.so.1`abort+0x10e(3a646d66, 4f424120, 203a5452, 75736e69, 63696666,
746e6569)
fmd_panic(80835e0, fda5ee84, 1, 0)
fmd_panic+0x12(80835e0, 50, 3e8, feb0e8e4)
fmd_alloc+0x81(50, 1, 80, e3b74613, 1, 3b9aca00)
fmd_event_create+0x18(1, 88fe3252, 11f57f, 0, fbd3ab30, 896ab30)
fmd_module_timeout+0x20(fbd3ab30, c4, 88fe3252, 11f57f, 8393e08)
fmd_timerq_exec+0x127(8393e00, 0, fed922a0, fef56000)
fmd_thread_start+0x5b(819e6d0, 0, 0, 0)
libc.so.1`_thrp_setup+0x88(fed92240)
libc.so.1`_lwp_start(fed92240, 0, 0, 0, 0, 0)

> ::showrev
Hostname: sgsan1n1
Release: 5.11
Kernel architecture: i86pc
Application architecture: i386
Kernel version: SunOS 5.11 i86pc omnios-6de5e81
Platform: i86pc

> ::status
debugging core file of fmd (32-bit) from sgsan1n1
file: /usr/lib/fm/fmd/fmd
initial argv: /usr/lib/fm/fmd/fmd
threading model: native threads
status: process terminated by SIGABRT (Abort), pid=559 uid=0 code=-1

> $r
%cs = 0x0043            %eax = 0x00000000
%ds = 0x004b            %ebx = 0xfef56000
%ss = 0x004b            %ecx = 0xfda5e9ac
%es = 0x004b            %edx = 0x00000000
%fs = 0x0000            %esi = 0x00000095
%gs = 0x01c3            %edi = 0x080835e0

 %eip = 0xfeee78b5 libc.so.1`_lwp_kill+0x15
 %ebp = 0xfda5e9c8
%kesp = 0x00000000

%eflags = 0x00000282
  id=0 vip=0 vif=0 ac=0 vm=0 rf=0 nt=0 iopl=0x0
  status=<of,df,IF,tf,SF,zf,af,pf,cf>

   %esp = 0xfda5e9ac
%trapno = 0xe
   %err = 0x6

> 0xfeee78b5::dis
libc.so.1`_lwp_kill:            call   +0x0     <libc.so.1`_lwp_kill+5>
libc.so.1`_lwp_kill+5:          popl   %edx
libc.so.1`_lwp_kill+6:          movl   $0xa3,%eax
libc.so.1`_lwp_kill+0xb:        movl   %esp,%ecx
libc.so.1`_lwp_kill+0xd:        addl   $0x10,%edx
libc.so.1`_lwp_kill+0x13:       sysenter
libc.so.1`_lwp_kill+0x15:       jae    +0xc     <libc.so.1`_lwp_kill+0x23>
libc.so.1`_lwp_kill+0x17:       cmpl   $0x5b,%eax
libc.so.1`_lwp_kill+0x1a:       jne    +0x9     <libc.so.1`_lwp_kill+0x25>
libc.so.1`_lwp_kill+0x1c:       movl   $0x4,%eax
libc.so.1`_lwp_kill+0x21:       jmp    +0x2     <libc.so.1`_lwp_kill+0x25>
libc.so.1`_lwp_kill+0x23:       xorl   %eax,%eax
libc.so.1`_lwp_kill+0x25:       ret
0xfeee78c6:                     leal   0x0(%esi),%esi
0xfeee78c9:                     leal   0x0(%edi),%edi
libc.so.1`_lwp_self:            call   +0x0     <libc.so.1`_lwp_self+5>
libc.so.1`_lwp_self+5:          popl   %edx

Looks like fmd crashed. Can anyone help me confirm that I am right?

Thanks & Regards.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20140225/54efcfc3/attachment.html>