<div dir="ltr">Hi Dan,<div><br></div><div>Thanks for sharing! I run OmniOS v11 r151012 on a home NFS server that gets turned on and off a lot; I was pretty consistently getting the nlockmgr maintenance state just after boot. I even went as far as to set the start timeout to 10 minutes:</div><div><br></div><div>svccfg -s nlockmgr setprop \start/timeout_seconds=count: 600<br></div><div><br></div><div>And then I edited the /lib/svc/method/nlockmgr startup script to retry up to 100 times:</div><div><br></div><div><div>diff --git a/nlockmgr b/nlockmgr</div><div>index 3a543d6..0b24bbc 100755</div><div>--- a/nlockmgr</div><div>+++ b/nlockmgr</div><div>@@ -61,4 +61,12 @@ then</div><div> fi</div><div> fi</div><div> </div><div>-exec /usr/lib/nfs/lockd</div><div>+i=0</div><div>+while ! (echo "$0: starting lockd (attempt $i)"; /usr/lib/nfs/lockd) ; do</div><div>+ if [ $i -ge 100 ] ; then</div><div>+ exit 1</div><div>+ fi</div><div>+ i=$((i + 1))</div><div>+done</div><div>+</div><div>+exit 0</div></div><div><br></div><div>An ugly hack but at least it freed me from having to baby-sit the NFS server after boot.</div><div><br></div><div>But then I saw your email and I decided to just remove the contents of /var/statmon/sm.bak:</div><div><br></div><div>rm /var/statmon/sm.bak/*<br></div><div><br></div><div>I have not had any nlockmgr trouble since. Yay! I only did the sm.bak cleanup today though.</div><div><br></div><div>Best wishes,</div><div><br></div><div>Jacob</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2014-11-17 16:12 GMT+01:00 Dan McDonald <span dir="ltr"><<a href="mailto:danmcd@omniti.com" target="_blank">danmcd@omniti.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">ISTR someone here had a problem where his/her nlockmgr SMF service wouldn't start. I've encountered this problem myself recently (on a OI VM, but it's running the same new open-source lockd that's in OmniOS with r151010 and later), and wanted to share a conversation from the illumos developer's list.<br> <br> FYI,<br> Dan<br> <br> <br> > Begin forwarded message:<br> ><br> > Subject: Re: [developer] lockd not starting?<br> > From: Dan McDonald <<a href="mailto:danmcd@omniti.com">danmcd@omniti.com</a>><br> > Date: November 17, 2014 at 10:10:38 AM EST<br> > Cc: illumos Developer <<a href="mailto:developer@lists.illumos.org">developer@lists.illumos.org</a>>, Matt Amdur <<a href="mailto:matt.amdur@delphix.com">matt.amdur@delphix.com</a>><br> > To: Sebastien Roy <<a href="mailto:sebastien.roy@delphix.com">sebastien.roy@delphix.com</a>><br> ><br> ><br> >> On Nov 16, 2014, at 7:54 PM, Sebastien Roy <<a href="mailto:sebastien.roy@delphix.com">sebastien.roy@delphix.com</a>> wrote:<br> >><br> >> Hey Dan,<br> >><br> >> I think we're looking into a similar problem here at Delphix. We've noticed that the nfs/nlockmgr service goes into maintenance mode after a timeout of the SM_CRASH call to statd.<br> >><br> >> Upon startup, the statd daemon is blocked notifying clients. These clients are cached in /var/statmon/sm.bak/... If any of these clients are unreachable, the SM_CRASH call issued by klm will timeout (this timeout is much shorter than that which statd uses to give up on client notifications), and the nfs/nlockmgr service will then end up in maintenance state.<br> >><br> >> We've been discussing possible fixes to this problem. One easy fix that Matt Amdur (Cc'ed) proposed would be to shorten the timeout that statd uses to notify clients to reduce the chance that lockd's SM_CRASH call would timeout in the event of an unreachable client. We haven't implemented the fix yet.<br> >><br> >> A workaround is to clear statd's cached clients.<br> ><br> > Okay! This makes some modicum of sense. Here's what I have now:<br> ><br> > # ls -lt /var/statmon/sm.bak/<br> > total 2<br> > lrwxrwxrwx 1 daemon daemon 10 Nov 12 19:55 ipv4.10.0.1.68 -> Everywhere<br> > lrwxrwxrwx 1 daemon daemon 28 Nov 11 12:44 ipv4.10.8.3.241 -> <a href="http://everywhere.office.omniti.com" target="_blank">everywhere.office.omniti.com</a><br> > # nslookup everywhere<br> > Server: 10.8.3.1<br> > Address: 10.8.3.1#53<br> ><br> > Name: <a href="http://everywhere.office.omniti.com" target="_blank">everywhere.office.omniti.com</a><br> > Address: 10.8.3.241<br> ><br> > #<br> ><br> > While travelling last week, I had to park myself on a <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> network. My VMs are configured to be link-sharers (same-link peers), so I was using NFS over the same link to my VM. Once I renumbered back to the intended OmniTI networks, things started going screwy.<br> ><br> > I'm going to remove the 10.0.1.68 one and try again... BINGO!<br> ><br> > # svcs -xv nlockmgr<br> > svc:/network/nfs/nlockmgr:default (NFS lock manager)<br> > State: maintenance since November 16, 2014 04:00:05 PM EST<br> > Reason: Start method failed repeatedly, last exited with status 1.<br> > See: <a href="http://illumos.org/msg/SMF-8000-KS" target="_blank">http://illumos.org/msg/SMF-8000-KS</a><br> > See: man -M /usr/share/man -s 1M lockd<br> > See: /var/svc/log/network-nfs-nlockmgr:default.log<br> > Impact: 1 dependent service is not running:<br> > svc:/network/nfs/server:default<br> > # rm /var/statmon/sm.bak/ipv4.10.0.1.68<br> > # svcadm clear nlockmgr<br> > # svcs -xv nlockmgr<br> > svc:/network/nfs/nlockmgr:default (NFS lock manager)<br> > State: online since November 17, 2014 10:08:53 AM EST<br> > See: man -M /usr/share/man -s 1M lockd<br> > See: /var/svc/log/network-nfs-nlockmgr:default.log<br> > Impact: None.<br> > #<br> ><br> > This is actually good, in that I believe the circumstances for this bug are now reproducible.<br> ><br> > Thanks!<br> > Dan<br> ><br> ><br> <br> _______________________________________________<br> OmniOS-discuss mailing list<br> <a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br> <a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br> </blockquote></div><br></div>