[OmniOS-discuss] How to get NFS read & write latency in OmniOS r151016

Thu Dec 24 22:44:35 UTC 2015

> On Dec 23, 2015, at 12:58 AM, Dan Vatca <dan at syneto.eu> wrote:
> 
> If you need latency, you will most likely need a latency distribution histogram, and not an average latency.
> With averages you will lose latency outliers that are very important. Here's a good read with lots of references on this topic: https://www.vividcortex.com/blog/why-percentiles-dont-work-the-way-you-think <https://www.vividcortex.com/blog/why-percentiles-dont-work-the-way-you-think>
> To currently do this on OmniOS, you need to use dtrace to aggregate (quantize) time differences between nfsv3:::op-read-start and nfsv3:::op-read-done (same for write).

Indeed, distributions are much more enlightening than averages.

Unfortunately, the new kstats added for NFS server operations on a per-mountpoint basis
are implemented using the Riemann sums (KSTAT_TYPE_IO) and it is not possible to obtain
per-operation information needed for min/max or distribution. These are the same type of 
kstat used for the iostat command.

Shameless plug, nfssvrtop has proven to be useful in watching NFS traffic and uses the 
op-read-start/op-read-done method.
https://github.com/richardelling/tools <https://github.com/richardelling/tools>

 — richard

> 
> 
> Dan Vâtca
> CTO at Syneto
> Tel: +40723604357, Skype: dan_vatca
>  <http://www.syneto.net/>
> On Wed, Dec 23, 2015 at 2:44 AM, 張峻宇 <jerry1209 at cht.com.tw <mailto:jerry1209 at cht.com.tw>> wrote:
> Hi all,
> 
>       According to the release note of OmniOS r151016, we could get “IOPS, bandwidth, and latency kstats for NFS server”
> 
>  
> 
>       there is lots of information showing when I use enter command #kstat,
> 
>       I want to get the “nfs read & write latency for NFS server”
> 
>      
> 
>       Q1 : Is the ‘nfs:0:rfsprocio_v4_write:wtime’ & ‘nfs:0:rfsprocio_v4_read:wtime’ meant write & read latency ?
> 
>       Q2 : I mounted the nfs share directory, and write lots file to it, the number of ‘nfs:0:rfsprocio_v4_write:wtime’ & ‘nfs:0:rfsprocio_v4_read:wtime’ still zero. Why ?
> 
>  
> 
>       #kstat –p –m nfs –n rfsprocio_v4_write
> 
>         nfs:0:rfsprocio_v4_write:class        rfsprocio_v4
> 
> nfs:0:rfsprocio_v4_write:crtime     50.833043074
> 
> nfs:0:rfsprocio_v4_write:nread      3932160
> 
> nfs:0:rfsprocio_v4_write:nwritten  5374607360
> 
> nfs:0:rfsprocio_v4_write:rcnt 0
> 
> nfs:0:rfsprocio_v4_write:reads       163840
> 
> nfs:0:rfsprocio_v4_write:rlastupdate     12048225488385
> 
> nfs:0:rfsprocio_v4_write:rlentime  33429565743
> 
> nfs:0:rfsprocio_v4_write:rtime       23992279289
> 
> nfs:0:rfsprocio_v4_write:snaptime 269635.483575440
> 
> nfs:0:rfsprocio_v4_write:wcnt        0
> 
> nfs:0:rfsprocio_v4_write:wlastupdate    0
> 
> nfs:0:rfsprocio_v4_write:wlentime 0
> 
> nfs:0:rfsprocio_v4_write:writes      163840            / number of writes /
> 
> nfs:0:rfsprocio_v4_write:wtime     0                      / wait queue - time spent waiting /
> 
>  
> 
>         #kstat –p –m nfs –n rfsprocio_v4_read
> 
>         nfs:0:rfsprocio_v4_read:class rfsprocio_v4
> 
> nfs:0:rfsprocio_v4_read:crtime      50.833003263
> 
> nfs:0:rfsprocio_v4_read:nread       0
> 
> nfs:0:rfsprocio_v4_read:nwritten   0
> 
> nfs:0:rfsprocio_v4_read:rcnt  0
> 
> nfs:0:rfsprocio_v4_read:reads        0
> 
> nfs:0:rfsprocio_v4_read:rlastupdate      0
> 
> nfs:0:rfsprocio_v4_read:rlentime   0
> 
> nfs:0:rfsprocio_v4_read:rtime        0
> 
> nfs:0:rfsprocio_v4_read:snaptime  269635.483080962
> 
> nfs:0:rfsprocio_v4_read:wcnt 0
> 
> nfs:0:rfsprocio_v4_read:wlastupdate     0
> 
> nfs:0:rfsprocio_v4_read:wlentime 0
> 
> nfs:0:rfsprocio_v4_read:writes       0
> 
> nfs:0:rfsprocio_v4_read:wtime      0
> 
>        
> 
>  
> 
>  
> 
> Best regards,
> 
> ---------------------------------------------
> 
> 張峻宇
> 
> 中華電信研究院雲端運算研究所
> 
> TEL: 03-4245663
> 
>  
> 
> 
> 
> 本信件可能包含中華電信股份有限公司機密資訊,非指定之收件者,請勿蒐集、處理或利用本信件內容,並請銷毀此信件. 如為指定收件者,應確實保護郵件中本公司之營業機密及個人資料,不得任意傳佈或揭露,並應自行確認本郵件之附檔與超連結之安全性,以共同善盡資訊安全與個資保護責任. 
> Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information.
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

--

Richard.Elling at RichardElling.com
+1-760-896-4422

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20151224/f8d10dfb/attachment-0001.html>