<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>NFS writes (especially for lots of small files) to omniOS
*really* benefit from having ZIL on those SSD.</p>
<p>You could remove the cache from the pool, carve of an 8GB chunk
for ZIL on each and the rest for L2arc if you want that. <br>
</p>
<p>then add a mirrored zil using the 8GB chunks and the other
partition for l2arc</p>
<p>An SSD zil helps with metadata update absorption and small files
writes that are synchronous over NFS. A lot. (that's my
experience)</p>
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 8/23/2018 12:38 PM, Lee Damon wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CABXAO0+u-4HO8_vzdz3ndcg3vsF550jAu-UVhmP5wrx70azc7A@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div dir="ltr">These are 12TB SAS drives (Seagate ST12000NM0027)
for data & hot spare. ZIL & L2ARC are 480GB INTEL
SSDSC2KG48 SSDs. Everything is left at default for sector size,
etc. They were basically prepared for into the pool with a
simple fdisk -B /dev/rdsk/drive.
<div><br>
</div>
<div>Ping never shows loss of connectivity. I ran this for about
5 minutes during a test:</div>
<div><br>
</div>
<div>
<div> 303 packets transmitted, 303 received, 0% packet loss,
time 302021ms</div>
<div> rtt min/avg/max/mdev = 0.109/0.281/2.881/0.227 ms</div>
</div>
<div><br>
</div>
<div>CIFS, scp, and rsync do not exhibit the problem. I forgot
to mention that local copies on the file server are also as
fast as I would expect (~2 min).
<div><br>
</div>
<div>
<div> pool: pool0<br>
</div>
<div> state: ONLINE</div>
<div> scan: none requested</div>
<div>config:</div>
<div><br>
</div>
<div> NAME STATE READ
WRITE CKSUM</div>
<div> pool0 ONLINE 0
0 0</div>
<div> raidz2-0 ONLINE 0
0 0</div>
<div> c0t5000C500A612DA93d0 ONLINE 0
0 0</div>
<div> c0t5000C500957D4A93d0 ONLINE 0
0 0</div>
<div> c0t5000C500957D4C1Bd0 ONLINE 0
0 0</div>
<div> c0t5000C500957D25B3d0 ONLINE 0
0 0</div>
<div> c0t5000C500957D27F3d0 ONLINE 0
0 0</div>
<div> c0t5000C500957D2553d0 ONLINE 0
0 0</div>
<div> logs</div>
<div> mirror-1 ONLINE 0
0 0</div>
<div> c0t55CD2E414EC0FF43d0s0 ONLINE 0
0 0</div>
<div> c3t0d0s0 ONLINE 0
0 0</div>
<div> cache</div>
<div> c0t55CD2E414EC0FF43d0s1 ONLINE 0
0 0</div>
<div> c3t0d0s1 ONLINE 0
0 0</div>
<div> spares</div>
<div> c0t5000C50095722E27d0 AVAIL </div>
</div>
<div><br>
</div>
<div>
<div>iostat:</div>
<div> extended device statistics
<br>
</div>
<div> r/s w/s kr/s kw/s wait actv wsvc_t asvc_t
%w %b device</div>
<div> 15.8 57.5 1302.5 5372.3 823.0 0.6 11233.2 8.4
9 9 pool0</div>
<div> 0.1 19.9 1.7 163.0 0.0 0.0 1.0 0.1
0 0 rpool</div>
<div> 0.1 10.2 0.9 81.5 0.0 0.0 0.0 0.0
0 0 c1t4d0</div>
<div> 0.0 10.1 0.8 81.5 0.0 0.0 0.0 0.1
0 0 c1t5d0</div>
<div> 2.7 19.5 337.8 1359.4 0.4 0.0 16.1 0.5
8 1 c3t0d0</div>
<div> 1.9 6.0 114.5 442.2 0.0 0.0 0.0 5.0
0 1 c0t5000C500957D27F3d0</div>
<div> 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1
0 0 c0t5000C50095722E27d0</div>
<div> 1.5 6.0 86.2 442.9 0.0 0.0 0.0 4.7
0 1 c0t5000C500957D25B3d0</div>
<div> 1.5 6.0 78.2 442.5 0.0 0.0 0.0 4.2
0 1 c0t5000C500957D4C1Bd0</div>
<div> 1.7 6.1 102.4 442.8 0.0 0.0 0.0 4.7
0 1 c0t5000C500A612DA93d0</div>
<div> 1.6 6.0 86.6 442.5 0.0 0.0 0.0 4.3
0 1 c0t5000C500957D2553d0</div>
<div> 2.0 5.9 122.2 442.2 0.0 0.0 0.0 5.0
0 1 c0t5000C500957D4A93d0</div>
<div> 2.9 19.5 374.7 1357.9 0.0 0.0 0.0 1.2
0 1 c0t55CD2E414EC0FF43d0</div>
<div>c1t4d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: ATA Product: INTEL SSDSC2KB24 Revision:
0121 Serial No: BTYS817407RE240 </div>
<div>Size: 240.06GB <240057409536 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c1t5d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: ATA Product: INTEL SSDSC2KB24 Revision:
0121 Serial No: BTYS817409YS240 </div>
<div>Size: 240.06GB <240057409536 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c3t0d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: ATA Product: INTEL SSDSC2KG48 Revision:
0121 Serial No: BTYM7405027L480 </div>
<div>Size: 480.10GB <480103981056 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500957D27F3d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0VFGX0000J74 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C50095722E27d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0S42H0000J75 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500957D25B3d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0VFJV0000J74 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500957D4C1Bd0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0P6050000J83 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500A612DA93d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0WCCN0000J80 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500957D2553d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0VFK30000J74 </div>
</div>
<div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t5000C500957D4A93d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: SEAGATE Product: ST12000NM0027 Revision:
E001 Serial No: ZJV0VBQ80000R81 </div>
<div>Size: 12000.14GB <12000138625024 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
<div>c0t55CD2E414EC0FF43d0 Soft Errors: 0 Hard Errors: 0
Transport Errors: 0 </div>
<div>Vendor: ATA Product: INTEL SSDSC2KG48 Revision:
0121 Serial No: BTYM740600ZT480 </div>
<div>Size: 480.10GB <480103981056 bytes></div>
<div>Media Error: 0 Device Not Ready: 0 No Device: 0
Recoverable: 0 </div>
<div>Illegal Request: 0 Predictive Failure Analysis: 0 </div>
</div>
<div><br>
</div>
<div>nomad</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OmniOS-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a>
<a class="moz-txt-link-freetext" href="http://lists.omniti.com/mailman/listinfo/omnios-discuss">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>