[OmniOS-discuss] Slow NFS writes in 151026
Doug Hughes
doug at will.to
Thu Aug 23 16:56:24 UTC 2018
NFS writes (especially for lots of small files) to omniOS *really*
benefit from having ZIL on those SSD.
You could remove the cache from the pool, carve of an 8GB chunk for ZIL
on each and the rest for L2arc if you want that.
then add a mirrored zil using the 8GB chunks and the other partition for
l2arc
An SSD zil helps with metadata update absorption and small files writes
that are synchronous over NFS. A lot. (that's my experience)
On 8/23/2018 12:38 PM, Lee Damon wrote:
> These are 12TB SAS drives (Seagate ST12000NM0027) for data & hot
> spare. ZIL & L2ARC are 480GB INTEL SSDSC2KG48 SSDs. Everything is left
> at default for sector size, etc. They were basically prepared for into
> the pool with a simple fdisk -B /dev/rdsk/drive.
>
> Ping never shows loss of connectivity. I ran this for about 5 minutes
> during a test:
>
> 303 packets transmitted, 303 received, 0% packet loss, time 302021ms
> rtt min/avg/max/mdev = 0.109/0.281/2.881/0.227 ms
>
> CIFS, scp, and rsync do not exhibit the problem. I forgot to mention
> that local copies on the file server are also as fast as I would
> expect (~2 min).
>
> pool: pool0
> state: ONLINE
> scan: none requested
> config:
>
> NAME STATE READ WRITE CKSUM
> pool0 ONLINE 0 0 0
> raidz2-0 ONLINE 0 0 0
> c0t5000C500A612DA93d0 ONLINE 0 0 0
> c0t5000C500957D4A93d0 ONLINE 0 0 0
> c0t5000C500957D4C1Bd0 ONLINE 0 0 0
> c0t5000C500957D25B3d0 ONLINE 0 0 0
> c0t5000C500957D27F3d0 ONLINE 0 0 0
> c0t5000C500957D2553d0 ONLINE 0 0 0
> logs
> mirror-1 ONLINE 0 0 0
> c0t55CD2E414EC0FF43d0s0 ONLINE 0 0 0
> c3t0d0s0 ONLINE 0 0 0
> cache
> c0t55CD2E414EC0FF43d0s1 ONLINE 0 0 0
> c3t0d0s1 ONLINE 0 0 0
> spares
> c0t5000C50095722E27d0 AVAIL
>
> iostat:
> extended device statistics
> r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
> 15.8 57.5 1302.5 5372.3 823.0 0.6 11233.2 8.4 9 9 pool0
> 0.1 19.9 1.7 163.0 0.0 0.0 1.0 0.1 0 0 rpool
> 0.1 10.2 0.9 81.5 0.0 0.0 0.0 0.0 0 0 c1t4d0
> 0.0 10.1 0.8 81.5 0.0 0.0 0.0 0.1 0 0 c1t5d0
> 2.7 19.5 337.8 1359.4 0.4 0.0 16.1 0.5 8 1 c3t0d0
> 1.9 6.0 114.5 442.2 0.0 0.0 0.0 5.0 0 1
> c0t5000C500957D27F3d0
> 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0 0
> c0t5000C50095722E27d0
> 1.5 6.0 86.2 442.9 0.0 0.0 0.0 4.7 0 1
> c0t5000C500957D25B3d0
> 1.5 6.0 78.2 442.5 0.0 0.0 0.0 4.2 0 1
> c0t5000C500957D4C1Bd0
> 1.7 6.1 102.4 442.8 0.0 0.0 0.0 4.7 0 1
> c0t5000C500A612DA93d0
> 1.6 6.0 86.6 442.5 0.0 0.0 0.0 4.3 0 1
> c0t5000C500957D2553d0
> 2.0 5.9 122.2 442.2 0.0 0.0 0.0 5.0 0 1
> c0t5000C500957D4A93d0
> 2.9 19.5 374.7 1357.9 0.0 0.0 0.0 1.2 0 1
> c0t55CD2E414EC0FF43d0
> c1t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: ATA Product: INTEL SSDSC2KB24 Revision: 0121 Serial No:
> BTYS817407RE240
> Size: 240.06GB <240057409536 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c1t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: ATA Product: INTEL SSDSC2KB24 Revision: 0121 Serial No:
> BTYS817409YS240
> Size: 240.06GB <240057409536 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: ATA Product: INTEL SSDSC2KG48 Revision: 0121 Serial No:
> BTYM7405027L480
> Size: 480.10GB <480103981056 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500957D27F3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0VFGX0000J74
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C50095722E27d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0S42H0000J75
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500957D25B3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0VFJV0000J74
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500957D4C1Bd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0P6050000J83
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500A612DA93d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0WCCN0000J80
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500957D2553d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0VFK30000J74
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t5000C500957D4A93d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: SEAGATE Product: ST12000NM0027 Revision: E001 Serial No:
> ZJV0VBQ80000R81
> Size: 12000.14GB <12000138625024 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c0t55CD2E414EC0FF43d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
> Vendor: ATA Product: INTEL SSDSC2KG48 Revision: 0121 Serial No:
> BTYM740600ZT480
> Size: 480.10GB <480103981056 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
>
> nomad
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20180823/3d01361f/attachment-0001.html>
More information about the OmniOS-discuss
mailing list