<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p></p>
<p></p>
<div style="-ms-word-wrap: break-word;">
<hr tabindex="-1" style="width: 98%; display: inline-block;">
<div id="divRplyFwdMsg" dir="ltr"><font color="#000000" face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b> Richard Elling <richard.elling@richardelling.com><br>
<b>Sent:</b> Monday, January 19, 2015 1:57 PM<br>
<b>To:</b> Rune Tipsmark<br>
<b>Cc:</b> omnios-discuss@lists.omniti.com<br>
<b>Subject:</b> Re: [OmniOS-discuss] ZFS Volumes and vSphere Disks - Storage vMotion Speed</font>
<div> </div>
</div>
<div><br>
<div>
<blockquote type="cite">
<div>On Jan 19, 2015, at 3:55 AM, Rune Tipsmark <<a href="mailto:rt@steait.net">rt@steait.net</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div><style type="text/css">
<!--
p
{margin-top:0px;
margin-bottom:0px}
-->
</style>
<div style="font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; background-color: rgb(255, 255, 255);" dir="ltr">
<p>hi all,</p>
<div> <br class="webkit-block-placeholder">
</div>
<p>just in case there are other people out there using their ZFS box against vSphere 5.1 or later... I found my storage vmotion were slow... really slow... not much info available and so after a while of trial and error I found a nice combo that works very
well in terms of performance, latency as well as throughput and storage vMotion.</p>
<div> <br class="webkit-block-placeholder">
</div>
<p>- <strong>Use</strong><strong> ZFS volumes</strong> instead of thin provisioned LU's - Volumes support two of the VAAI features</p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
AFAIK, ZFS is not available in VMware. Do you mean run iSCSI to connect the ESX box to</div>
<div>the server running ZFS? If so...<br>
</div>
<div>>> I run 8G Fibre Channel<br>
</div>
<blockquote type="cite">
<div>
<div style="font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; background-color: rgb(255, 255, 255);" dir="ltr">
<p>- <strong>Use thick provisioning disks</strong>, lazy zeroed disks in my case <strong>
reduced storage vMotion by 90%</strong> or so - machine 1 dropped from 8½ minutes to 23 seconds and machine 2 dropped from ~7 minutes to 54 seconds... a rather nice improvement simply by changing from thin to thick provisioning.</p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div></div>
<div><br>
</div>
<div>This makes no difference in ZFS. The "thick provisioned" volume is simply a volume with a reservation.</div>
<div>All allocations are copy-on-write. So the only difference between a "thick" and "thin" volume occurs when</div>
<div>you run out of space in the pool.<br>
</div>
<div>>> I am talking thick provisioning in VMware, that's where it makes a huge difference<br>
</div>
<blockquote type="cite">
<div>
<div style="font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; background-color: rgb(255, 255, 255);" dir="ltr">
<p>- I dropped my Qlogic HBA <strong>max queue depth from default 64 to 16</strong> on all ESXi hosts and now I see an average latency of less than 1ms per data store (on 8G fibre channel). Of course there are spikes when doing storage vMotion at these speeds
but its well worth it.</p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div></div>
<div><br>
</div>
<div></div>
<div>I usually see storage vmotion running at wire speed for well configured systems. When you get </div>
<div></div>
<div>into the 2GByte/sec range this can get tricky, because maintaining that flow through the RAM</div>
<div></div>
<div>and disks requires nontrivial amounts of hardware.</div>
<div></div>
<div>>> I don't even get close to wire speed unfortunately my SLOGs can only do around 5-600 MBbyte/sec with sync=always.<br>
</div>
<div></div>
<div>More likely, you're seeing the effects of caching, which is very useful for storage vmotion and</div>
<div></div>
<div>allows you to hit line rate.</div>
<div><br>
</div>
<div><span>>> Not sure this is the case with using sync=always? </span></div>
<blockquote type="cite">
<div>
<div style="font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; background-color: rgb(255, 255, 255);" dir="ltr">
<div> <br class="webkit-block-placeholder">
</div>
<p>I am getting to the point where I am almost happy with my ZFS backend for vSphere.</p>
<div><br>
</div>
</div>
</div>
</blockquote>
<div></div>
<div><br>
</div>
<div></div>
<div>excellent!</div>
<div></div>
<div> -- richard</div>
<div></div>
<div><br>
</div>
<div></div>
<br>
</div>
</div>
</body>
</html>