[OmniOS-discuss] 2TB vs 4TB NVMe drives?

[Bob Friesenhahn]

On Tue, 14 Nov 2017, Stephan Budach wrote:

> we are planning on purchasing a Supermicro NVMe server with 48 .U2 
> slots. I intended to initially load it with 24 x 2TB DC P4500, 
> leaving 24 slots empty.
>
>
> Now… I've been also offered an Intel chassis with only 24 slots and 
> thus the offer also included the 4TB P4500s. Just without thinking 
> very long, I instinctively wayed towards the 2TB drives, mainly for 
> the reason, that should a drive really fail, I'd have of course a 
> longer resilver at hand, usind 4TB NVMe drives.
>
>
> Which ones would you choose?

Assuming that OmniOS works with these devices at all, from a power 
consumption, heat, complexity, and reliability standpoint, the larger 
devices appear to be a win (1/2 the power and 2X the MTBF for the same 
storage capacity).  Resilver time is important but NVMe drives do not 
have the rotational latency and seek time issues of rotating media so 
resilver time should not be such an issue and there should only be a 
factor of 2 difference in resilver time.

A consideration is what zfs pool configuration you would be putting on 
these drives.  For throughput, more devices and more vdevs is better. 
It sounds like you would initially (and perhaps forever) have the same 
number of devices.

Are you planning to use zfs mirrors, or raidz2/raidz3?  What about 
dedicated zfs intent log devices?  If synchronous writes are important 
to you, dedicated zfs intent log devices should still help with pool 
performance and long-term health by deferring writes to the vdevs so 
writes can be larger and more sequential.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/