[OmniOS-discuss] 4kn or 512e with ashift=12

[Chris Siebenmann]

> >  This implicitly assumes that the only reason to set ashift=12 is
> > if you are currently using one or more drives that require it. I
> > strongly disagree with this view. Since ZFS cannot currently replace
> > a 512n drive with a 512e one, I feel [...]
> 
> *In theory* this replacement should work well if the lie works *correctly*.
>  In ZoL, for the "-o ashift" is supported in "zpool replace", the
> replacement should also work in mixed sector sizes.
>  And in illumos the whitelist will do the same.
>  What errors have you ever seen?

 We have seen devices that changed between (claimed) 512n and
(claimed) 512e/4k *within the same model number*; the only thing that
distinguished the two was firmware version (which is not something that
you can match in sd.conf). This came as a complete surprise to us the
first time we needed to replace an old (512n) one of these with a new
(512e) one.

 The sd.conf whitelist also requires a reboot to activate if you need
to add a new entry, as far as I know.

(Nor do I know what happens if you have some 512n disks and some
512e disks, both correctly recognized and in different pools, and
now you need to replace a 512n disk with a spare 512e disk so you
change sd.conf to claim that all of the 512e disks are 512n. I'd
like to think that ZFS will carry on as normal, but I'm not sure.
This makes it somewhat dangerous to change sd.conf on a live system.)

> >  For many usage cases, somewhat more space usage and perhaps
> > somewhat slower pools are vastly preferable to a loss of pool
> > redundancy over time. I feel that OmniOS should at least give you
> > the option here (in a less crude way than simply telling it that
> > absolutely all of your drives are 4k drives, partly because such
> > general lies are problematic in various situations).
>
> The whitelist (sd.conf) should fit into this consideration. But not
> sure how mixed sector sizes impact the performance.

 Oh, 512e disks in a 512n pool will probably have not great performance.
ZFS does a lot of unaligned reads and writes, unlike other filesystems;
if you say your disks are 512n, it really believes you and behaves
accordingly.

	- cks