[OmniOS-discuss] 4kn or 512e with ashift=12

[Fred Liu]

> -----Original Message-----
> From: Richard Elling [mailto:richard.elling at richardelling.com]
> Sent: 星期三, 三月 23, 2016 4:53
> To: Chris Siebenmann
> Cc: Fred Liu; omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] 4kn or 512e with ashift=12
> 
> 
> 	On Mar 22, 2016, at 7:41 AM, Chris Siebenmann <cks at cs.toronto.edu
> <mailto:cks at cs.toronto.edu> > wrote:
> 
> 
> 			This implicitly assumes that the only reason to set ashift=12 is
> 			if you are currently using one or more drives that require it. I
> 			strongly disagree with this view. Since ZFS cannot currently
> replace
> 			a 512n drive with a 512e one, I feel [...]
> 
> 
> 
> 		*In theory* this replacement should work well if the lie works
> *correctly*.
> 		In ZoL, for the "-o ashift" is supported in "zpool replace", the
> 		replacement should also work in mixed sector sizes.
> 		And in illumos the whitelist will do the same.
> 		What errors have you ever seen?
> 
> 
> 
> 	We have seen devices that changed between (claimed) 512n and
> 	(claimed) 512e/4k *within the same model number*; the only thing that
> 	distinguished the two was firmware version (which is not something that
> 	you can match in sd.conf). This came as a complete surprise to us the
> 	first time we needed to replace an old (512n) one of these with a new
> 	(512e) one.
> 
> 	The sd.conf whitelist also requires a reboot to activate if you need
> 	to add a new entry, as far as I know.
> 
> 	(Nor do I know what happens if you have some 512n disks and some
> 	512e disks, both correctly recognized and in different pools, and
> 	now you need to replace a 512n disk with a spare 512e disk so you
> 	change sd.conf to claim that all of the 512e disks are 512n. I'd
> 	like to think that ZFS will carry on as normal, but I'm not sure.
> 	This makes it somewhat dangerous to change sd.conf on a live system.)

There are two cases if we don't use the remedy (whitelist in illumos or -o ashift in ZoL) here:
a): 512n <---> 512e. This replacement should work *in theory* if the lie works *correctly*.
b): 512n <-x-> 4kn.  This replacement may not work for the different physical sector sizes.

Your surprise may come from case b.
> 
> 
> 
> What is missing from
> http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks
> is:
> 
> 1. how to change the un_phy_blocksize for any or all uns 2. how to set a default
> setting for all drives in sd.conf by setting attributes to
>     the "<vid+pid>" of ""  (see sd(7d))
> 
> I am aware of no new HDDs with 512n, so this problem will go away for HDDs.
> However, there are many SSDs that work better with un_phy_blocksize = 8192
> and some vendors set sd.conf or source appropriately.
>  -- richard
> 
> 
> 
> 			For many usage cases, somewhat more space usage and
> perhaps
> 			somewhat slower pools are vastly preferable to a loss of pool
> 			redundancy over time. I feel that OmniOS should at least give
> you
> 			the option here (in a less crude way than simply telling it that
> 			absolutely all of your drives are 4k drives, partly because such
> 			general lies are problematic in various situations).
> 
> 
> 
> 		The whitelist (sd.conf) should fit into this consideration. But not
> 		sure how mixed sector sizes impact the performance.
> 
> 
> 
> 	Oh, 512e disks in a 512n pool will probably have not great performance.
> 	ZFS does a lot of unaligned reads and writes, unlike other filesystems;
> 	if you say your disks are 512n, it really believes you and behaves
> 	accordingly.

I am just curious about if the mixed sector sizes(512n+4kn) will impact performance.

Thanks.

Fred