[OmniOS-discuss] 4kn or 512e with ashift=12

[Chris Siebenmann]

> > 	The sd.conf whitelist also requires a reboot to activate if you need
> > 	to add a new entry, as far as I know.
> > 
> >     (Nor do I know what happens if you have some 512n disks and
> >     some 512e disks, both correctly recognized and in different
> >     pools, and now you need to replace a 512n disk with a spare 512e
> >     disk so you change sd.conf to claim that all of the 512e disks
> >     are 512n. I'd like to think that ZFS will carry on as normal,
> >     but I'm not sure.  This makes it somewhat dangerous to change
> >     sd.conf on a live system.)
> 
> There are two cases if we don't use the remedy (whitelist in illumos
> or -o ashift in ZoL) here:
> a): 512n <---> 512e. This replacement should work *in theory* if the
> lie works *correctly*.

 This will not work without the sd.conf workaround in Illumos.

 All 512e disks that I know of correctly report their actual physical
disk size to Illumos (and to Linux/ZoL). When a disk reports a 4K
physical sector size, ZFS will refuse to allow it into an ashift=9
vdev *regardless* of the fact that it is 512e and will accept reads
and writes in 512-byte sectors.

 In Illumos, you can use sd.conf to lie to the system and claim that
this is not a 512e but a 512n disk (ie, it has a 512 byte physical
sector size). I don't believe there's an equivalent on ZoL, but I
haven't looked.

 This absolute insistence on ZFS's part is what makes ashift=9 vdevs so
dangerous today, because you cannot replace existing 512n disks in them
with 512e disks without (significant) hackery.

(Perhaps I'm misunderstanding what people mean by '512e' here; I've
been assuming it means a disk which reports 512 byte logical sectors and
4k physical sectors. Such disks are what you commonly get today.)

	- cks