[OmniOS-discuss] Request advise on pool upgrade

[Andries Annema]

Hi Bob,

Appreciate your thoughts!

You are right that I should do something to rebalance the data across 
added space. Moving some datasets with zfs send| zfs receive should do 
the trick, right?

Smaller disks perform better on resilvering, sure. But I suspect there 
comes a time anyhow that I will want to expand once again, so avoiding 
disks larger than 4TB will not go indefinitely. Therefore I was thinking 
I might as well skip the step to add a 3rd vdev out of 4TB drives 
altogether.
Performance is not really an issue either. It has no difficulty 
saturating my gigabit-network as it is, and there are not a lot of users 
it has to serve either.

The proverb you are referring to is a wise one, and from the beginning 
it has been my plan to add a third vdev out of the same drives, but 
looking at the line of drives currently available plus the fact that I 
don't expect to be able to avoid touching the first and second vdev 
forever anyway, I concluded I might as well start rebuilding/upgrading 
the pool in place.

Andries


On 2018-02-07 1:02, Bob Friesenhahn wrote:
> On Tue, 6 Feb 2018, Andries Annema wrote:
>>
>> Right now I am leaning towards options 2 or 3, with maybe a little 
>> preference for option 3, I think.
>>
>> Any thoughts you guys can share on this matter? Would appreciate it. 
>> Thanks!
>
> Since your pool is already excessively full, adding a new vdev would 
> result in most of the freshly-written data being written there.  This 
> would result in reduced performance as compared to uniformly available 
> space in the vdevs.  There are things you can do after adding another 
> vdev to rebalance the data across the vdevs.
>
> Smaller size disks are better from a recovery/resilver standpoint. 
> More disks are better from a performance standpoint.  Replacing a 
> drive with 8TB of data would take a very long time.  4TB disks are 
> already huge.  Larger size disks are often slower than smaller disks.
>
> Lastly, there is always the "Let sleeping dogs lie" proverb, which 
> suggests that not touching existing disks is less likely to result in 
> trouble than adding additional disks.
>
> I prefer option 1 to add a 3rd vdev with the 4TB disks.
>
> Bob