[OmniOS-discuss] dedup causes zfs/omnios to drop connections.

[Dan McDonald]

> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi all,
>  
> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online.
> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash.
>  
> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. 

Dedup is a memory pig and not very well implemented in ZFS.  I'd highly recommend against it in production.  Either that, or really increase your memory for your system in question.  There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled.

Sorry,
Dan