[OmniOS-discuss] Slow performance with ZeusRAM?

[Bob Friesenhahn]

On Thu, 22 Oct 2015, Matej Zerovnik wrote:
> Bob:
> Yes, my ZFS is ashift=12, since all my drives report 4k blocks (is that what you ment?). The pool is completly empty, so there is enough place for
> writes, so write speed should not be limited because of COW. Looking at iostat, there are no reads on the drives at all.
> I’m not sure where fio gets its data, probably from /dev/zero or somewhere?

To be clear, zfs does not overwrite blocks.  Instead zfs modifies (in 
memory) any prior data from a block, and then it writes the block data 
to a new location.  This is called "copy on write".  If the prior data 
would not be entirely overwritten and is not already cached in memory, 
then it needs to be read from underlying disk.

It is interesting that you say there are no reads on the drives at 
all.

>       Try umounting and re-mounting your zfs filesystem (or 'zfs destroy' followed by 'zfs create') to see how performance differs on a freshly
>       mounted filesystem.  The zfs ARC caching will be purged when the filesystem is unmounted.
> 
> If I understand you correctly, you are saying I should destroy my folders, set recordsize=4k on my pool and then create zfs folders?

It should suffice to delete the file and set recordsize=4k on the 
filesystem, and then use fio to create a new file.  The file retains 
its original recordsize after it has been created so you would need to 
create a new file.

There is maximum performance for random write if the write blocksize 
matches the filesystem blocksize.  There is still a catch though if 
the random write is truely random because the writes may still not 
match up perfectly to underlying blocks.  For example (top is logical 
write data and bottom is file block data):

aligned "random write" data:

XXXXXXXXXXXXXX  XXXXXXXXXXXXXX
XXXXXXXXXXXXXX  XXXXXXXXXXXXXX

vs unaligned "random write" data:

        XXXXXXXXXXXXXX   XXXXXXXXXXXXXX
XXXXXXXXXXXXXX  XXXXXXXXXXXXXX  XXXXXXXXXXXXXX

The writes aligned to the start of the underlying block will be much 
faster.

A major benefit of your ZIL device is to help turn random writes into 
fewer random writes or even sequential writes when the TXG is written 
to your data drives.

It is very difficult to test raw hardware performance through zfs 
filesytem access.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/