[OmniOS-discuss] Poor read performance on fresh zpool

[gijs at in2ip.nl]

Hi,

on a OmniosCE r151022 system we have rebuild our zfs pool. The pool is 
constructed out of 18 mirror vdevs, each consisting of 2 1,8TB SAS 
drives. Since about half the disks are 512b and the other half are 512e 
(physical 4k) we opted to use 4k sectors using sd.conf for all devices; 
thus all vdevs report as ashift 12. During testing today we experienced 
extremely poor read performance. Doing sequential reads with dd of the 
pool results in a read rate of 300MB/s, benchmarking with bonnie++ gives 
about 600MB/s
Write performance is as expected, around 1,4GB/s.
Scrub also performs as expected, zpool status shows a scrub speed of 
2GB/s, iostat -x shows a total pool speed of 4GB/s.

Any hints as to what might be causing our poor read performance?

Sincerely,

Gijs Peskens