[OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade

[Michael Mounteney]

Sorry to take so long to get back to you Tobias and I hope this is
still relevant.  As described elsewhere in this list, I had temporarily
to downgrade ssh to achieve interoperability between the OmniOS (bloody)
host and the Gentoo Linux guests.

First, ssh imposes some overhead:

mounty at pantry ~ $ time ssh people exit

real	0m0.724s
user	0m0.032s
sys	0m0.012s

that real figure averages around the 0.750s mark.  So I decided to
perform much bigger transfers to minimise its effect:

mounty at pantry ~ $ dd if=/dev/zero bs=1M count=2000 | ssh people dd of=/dev/null
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 138.436 s, 15.1 MB/s
4096000+0 records in
4096000+0 records out
2097152000 bytes transferred in 137.657582 secs (15234555 bytes/sec)

mounty at pantry ~ $ ssh people dd if=/dev/zero bs=1M count=2000 | dd of=/dev/null
2000+0 records in
2000+0 records out
2097152000 bytes transferred in 51.692313 secs (40569901 bytes/sec)
4096000+0 records in
4096000+0 records out
2097152000 bytes (2.1 GB) copied, 52.4503 s, 40.0 MB/s

It is puzzling that the in and out figures are so different but I did
perform each test three times and the results were approximately the
same each time.  On the read-off-disk test, here are all three runs:

pantry ~ # dd if=/dev/vda of=/dev/zero bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 65.3406 s, 16.0 MB/s
pantry ~ # dd if=/dev/vda of=/dev/zero bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.19789 s, 875 MB/s
pantry ~ # dd if=/dev/vda of=/dev/zero bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.85877 s, 564 MB/s

which I've quoted to show that the disk must be cached.  So I tried
again with more data to eliminate that effect:

pantry ~ # dd if=/dev/vda of=/dev/zero bs=1M count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 710.215 s, 15.1 MB/s

I hope that's helpful.

Michael.