[OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012 upgrade

[Dan McDonald]

> 
> I have not tested disk performance explicitly, but even booting a
> windows host took ages ... so I suspect whatever is causing this
> influences all kvm guest IO.

What's really REALLY weird about this is that we did not alter anything about how we built KVM between these releases.

Tell me, can you run "lockstat sleep <time-of-test>" in the global zone while you run your KVM tests?  They will produce a lot of output, but they may be very informative about what's going on.

Also, I'd be curious if you might (BEs and rpool space being available) upgrade a BE to bloody and repeat your tests?

We don't have the facilities to stress out VMs like this, which is why we didn't notice this before 012 went out the door.  Clearly something's messing up KVM performance (you're not the first to report this, but you seem to have a decent environment for comparisons).  Before the next stable (and incidentally long-term-support as well) release, I hope to have these problems cleared up.  One thing that should happen soon is that Joyent is upstreaming the VND changes into illumos-gate, which will allow us to be fully caught up to their illumos-kvm-cmd source, which we've frozen at revision 1c6181be55d1cadc4426069960688307a6083131 since r151010.

Thanks, and I wish I could be of more immediate assistance!

Dan