[OmniOS-discuss] Potential KVM Virtio Performance Issues

[John Barfield]

Actually the numbers I sent for the SmartOS VM to VM test were on a switch with Jumbo frames (switch = 9216 mtu...SmartOS GZ MTU = 9000) (Extreme Networks Summit X440-48t release 15.2.3 patch12) Theyre also sitting in Q-in-Q tagged VLANs. Admin tagged nic sits in Vman (provider bridge) 10 the VMs were tagged in VLAN 1674. (not bad :) really)

As far as everyone who is wondering how I got 952 Mbps on OmnisOS KVM while running in a zone I plan to write up a how-to that can be posted to the core site if you'd like. There are several caveats that are not documented today for running KVM in a zone. Not that I didnt reverse engineer some of Joyents work of course.



Thanks and have a great day,

John Barfield

> On Mar 24, 2015, at 7:40 PM, Phil Harman <phil.harman at gmail.com> wrote:
> 
> John,
> 
> Interesting work and data. Thanks for sharing.
> 
> I've also been playing with Oracle Solaris 11.2 vs Linux vs FreeBSD on SmartOS vs ESX5.5 (free edition) both VM2VM and VM to remote host over a couple of Intel 10GBASE-T cards.
> 
> As far as I can tell, there remains no virtio-net driver for Solaris / Illumos guests, so I've been using e1000g, which really sucks.
> 
> I found virtio-net works ok under KVM, but was blown away by vmxnet3 under ESX performance (for which a Solaris / Illumos drivers do exist), being able to get close to 8gbps from the guest over the wire!
> 
> To achieve this I had to use jumbo frames (something the current Solaris 11.2 e1000g appears unable to do at all any more).
> 
> So I was wondering, while you are there, whether you've got (or can get) any data for KVM virtio-net VM2VM using jumbo frames?
> 
> My expectation would be at least 2x for MTU 9000 vs 1500.
> 
> I also wonder whether like for like comparison with ESX might encourage further improvements?
> 
> As someone used to say at Sun "If Linux is faster, it's a Solaris bug!". It would be great if the community could agree to the same for ESX vs KVM :)
> 
> Cheers,
> Phil
> 
> 
>> On 24 Mar 2015, at 23:45, John Barfield <john.barfield at bissinc.com> wrote:
>> 
>> Btw I did go ahead and test both virtio methods...I gave a vm the -device argument on one interface and the -net argument for another the results where....
>> 
>> -device = eth0 = 952mbps
>> -net = eth1 = 199 mbps
>> 
>> Thanks and have a great day,
>> 
>> John Barfield
>> 
>>> On Mar 24, 2015, at 6:12 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>> 
>>> 
>>>> On Mar 24, 2015, at 7:04 PM, Dominik Hassler <hasslerd at gmx.li> wrote:
>>>> 
>>>> Dan,
>>>> 
>>>>>> After further testing I achieved 952 MBytes on a VM-2-VM
>>>>>> connection...1
>>>>>> linux Ubuntu 12.04 vm to another CentOS 6.6 VM running on two
>>>>>> different SmartOS host machines (through an extreme networks switch).
>>>> 
>>>> if I got John correctly, he was running his second test on SmartOS hosts...
>>>> 
>>>> We did a lot of testing on OmniOS with -net vnic and -device
>>>> virtio-net-pci but sadly to no avail...
>>>> 
>>>> I think we have to hope that SmartOS kvm improvements will get
>>>> upstreamed sooner or later.
>>> 
>>> Ahh yes.
>>> 
>>> I was hoping to have them ready for 014, but it's a complicated process to upstream larger projects, and Joyent was in the middle of getting their new Triton release out the door.
>>> 
>>> Dan
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss