[OmniOS-discuss] Potential KVM Virtio Performance Issues

Fri Mar 27 08:36:35 UTC 2015

Hi John,

this sounds encurraging ...

could you provied complete details of your setup ...

* start up commandline for qemu-system-x86_64
* kernel version running on on your ubuntu
* any special config on the ubuntu side (eg in /etc/systctl.conf) or so ...

cheers
tobi

Yesterday John Barfield wrote:

> So I was still having issues with virtio performance. I?ve finally determined that its the child zone that is  capping the throughput at 85mbps.
>
> If I halt the zone and launch the same VM from the GZ I get 955mbps.
>
> Another thing?the virtio driver in Centos6.6 does not work well with OmniOS kvm.
>
> I can boot Centos from either the GZ or a CZ and I?m actually getting results in the Kb now instead of mbps with iperf. May have something to do with the tcp window being 19.5 kb on CentOS vs 85kb on Ubuntu. Assuming this is a driver problem.
>
> The only OS I get good speeds with are Ubuntu server 14.04 running the Global Zone. (Have only tested two though :))
>
> So my recipe for decent virtio performance on OmniOS:
>
> Ubuntu Linux Server 14.04 running in Global Zone.
>
> Does anyone have any idea why the child zone is capping my throughput?
>
> Am I missing a zone cfg parameter to allow the child zone to have full 1GB bandwidth?
>
>
>
> From: Theo Schlossnagle
> Date: Wednesday, March 25, 2015 at 6:56 AM
> To: John Barfield
> Cc: Phil Harman, "omnios-discuss at lists.omniti.com<mailto:omnios-discuss at lists.omniti.com>"
> Subject: Re: [OmniOS-discuss] Potential KVM Virtio Performance Issues
>
> +1 John.  That documentation would be very welcome.
>
> On Tue, Mar 24, 2015 at 9:50 PM, John Barfield <john.barfield at bissinc.com<mailto:john.barfield at bissinc.com>> wrote:
> Actually the numbers I sent for the SmartOS VM to VM test were on a switch with Jumbo frames (switch = 9216 mtu...SmartOS GZ MTU = 9000) (Extreme Networks Summit X440-48t release 15.2.3 patch12) Theyre also sitting in Q-in-Q tagged VLANs. Admin tagged nic sits in Vman (provider bridge) 10 the VMs were tagged in VLAN 1674. (not bad :) really)
>
> As far as everyone who is wondering how I got 952 Mbps on OmnisOS KVM while running in a zone I plan to write up a how-to that can be posted to the core site if you'd like. There are several caveats that are not documented today for running KVM in a zone. Not that I didnt reverse engineer some of Joyents work of course.
>
>
>
> Thanks and have a great day,
>
> John Barfield
>
> > On Mar 24, 2015, at 7:40 PM, Phil Harman <phil.harman at gmail.com<mailto:phil.harman at gmail.com>> wrote:
> >
> > John,
> >
> > Interesting work and data. Thanks for sharing.
> >
> > I've also been playing with Oracle Solaris 11.2 vs Linux vs FreeBSD on SmartOS vs ESX5.5 (free edition) both VM2VM and VM to remote host over a couple of Intel 10GBASE-T cards.
> >
> > As far as I can tell, there remains no virtio-net driver for Solaris / Illumos guests, so I've been using e1000g, which really sucks.
> >
> > I found virtio-net works ok under KVM, but was blown away by vmxnet3 under ESX performance (for which a Solaris / Illumos drivers do exist), being able to get close to 8gbps from the guest over the wire!
> >
> > To achieve this I had to use jumbo frames (something the current Solaris 11.2 e1000g appears unable to do at all any more).
> >
> > So I was wondering, while you are there, whether you've got (or can get) any data for KVM virtio-net VM2VM using jumbo frames?
> >
> > My expectation would be at least 2x for MTU 9000 vs 1500.
> >
> > I also wonder whether like for like comparison with ESX might encourage further improvements?
> >
> > As someone used to say at Sun "If Linux is faster, it's a Solaris bug!". It would be great if the community could agree to the same for ESX vs KVM :)
> >
> > Cheers,
> > Phil
> >
> >
> >> On 24 Mar 2015, at 23:45, John Barfield <john.barfield at bissinc.com<mailto:john.barfield at bissinc.com>> wrote:
> >>
> >> Btw I did go ahead and test both virtio methods...I gave a vm the -device argument on one interface and the -net argument for another the results where....
> >>
> >> -device = eth0 = 952mbps
> >> -net = eth1 = 199 mbps
> >>
> >> Thanks and have a great day,
> >>
> >> John Barfield
> >>
> >>> On Mar 24, 2015, at 6:12 PM, Dan McDonald <danmcd at omniti.com<mailto:danmcd at omniti.com>> wrote:
> >>>
> >>>
> >>>> On Mar 24, 2015, at 7:04 PM, Dominik Hassler <hasslerd at gmx.li<mailto:hasslerd at gmx.li>> wrote:
> >>>>
> >>>> Dan,
> >>>>
> >>>>>> After further testing I achieved 952 MBytes on a VM-2-VM
> >>>>>> connection...1
> >>>>>> linux Ubuntu 12.04 vm to another CentOS 6.6 VM running on two
> >>>>>> different SmartOS host machines (through an extreme networks switch).
> >>>>
> >>>> if I got John correctly, he was running his second test on SmartOS hosts...
> >>>>
> >>>> We did a lot of testing on OmniOS with -net vnic and -device
> >>>> virtio-net-pci but sadly to no avail...
> >>>>
> >>>> I think we have to hope that SmartOS kvm improvements will get
> >>>> upstreamed sooner or later.
> >>>
> >>> Ahh yes.
> >>>
> >>> I was hoping to have them ready for 014, but it's a complicated process to upstream larger projects, and Joyent was in the middle of getting their new Triton release out the door.
> >>>
> >>> Dan
> >> _______________________________________________
> >> OmniOS-discuss mailing list
> >> OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
> --
>
> Theo Schlossnagle
>
> http://omniti.com/is/theo-schlossnagle
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902