[OmniOS-discuss] oracle vm server in kvm enabled zone

Thu Aug 11 08:57:48 UTC 2016

10 августа 2016 г. 19:51:38 CEST, pieter van puymbroeck <pietervanpuymbroeck at hotmail.com> пишет:
>Clear.
>So I don't have to expect it soon.
>Pity, because I love this Solaris fork and besides hobying (for work), 
>a use case might be to make abstraction of the underlying architecture
>(Intel/amd/...).
>But ok, I didn't make a mistake it's just not possible using omnios,
>right?
>
>
>
>
>
>> Op 10 aug. 2016 om 19:12 heeft Jim Klimov <jimklimov at cos.ru> het
>volgende geschreven:
>> 
>> 10 августа 2016 г. 16:52:56 CEST, pieter van puymbroeck
><pietervanpuymbroeck at hotmail.com> пишет:
>>> Isn't it supported in omnios?
>>> 
>>> It is supported in different linux flavours, but I don't like to go
>on
>>> that way.
>>> 
>>> 
>>> I read on the omniti website that kvm is one of the key-features, so
>>> nesting was something which I'd expected to be there. Don't get me
>>> wrong, the not nested kvm is working perfectly (and fast!).
>>> 
>>> 
>>> Is there an estimated timeline in which it should be supported or a
>>> place to launch enhancement requests?
>>> 
>>> 
>>> ________________________________
>>> From: Michael Rasmussen <mir at miras.org>
>>> Sent: Wednesday, August 10, 2016 2:48 PM
>>> To: pieter van puymbroeck; omnios-discuss at lists.omniti.com
>>> Subject: Re: [OmniOS-discuss] oracle vm server in kvm enabled zone
>>> 
>>> Seem to remember nested kvm is not supported.
>>> 
>>> On August 10, 2016 3:53:11 PM GMT+02:00, pieter van puymbroeck
>>> <pietervanpuymbroeck at hotmail.com> wrote:
>>> 
>>> Hello,
>>> 
>>> Currently I'm building a new homelab based on omnios
>(omnios-r151018)
>>> 
>>> There a kvm enabled zone is created and inside the zone I have 3
>>> kvm-guests running. These guests are running oracle vm (xen based).
>All
>>> is running fine except if I want to start a linux-testvm inside
>oracle
>>> vm, the system complains about:
>>> 
>>> 
>>> Error: HVM guest support is unavailable: is VT/AMD-V supported by
>your
>>> CPU and enabled in your BIOS?
>>> 
>>> 
>>> Inside the kvm enabled zone, kvm is enabled:
>>> 
>>> root at ovm1:/root# ls -l /dev/kvm
>>> 
>>> crw------- 1 root sys 125, 0 Aug 10 12:48 /dev/kvm
>>> 
>>> root at ovm1:/root# modinfo |grep -i kvm
>>> 
>>> 207 fffffffff840a000  3a030 125   1  kvm (kvm driver v0.1)
>>> 
>>> root at ovm1:/root#
>>> 
>>> 
>>> For sure I checked the processor itself from the global zone if it
>is
>>> supported:
>>> 
>>> # isainfo -v
>>> 
>>> 64-bit amd64 applications
>>> 
>>> vmx pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp cx16 sse3 sse2
>>> 
>>> sse fxsr mmx cmov amd_sysc cx8 tsc fpu
>>> 
>>> 32-bit i386 applications
>>> 
>>> vmx pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp ahf cx16 sse3
>>> 
>>> sse2 sse fxsr mmx cmov sep cx8 tic fpu
>>> 
>>> #
>>> 
>>> 
>>> 
>>> and to me it looks I have the required flags available.
>>> 
>>> inside the lvm-guest the flags are indeed not given to the guest:
>>> 
>>> 
>>> [root at ovm2 ~]# uname -s -r -i -p
>>> 
>>> Linux 2.6.39-400.279.1.el5uek x86_64 x86_64
>>> 
>>> [root at ovm2 ~]# cat /etc/oracle-release
>>> 
>>> Oracle VM Server release 5.7
>>> 
>>> [root at ovm2 ~]# grep flags /proc/cpuinfo
>>> 
>>> flags : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush mmx
>>> fxsr sse sse2 ss syscall nx lm constant_tsc up nopl pni ssse3 cx16
>>> sse4_1 sse4_2 popcnt aes hypervisor lahf_lm
>>> 
>>> [root at ovm2 ~]#
>>> 
>>> 
>>> it does not matter if I start the kvm guest using
>>> 
>>> -cpu host \
>>> 
>>> or using
>>> 
>>> -cpu qemu64,+vmx,+aes,+sse4.2,+sse4.1,+ssse3 \  # --> this one is
>>> faster
>>> 
>>> 
>>> despite I have an intel cpu I use the -enable-nested flag as well,
>but
>>> it doesn't seem to make a difference.
>>> 
>>> 
>>> How can I fix this?
>>> 
>>> 
>>> Thanks and best regards,
>>> 
>>> Pieter
>>> 
>>> ________________________________
>>> 
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> OmniOS-discuss
>>> Info Page - lists.omniti.com Mailing
>>> Lists<http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>>> lists.omniti.com
>>> OmniOS
>>> is a minimalist, self-hosting, Illumos-based distribution aimed at
>>> server deployments. Release and update announcements are also made
>to
>>> this list.
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> 
>>> 
>>>
>------------------------------------------------------------------------
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> 
>> AFAIK this is a common limitation with x86 HW hypervisors that only
>the barebone system may grab and lock the CPU extensions that are the
>HW hypervisor implementation. Guest OSes may only do unaccelerated
>software emulation for their sub-guests.
>> 
>> For example, to use VirtualBox on Windows (Servers) you must ensure
>that HyperV is disabled and its drivers are not loaded at all.
>> 
>> Jim
>> --
>> Typos courtesy of K-9 Mail on my Samsung Android

Hi Pieter,

Is there a particular reason for you to stack hypervisors? Why run guests under OVM under KVM, and not guests in KVM directly? (Or if compatibility of this KVM port is insufficient for your usecase, perhaps try VirtualBox instead)?

As for "others" - this is the first time I hear of having accelerated second-layer hypervisors, so the rest is my educated guesswork and pure speculation. Granted, I do not have much experience with KVM nor Linux hypervisors lately... so my guess would be that certain code intentionally plays well together (e.g. top-level KVM providing or wrapping access to CPU VT-* extensions to guest KVMs), but this is not likely a universal solution that lets everyone play with anyone (KVM, HyperV, VirtualBox, VMWare, ...), that Linux or everyone else has and only Solaris/illumos lacks. Or maybe a lot happened in upstream since the port was done ;)

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android