[OmniOS-discuss] oracle vm server in kvm enabled zone

Thu Aug 11 23:15:25 UTC 2016

Agreed, we run the db software in zones for several customers, no problem there, but most have Linux, and that's where kvm/Ovm pops in. 
As well for the fancy point and click provisioning.

Verstuurd vanaf mijn iPhone

> Op 11 aug. 2016 om 22:50 heeft Jim Klimov <jimklimov at cos.ru> het volgende geschreven:
> 
> 11 августа 2016 г. 12:36:35 CEST, pieter van puymbroeck <pietervanpuymbroeck at hotmail.com> пишет:
>> Hi Jim,
>> 
>> 
>> Basically it's a personal experiment. I'm an oracle dba and I test
>> everything in my home-lab before talking to customers.  I'm very used
>> to ovm. But I can't use that because I want to stick with one physical
>> box, so the kvm enabled zones are exactly what I need.
>> 
>> 
>> You're 100% right that everything I want to do it possible directly in
>> kvm as well except one thing. A self service portal on which I can
>> prepare my environments and that they get cleaned up automatically
>> after the tests, that's the main drive to use ovm.
>> 
>> 
>> I do like the solaris (well omnios) zones that much that I use them for
>> the rest of my home network (web server, dns, mysql,... the usual
>> stuff) but at this point I did not search good enough I think, but I'm
>> looking for something that I can click in a webpage "I want such a vm,
>> that os, that storage, that oracle version and a test db yes or no".
>> 
>> You're right I can start scripting it by building a pxe server,
>> preparing the images and build a bunch of ansible / puppet / bash
>> scripts which kick off when needed, but it's a lot of work and I was
>> thinking, I can build it easily with cloud control communicating to
>> ovm, so why shouldn't it work.
>> 
>> And to a certain height,it does work. The ovm servers are running
>> wonderfully stable, except the kvm's don't allow nesting and that's
>> what I need [��]
>> 
>> 
>> If you have alternatives for such kind of selfservice portal (personal
>> use, so must be quick and easy to configure!!!) I love to hear it, then
>> the problem is solved as well.
>> 
>> Best regards,
>> Pieter
>> ________________________________
>> From: Jim Klimov <jimklimov at cos.ru>
>> Sent: Thursday, August 11, 2016 8:57 AM
>> To: pieter van puymbroeck; Jim Klimov
>> Cc: Michael Rasmussen; omnios-discuss at lists.omniti.com
>> Subject: Re: [OmniOS-discuss] oracle vm server in kvm enabled zone
>> 
>> 10 августа 2016 г. 19:51:38 CEST, pieter van puymbroeck
>> <pietervanpuymbroeck at hotmail.com> пишет:
>>> Clear.
>>> So I don't have to expect it soon.
>>> Pity, because I love this Solaris fork and besides hobying (for work),
>>> a use case might be to make abstraction of the underlying architecture
>>> (Intel/amd/...).
>>> But ok, I didn't make a mistake it's just not possible using omnios,
>>> right?
>>> 
>>> 
>>> 
>>> 
>>> 
>>>>> Op 10 aug. 2016 om 19:12 heeft Jim Klimov <jimklimov at cos.ru> het
>>>> volgende geschreven:
>>>> 
>>>> 10 августа 2016 г. 16:52:56 CEST, pieter van puymbroeck
>>> <pietervanpuymbroeck at hotmail.com> пишет:
>>>>> Isn't it supported in omnios?
>>>>> 
>>>>> It is supported in different linux flavours, but I don't like to go
>>> on
>>>>> that way.
>>>>> 
>>>>> 
>>>>> I read on the omniti website that kvm is one of the key-features,
>> so
>>>>> nesting was something which I'd expected to be there. Don't get me
>>>>> wrong, the not nested kvm is working perfectly (and fast!).
>>>>> 
>>>>> 
>>>>> Is there an estimated timeline in which it should be supported or a
>>>>> place to launch enhancement requests?
>>>>> 
>>>>> 
>>>>> ________________________________
>>>>> From: Michael Rasmussen <mir at miras.org>
>>>>> Sent: Wednesday, August 10, 2016 2:48 PM
>>>>> To: pieter van puymbroeck; omnios-discuss at lists.omniti.com
>>>>> Subject: Re: [OmniOS-discuss] oracle vm server in kvm enabled zone
>>>>> 
>>>>> Seem to remember nested kvm is not supported.
>>>>> 
>>>>> On August 10, 2016 3:53:11 PM GMT+02:00, pieter van puymbroeck
>>>>> <pietervanpuymbroeck at hotmail.com> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> Currently I'm building a new homelab based on omnios
>>> (omnios-r151018)
>>>>> 
>>>>> There a kvm enabled zone is created and inside the zone I have 3
>>>>> kvm-guests running. These guests are running oracle vm (xen based).
>>> All
>>>>> is running fine except if I want to start a linux-testvm inside
>>> oracle
>>>>> vm, the system complains about:
>>>>> 
>>>>> 
>>>>> Error: HVM guest support is unavailable: is VT/AMD-V supported by
>>> your
>>>>> CPU and enabled in your BIOS?
>>>>> 
>>>>> 
>>>>> Inside the kvm enabled zone, kvm is enabled:
>>>>> 
>>>>> root at ovm1:/root# ls -l /dev/kvm
>>>>> 
>>>>> crw------- 1 root sys 125, 0 Aug 10 12:48 /dev/kvm
>>>>> 
>>>>> root at ovm1:/root# modinfo |grep -i kvm
>>>>> 
>>>>> 207 fffffffff840a000  3a030 125   1  kvm (kvm driver v0.1)
>>>>> 
>>>>> root at ovm1:/root#
>>>>> 
>>>>> 
>>>>> For sure I checked the processor itself from the global zone if it
>>> is
>>>>> supported:
>>>>> 
>>>>> # isainfo -v
>>>>> 
>>>>> 64-bit amd64 applications
>>>>> 
>>>>> vmx pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp cx16 sse3 sse2
>>>>> 
>>>>> sse fxsr mmx cmov amd_sysc cx8 tsc fpu
>>>>> 
>>>>> 32-bit i386 applications
>>>>> 
>>>>> vmx pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp ahf cx16 sse3
>>>>> 
>>>>> sse2 sse fxsr mmx cmov sep cx8 tic fpu
>>>>> 
>>>>> #
>>>>> 
>>>>> 
>>>>> 
>>>>> and to me it looks I have the required flags available.
>>>>> 
>>>>> inside the lvm-guest the flags are indeed not given to the guest:
>>>>> 
>>>>> 
>>>>> [root at ovm2 ~]# uname -s -r -i -p
>>>>> 
>>>>> Linux 2.6.39-400.279.1.el5uek x86_64 x86_64
>>>>> 
>>>>> [root at ovm2 ~]# cat /etc/oracle-release
>>>>> 
>>>>> Oracle VM Server release 5.7
>>>>> 
>>>>> [root at ovm2 ~]# grep flags /proc/cpuinfo
>>>>> 
>>>>> flags : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush
>> mmx
>>>>> fxsr sse sse2 ss syscall nx lm constant_tsc up nopl pni ssse3 cx16
>>>>> sse4_1 sse4_2 popcnt aes hypervisor lahf_lm
>>>>> 
>>>>> [root at ovm2 ~]#
>>>>> 
>>>>> 
>>>>> it does not matter if I start the kvm guest using
>>>>> 
>>>>> -cpu host \
>>>>> 
>>>>> or using
>>>>> 
>>>>> -cpu qemu64,+vmx,+aes,+sse4.2,+sse4.1,+ssse3 \  # --> this one is
>>>>> faster
>>>>> 
>>>>> 
>>>>> despite I have an intel cpu I use the -enable-nested flag as well,
>>> but
>>>>> it doesn't seem to make a difference.
>>>>> 
>>>>> 
>>>>> How can I fix this?
>>>>> 
>>>>> 
>>>>> Thanks and best regards,
>>>>> 
>>>>> Pieter
>>>>> 
>>>>> ________________________________
>>>>> 
>>>>> OmniOS-discuss mailing list
>>>>> OmniOS-discuss at lists.omniti.com
>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> OmniOS-discuss Info Page - lists.omniti.com Mailing
>> Lists<http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>> lists.omniti.com
>> OmniOS is a minimalist, self-hosting, Illumos-based distribution aimed
>> at server deployments. Release and update announcements are also made
>> to this list.
>> 
>> 
>> 
>>>>> OmniOS-discuss
>>>>> Info Page - lists.omniti.com Mailing
>>>>> Lists<http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>>>>> lists.omniti.com
>>>>> OmniOS
>>>>> is a minimalist, self-hosting, Illumos-based distribution aimed at
>>>>> server deployments. Release and update announcements are also made
>>> to
>>>>> this list.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> ------------------------------------------------------------------------
>>>>> 
>>>>> _______________________________________________
>>>>> OmniOS-discuss mailing list
>>>>> OmniOS-discuss at lists.omniti.com
>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>> 
>>>> AFAIK this is a common limitation with x86 HW hypervisors that only
>>> the barebone system may grab and lock the CPU extensions that are the
>>> HW hypervisor implementation. Guest OSes may only do unaccelerated
>>> software emulation for their sub-guests.
>>>> 
>>>> For example, to use VirtualBox on Windows (Servers) you must ensure
>>> that HyperV is disabled and its drivers are not loaded at all.
>>>> 
>>>> Jim
>>>> --
>>>> Typos courtesy of K-9 Mail on my Samsung Android
>> 
>> Hi Pieter,
>> 
>> Is there a particular reason for you to stack hypervisors? Why run
>> guests under OVM under KVM, and not guests in KVM directly? (Or if
>> compatibility of this KVM port is insufficient for your usecase,
>> perhaps try VirtualBox instead)?
>> 
>> As for "others" - this is the first time I hear of having accelerated
>> second-layer hypervisors, so the rest is my educated guesswork and pure
>> speculation. Granted, I do not have much experience with KVM nor Linux
>> hypervisors lately... so my guess would be that certain code
>> intentionally plays well together (e.g. top-level KVM providing or
>> wrapping access to CPU VT-* extensions to guest KVMs), but this is not
>> likely a universal solution that lets everyone play with anyone (KVM,
>> HyperV, VirtualBox, VMWare, ...), that Linux or everyone else has and
>> only Solaris/illumos lacks. Or maybe a lot happened in upstream since
>> the port was done ;)
>> 
>> Jim
>> --
>> Typos courtesy of K-9 Mail on my Samsung Android
> 
> You can run Oracle DB for Solaris in zones ;) For the installer you can set up X11 (+vnc on server, or direct for workstation) and use `xhosts` to propagate the session into local zones.
> 
> For tests and shows you can pre-install a zone with maybe an empty database, shut it down, and then clone and tear down for practical short-lived purposes. Being zfs clones, this would involve minimal overhead on storage side.
> 
> Likewise, you can clone golden datasets with virtualbox vm's and rewrite virtual disk id's (documented) and machine manifests to attach a new/cloned vm to such cloned disk. Likely similar approach is doable with KVM. But this is a bit of scriptable or manual work...
> 
> But sorry, no point-and-click magic that I'd know of...
> 
> Jim
> --
> Typos courtesy of K-9 Mail on my Samsung Android