[OmniOS-discuss] Ang: Re: Ang: Re: CoreOS

Sat Dec 20 13:36:50 UTC 2014

Hey Johan,

I just poked at the qemu image... it seems it wants some stuff not in 
our old qemu-kvm fork. e.g. fsdev (mouting a filesystem from host to 
guest).

But let's try anyway!

# convert qcow2 to raw
qemu-img convert coreos_production_qemu_image.img 
coreos_production_qemu_image.dd
# dump this on our zvol
dd if=coreos_production_qemu_image.dd 
of=/dev/zvol/rdsk/core/vms/hosts/coreos/disk0

We now have the correctly formatted data on our zvol...

On the plus side it does output nicely to ttya if added to a vm :)

So... here is where the kernel dies: (oh it does some kexec bits which 
are a PITA)
---
[    0.001000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.17.2 #2
[    0.001000] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[    0.001000]  0000000000000008 ffff88007a3e7db8 ffffffff814e8915 
000000000000e
[    0.001000]  ffffffff81798190 ffff88007a3e7e38 ffffffff814e4c97 
0000000000006
[    0.001000]  0000000000000008 ffff88007a3e7e48 ffff88007a3e7de8 
00000000fffb0
[    0.001000] Call Trace:
[    0.001000]  [<ffffffff814e8915>] dump_stack+0x46/0x58
[    0.001000]  [<ffffffff814e4c97>] panic+0xc1/0x1f5
[    0.001000]  [<ffffffff81b10205>] setup_IO_APIC+0x7d6/0x83d
[    0.001000]  [<ffffffff81b0d0cb>] native_smp_prepare_cpus+0x2bc/0x337
[    0.001000]  [<ffffffff81afc1a1>] kernel_init_freeable+0xcd/0x212
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000]  [<ffffffff814daf3e>] kernel_init+0xe/0xf0
[    0.001000]  [<ffffffff814ef8fc>] ret_from_fork+0x7c/0xb0
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000] Rebooting in 60 seconds..
---

I actually also have this on a ubuntu vm I am using, it needs noapic 
kernel option... on the grub prompt (really nice is coreos seems to have 
grub + console on both tty0 (vga) and ttyS0 (serial ttya).

Woo hoo we got past that bit where it fails on IO-APIC, now we just hang 
on smpboot :(

---
[    0.001000] CPU: Physical Processor ID: 0
[    0.001000] CPU: Processor Core ID: 0
[    0.001000] mce: CPU supports 10 MCE banks
[    0.001000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.001000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.001000] Freeing SMP alternatives memory: 20K (ffffffff82fa1000 - 
fffffff)
[    0.001000] ftrace: allocating 19518 entries in 77 pages
[    0.001000] smpboot: CPU0: Intel QEMU Virtual CPU version 0.14.1 
(fam: 06, m
---

Pretty much stuck here... I tried some variations of cpu type (qemu64, 
Nehalem and host) I also tried using one vcpu but still stuck.

Let's just cripple the entire thing and plow are way through: adding 
'nosmp noapic noacpi'

So yeah at this point coreos is pretty useless... but we fly past 
smpboot!
And... land here:

---
[    0.239823] scsi host0: ata_piix
[    0.239823] scsi host1: ata_piix
[    0.239823] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 
irq 14
[    0.239823] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 
irq 15
---

If it is docker you want, you may as well look at SmartOS's LX Brand 
stuff, they are racing towards workable docker zones.

But since I came this far, lets see if I can make it to the finish...
I am using virtio... lets try scsi... nothing... ide... nothing...

So this is were it ends.

Our qemu-kvm fork is probably just too old.

Regards

Jorge

On 2014-12-20 12:47, Johan Kragsterman wrote:
> Hi, Jorge and all!
> 
> 
> I would be interested in discussing this further, but perhaps
> omnios-discuss isn't the right place? Since I don't know if this is
> omnios/illumos/coreos specific...
> 
> I did some experimenting:
> 
> I only used CoreOS stable in my tests.
> 
> I tried the iso, but the iso isn't full featured, and doesn't run
> docker out of the box. And the docker implementation is of coarse what
> everybody is interested in. I got it to boot without problems, but I
> had big problems with VNC keymapping due to my Swedish keyboard and
> perhaps my Swedish client computer. So I could actually never do
> something with it, and since it is not full featured, it is not what I
> want to use.
> 
> So instead, I downloaded the img file for qemu, created a volume, and
> dd'ed the image to the volume, and then set this volume as boot. That
> went fine, to get it to boot. But then, with the default boot option
> in grub, it panicked, and restarted every 60 seconds.
> 
> I stopped the grub booting, and chosed the B option. That didn't
> panic, but it didn't work either, it was too much that didn't work.
> But option A went fine, no panic, and everything seem to work more or
> less without problems. The only problem here seem to be that I can't
> log in, due to the "first log in"-principles they seem to have: It is
> only possible to log in via ssh, which means the network have to be
> up, and I couldn't get the network to come up....so there I am right
> now...
> 
> Regards Johan
> 
> 
> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
> Från: Jorge Schrauwen <sjorge+ml at blackdot.be>
> Datum: 2014-12-18 18:07
> Kopia: omnios-discuss at lists.omniti.com
> Ärende: Re: Ang: Re: [OmniOS-discuss] CoreOS
> 
> 
> On 2014-12-18 17:57, Johan Kragsterman wrote:
>> Jorge, I was thinking about you when I posted this! I thought you
>> would be a possible contributor to this thread...  More furhter
>> down...
>> 
>> 
>> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
>> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
>> Från: Jorge Schrauwen <sjorge+ml at blackdot.be>
>> Datum: 2014-12-18 17:38
>> Kopia: omnios-discuss at lists.omniti.com
>> Ärende: Re: [OmniOS-discuss] CoreOS
>> 
>> Something like this will probably work:
>> 
>> 
>>   /usr/bin/qemu-system-x86_64
>>    -name coreos \
>>    -enable-kvm \
>>    -no-hpet \
>>    -m 4096
>>    -cpu Nehalem \
>>    -smp sockets=1,cores=4,threads=2 \
>>    -rtc base=utc,driftfix=slew \
>>    -pidfile /tank/coreo/coreos.pid  \
>>    -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay  \
>>    -vga std  \
>>    -vnc :1  \
>>    -nographic \
>>    -drive
>> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
>>    -drive
>> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on
>> \
>>    -boot order=cd,once=d \
>>    -device
>> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
>> \
>>    -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
>>    -chardev
>> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
>>    -serial chardev:serial0 \
>>    -usb \
>>    -usbdevice tablet \
>>    -daemonize
>> 
>> You should get vnc at port 5901, seemed to boot for me but I did not
>> complete the install.
>> 
>> 
>> 
>> At the CoreOS site they say: Start like this:
>> 
>> ./coreos_production_qemu.sh -nographic
>> 
>> and they pass on that string -nographic  ...?
>> 
>> It makes me wonder, because they tell you to connect with the instans
>> only over ssh with: ssh -l core -p 2222 localhost   ...
>> 
>> So I'm not sure if it is possible to connect via VNC...did you
>> actually check VNC, to confirm you had a VNC connection?
>> 
>> It should boot and run from the image r/o, so perhaps you just need
>> one "disk"? I can see you got two configured, or at least the iso
>> file, and then a disk. Don't you think it would be enough with just
>> the image file?
>> 
>> Perhaps I just try...
> I used the install iso to see if it booted.
> -nographic just mean don't spawn a graphical console AKA SDL or simular
> window. It does not prevent '-vnc :1' from working.
> 
>> 
>> 
>> 
>> Regards
>> 
>> Jorge
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> On 2014-12-18 16:57, Johan Kragsterman wrote:
>>> Hi!
>>> 
>>> 
>>>  I've been looking at CoreOS and finds it interesting! Since I'd like
>>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
>>> Haven't tested yet, but I downloaded the startscript for qemu, and it
>>> looks a little bit "too much" for Illumos KVM...
>>> 
>>>  It would be nice to get some views on people that have been
>>> considering this as well, perhaps some already tested or already
>>> running...?
>>> 
>>>  I've seen that Frederic Alix on this list been blogging about it, 
>>> but
>>> haven't seen if he managed to run it as a KVM guest on OmniOS.
>>> 
>>>  For me it seems to be some complications at first startup, mainly. 
>>> It
>>> doesn't seem to be reachable by VNC...
>>> 
>>>  Hope to get some input from you guys...
>>> 
>>> 
>>> Best regards from/Med vänliga hälsningar från
>>> 
>>> Johan Kragsterman
>>> 
>>> Capvert
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss