Xen : OpenSolaris 2008.11 DomU running on a Linux Dom0

This post is a step by step explanation about how to get an OpenSolaris 2008.11 run as a Xen DomU on a Linux Dom0.
To follow this, you’ll need a Linux machine ready for Xen (I run Xen 3.3.0), with vncviewer installed.

This post doesn’t explain the basics of Xen, so you might want to start by learning Xen if you don’t already know a bit of it.

Step 1 : Download OpenSolaris 2008.11

To download the Installation/Live CD-ROM image, I highly recommend using BitTorrent… it is much faster and anyway, I’ve never managed to get a complete download through HTTP.
The torrent file is available from the download page of OpenSolaris 2008.11.

Step 2 : Extract the OpenSolaris Kernel and Ramdisk from the CD image

First mount the ISO image :

roadrunner osol_vm # mount -o loop,ro osol-0811.iso mnt/
roadrunner osol_vm # ls mnt/
bin   dev      jack  pkg.zlib  proc         root  solaris.zlib      system
boot  devices  mnt   platform  reconfigure  save  solarismisc.zlib  tmp
roadrunner osol_vm #

Then copy the following files to some place on the Dom0 :

roadrunner osol_vm # cp mnt/{platform/i86xpv/kernel/unix,boot/x86.microroot} /files/vservers/kernels/

Step 3 : Create a disk image to install OpenSolaris onto

We’ll create a 4Gb file as the root disk we’ll install OpenSolaris on :

roadrunner osol_vm # dd if=/dev/zero of=root.img bs=4G count=1

Step 4 : Install OpenSolaris

Step 4.1 : Create a Xen VM configuration file for the installation

This configuration will be used only for installing OpenSolaris. It boots from the CD image you downloaded. Here is the configuration file I used to proceed with the installation :

roadrunner osol_vm # cat osol_install_cdrom.py
name = "osol-install"
vcpus = 1
memory = "1024"
bootloader = "/usr/bin/pygrub"
kernel = "/platform/i86xpv/kernel/unix"
ramdisk = "/boot/x86.microroot"
extra = "/platform/i86xpv/kernel/unix -B console=ttya,livemode=text"
disk = ['file:/files/vservers/osol_vm/osol-0811.iso,6:cdrom,r',
'file:/files/vservers/osol_vm/root.img,0,w']
vif = ['']
on_shutdown = "destroy"
on_reboot = "destroy"
on_crash = "preserve"
roadrunner osol_vm #

Step 4.2 : Start the OpenSolaris DomU for installing

roadrunner backup # xm create -c osol_install_cdrom.py
Using config file "./osol_install_cdrom.py".
Started domain backup
v3.3.0 chgset 'unavailable'
SunOS Release 5.11 Version snv_101b 32-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing live image for use
[...]

Then answer the questions about the language / keyboard etc … and login as the user jack with the password “jack”.

Step 4.3 : Launch the installer

There is unfortunately no text mode installer at the moment … we’ll need to take extra steps in order to be able to have a graphical display in which to run the installer. We’ll use VNC for this purpose.

First let’s collect the IP address of the DomU (because I am using DHCP, if you are not, then I guess you’ll have to setup networking properly at this point) :

jack@opensolaris:~$ ifconfig xnf0
xnf0: flags=201004843 mtu 1500 index 2
inet 10.253.21.157 netmask ffffff00 broadcast 10.253.21.255
jack@opensolaris:~$

The IP address here is 10.253.21.157.

Now let’s start the VNC server on the OpenSolaris DomU and pick up a password :

jack@opensolaris:~$ mkdir .vnc && cp .Xclients .vnc/xstartup && vncserver

You will require a password to access your desktops.

Password:
Verify:
xauth:  creating new authority file /jack/.Xauthority

New 'opensolaris:1 ()' desktop is opensolaris:1

Starting applications specified in /jack/.vnc/xstartup
Log file is /jack/.vnc/opensolaris:1.log

jack@opensolaris:~$

(notice that the display number is “:1”)

Now you can access the graphical display of the OpenSolaris DomU on the Dom0 with vncviewer

kattoo@roadrunner /home/kattoo $ vncviewer 10.253.21.157:1
Connected to RFB server, using protocol version 3.8
Performing standard VNC authentication
Password:
Authentication successful
Desktop name "opensolaris:1 ()"
[...]

There you go, a VNC window should now pop up, with the graphical environment of the OpenSolaris DomU.

Click on the desktop icon “Install OpenSolaris” to start the installer and follow the simple steps to get it installed on the disk image.

Once finished, you’ll have a fully installed OpenSolaris on ZFS filesystems in the disk image.

Step 5 : Start the installed OpenSolaris from the disk image

Now that the OpenSolaris system is installed on the disk image, we can use that to boot from. As of OpenSolaris 2008.11, the root volume is ZFS. I didn’t manage to get Pygrub to boot from ZFS, that’s why we started by collecting the Kernel and ramdisk at the Step 2.

Here is the Xen configuration file for the installed OpenSolaris DomU :

roadrunner osol_vm # cat osol.py
name = "osol"
vcpus = 1
memory = "1024"
kernel = "/files/vservers/kernels/unix"
ramdisk = "/files/vservers/kernels/x86.microroot"
extra = '/platform/i86xpv/kernel/unix -B console=ttya,zfs-bootfs=rpool/52,bootpath="/xpvd/xdf@0:a"'
disk = [ 'file:/files/vservers/osol_vm/root.img,0,w' ]
vif = ['']
on_shutdown = "destroy"
on_reboot = "destroy"
on_crash = "preserve"
roadrunner osol_vm #

You can start this with :

roadrunner backup # xm create -c osol.py
Using config file "./osol.py".
Started domain osol
module /platform/i86xpv/kernel/unix: text at [0xf4c00000, 0xf4cd7e03] data at 0xf5000000
module /kernel/genunix: text at [0xf4cd7e08, 0xf4e9f6f7] data at 0xf5068bc0
v3.3.0 chgset 'unavailable'
SunOS Release 5.11 Version snv_101b 32-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
features: 630e66c6
mem = 1048576K (0x40000000)
Using default device instance data
root nexus = i86xpv
pseudo0 at root
pseudo0 is /pseudo
scsi_vhci0 at root
scsi_vhci0 is /scsi_vhci
xpvd0 at root
xdf@0, xdf0
xdf0 is /xpvd/xdf@0
/xpvd/xdf@0 (xdf0) online
xdf@0: 16777216 blocksroot on rpool/ROOT/opensolaris fstype zfs
/cpus (cpunex0) online
pseudo-device: dld0
dld0 is /pseudo/dld@0
xencons@0, xencons0
xencons0 is /xpvd/xencons@0
cpu0: x86 (GenuineIntel 6FB family 6 model 15 step 11 clock 2400 MHz)
cpu0: Intel(r) Core(tm)2 Quad CPU    Q6600  @ 2.40GHz
[...]

That’s it.

Thanks to VirtualGuru for pointing in the right direction.