ometimes you can stumble upon interesting discovers while actually looking for something else. That happened to me when I decided to try another Linux distribution besides my favorite one: Debian. I have used Debian since 1998 when I was a budding Java developer trying to install a Java Virtual Machine under Linux. Back then, Debian 2.1 (slink) was the only distribution that allowed me to do this. I’ve used Debian ever since and have become an experienced Debian user and administrator who enjoys solving a lot of usability challenges manually.

All along I have resisted the lure of other Linux distributions and I believe I did not miss much during these years. However, as Ubuntu increasingly became a worldwide phenomenon, I eventually decided to try it but didn’t want to leave Debian behind. So I decided to use virtualization: I would run Linux (Ubuntu) on Linux (Debian).

Nowadays, virtualization solutions are everywhere; some run directly over BIOS, others under Linux or Windows. I discovered that even the Linux kernel offers a way to run itself with the User Mode Linux framework (UML). UML is a nested Linux instance (called guest) running inside another Linux instance (called host). The guest Linux will run in user space as a standard application, which prevents any operation in the guest from causing a crash in the host system.

The aim of this article is to present my experience building and running two separate UML instances of Ubuntu distributions. For Debian users like me, it can serve as a starting point for understanding UML’s potential and exploring the recent Ubuntu 7.10 (Gutsy Gibbon) release without moving away from their familiar Debian environment.

UML Building Blocks
In order to run my Linux user space, I had to:

1. Configure a Linux host system.
2. Prepare a standard and complete Linux filesystem and then customize it.
3. Compile the special executable Linux kernel to run inside the Linux host.

I used Linux kernel version 2.6.24.2, the stable one at the time of this writing. For the creation of the UML root filesystem, I needed the support of a loopback device and ext3 filesystems. So I entered this in the kernel configuration menu:

Device Drivers  --->     [*] Block devices  --->          <*>   Loopback device supportFile systems  --->     <*> Ext3 journaling file system support     [*]   Ext3 extended attributes

Creating and running multiple Linux distributions inside a Linux host system was not enough, however. I needed to reach them on behalf of a true network connection. (It is no accident that UML is good for network simulation and study; it can be configured and interact like a network in several ways.) I used the Universal TUN/TAP device driver, described as follows in file tuntap.txt in the Linux kernel tree:

“It provides packet reception and transmission for user space programs. It can be seen as a simple Point-to-Point or Ethernet device, which, instead of receiving packets from physical media, receives them from user space program and instead of sending packets via physical media writes them to the user space program.“

TUN/TAP support makes it possible to establish virtual network interfaces. TUN or TAP interfaces behave as an IP or Ethernet network interface or as a character device, respectively. In my case, I created a TAP instance on the host machine for Ethernet frames exchange, talking with the guest Ethernet network interface on one side and with the host kernel on the other. In this way, the guest system sees an eth0 device because it had a real Ethernet card (i.e., its eth0 device file will talk with the host Linux system TAP device).

Of course I need TUN/TAP support on both host and guest Linux kernels:

Device Drivers  --->     [*] Network device support  --->             Universal TUN/TAP device driver support

Next, I had to decide if I wanted the host to act as a switch connecting different network segments on the basis of TCP addresses or on the basis of MAC addresses. In the former case, the host becomes a router (Layer 3 switch), in the latter, an Ethernet bridge (Layer 2 switch). I chose the latter solution, assuming that I had additional available IP addresses for my Ethernet LAN. I also assumed that my LAN had a machine acting as a router to exit on to the Internet. Of course, the kernel host machine had to support Ethernet bridging:

Networking  --->     Networking options  --->           802.1d Ethernet Bridging

Moreover, I needed the bridge-util package installed on the host machine:

apt-get install bridge-util

The Linux Host Setup
The following is the setup for my scenario:

• Machines in my LAN (host network) have IP addresses such as 192.168.0.X.
• The host IP address is 192.168.0.2.
• The two UML guest machines are named Uml1 and uml2 and their addresses are 192.168.0.201 and 192.168.0.202, respectively.
• The Internet gateway in the LAN has the IP address 192.168.0.1.

I had to add a bridge instance in the host machine using brctl, the utility provided by the bridge-util package for Ethernet bridge administration:

brctl addbr the_bridge

You can set other bridge parameters with the brctl utility as well (such as settings for variables affecting the Spanning Tree protocol), but these are related to complex network topologies and are beyond the scope of this article.

I then started up the bridge interface as I do with a normal network interface, by calling it the_bridge and assigning the IP address with a relative netmask that I would assign to the host if it were normally on the LAN:

ifconfig the_bridge 192.168.0.2 netmask 255.255.255.0 up

Once the bridge was up (you can check it with ifconfig), it was like I had a physical Ethernet switch with sockets to connect the other machines in the network. The first machine I needed to connect was the host itself. Every connection to the bridge consists of the configuration of a standard eth0 interface as a promiscuous one with the IP address 0.0.0.0, which has to be successively added to the bridge:

ifconfig eth0 0.0.0.0 promisc upbrctl addif the_bridge eth0

Then I needed to add my default gateway to access the Internet:

route add default gw 192.168.0.1

Figure 1. Logic Schema of Network Solution: I had to plug in all the other UML machines that I wanted to power on inside the Linux host.

Next, I had to plug in all the other UML machines that I wanted to power on inside the Linux host. Each UML in the host required that I:

• create the TAP interface to be associated to the network interface eth0 of UML and assign it to a particular user (host-user);
• configure and rise up the TAP as a promiscuous interface;
• add the TAP interface to the bridge.

For the first and second UML machines, I run as follows (see Figure 1):

tunctl -u host-user -t the_uml_conn1ifconfig the_uml_conn1 0.0.0.0 promisc upbrctl addif the_bridge the_uml_conn1tunctl -u host-user -t the_uml_conn2ifconfig the_uml_conn2 0.0.0.0 promisc upbrctl addif the_bridge the_uml_conn2

The UML Root Filesystem
The installation of a Linux system always requires the creation of a consistent filesystem (called root filesystem) that contains a standard directories tree. Because I did not use any standard installation procedure, I needed to create this filesystem separately. A lot of programs are available for this task; the Debian one, debootstrap, works perfectly when used with an Internet connection. Ubuntu is derived from Debian, so it allows the use of the same package manager and offers the same software packages.

I enclosed the entire filesystem in a 4GB file (uml1-root or uml2-root), and created a clean one on the host machine as root user:

dd if=/dev/zero of=uml1-root bs=4096 seek=1M count=1

I then formatted this as ext3 filesystem:

mkfs.ext3 uml1-root

Next, I made it available via a loopback device by mounting it under /mnt as a loopback filesystem:

mount -o loop uml1-root /mnt

Next, I installed the debootstrap application and used it to install a complete root filesystem for Ubuntu 7.10 (Gutsy Gibbon) under /mnt:

apt-get install debootstrapdebootstrap --arch i386 gutsy /mnt http://us.archive.ubuntu.com/ubuntu

After that, I had a basic but complete Linux filesystem. Before booting uml1, however, I had to customize it to provide a better start.

UML runs as a normal user process of the Linux host system, so it doesn’t talk directly to hardware as a normal Linux system does. Instead, it talks with the hardware presented by the Linux host system. One great advantage of this is that you can run UML on whatever virtual hardware configuration, regardless of the real one.

I accomplished the virtual device assignment in the command line by associating a variable switch. For example, UML block devices are defined by the ubd variables (i.e., ubd0 for the first block device, ubd1 for the second one, and so on). With the assignment ubd0=uml1-root in the command line, I associated the loopback filesystem containing the root filesystem and the first UML block device; so I needed to edit /mnt/etc/fstab in this way:

/dev/ubd0     /     ext3     defaults     0 1proc  /     proc     proc     defaults     0 0

And then create the device with the right permissions:

cd /mnt/devmknod --mode=660 ubd0 b 98 0chown root:disk ubd0

For the machine name, I put the name of the machine in the file /mnt/etc/hostname and created a plain vanilla file /mnt/etc/hosts with only this line:

127.0.0.1 localhost

Ubuntu, like Debian, uses the ifupdown package to configure and deconfigure network interfaces. So I could edit the file /mnt/etc/networking/interfaces to have the UML eth0 device automatically up at boot time. For uml1:

auto loiface lo inet loopbackauto eth0iface eth0 inet static    address 192.168.0.201    netmask 255.255.255.0    network 192.168.0.0    broadcast 192.168.0.255    gateway 192.168.0.1

I had to configure tty0 to be a UML terminal allowed to login as root and erase the other ones:

echo "tty0" >> /mnt/etc/securettyecho "ttys/0" >> /mnt/etc/securettyrm /mnt/etc/event.d/tty2rm /mnt/etc/event.d/tty3rm /mnt/etc/event.d/tty4rm /mnt/etc/event.d/tty5rm /mnt/etc/event.d/tty6

By the way, there is a little difference between standard Debian and Ubuntu here. Ubuntu replaced the famous init daemon with the new upstart framework, and you have to edit files under the /etc/event.d directory instead of modifying the file /etc/inittab.

I needed to erase the standard rule of the udev framework that recreates a new ethX device every time the system boots:

rm /mnt/etc/udev/rules.d/75-persistent-net-generator.rules

When the umount /mnt filesystem has assigned the rights to host-user, that will boot UML instances.

Compiling and Booting the Linux Executable
Just by installing the Debian package user-mode-linux, I already had a compiled UML kernel ready to boot in user space. (I suggest compiling the kernel by yourself to better understand some concepts of UML.) However, I had to install the uml-utilities packages for UML administration and configuration. Under the home directory of a normal user, I got and extracted the last available kernel as follows:

wget –c http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.2.tar.bz2tar –xvjf linux-2.6.24.2.tar.bz2

If you are familiar with Linux kernel compiling, you will recognize all the commands in the remainder of the article, with the exception of the variable setting ARCH=um. This is an important setting that tells the compiler how to build the kernel for the special user-mode architecture. In other words, instead of compiling the kernel for the x86 or ARM CPU architecture, I obtained a kernel running for the user-mode virtual architecture (Jeff Dike, the UML creator, defines it as a port of Linux to Linux) with this command:

make ARCH=um defconfig

I then had a standard configuration of UML that I changed only a little with this command :

make ARCH=um menuconfig

At that point, I completed the following steps to create my Linux executable file:

1. Have no modules (otherwise I would have to install the compiled modules in the right place under UML root filesystem):

   [ ] Enable loadable module support  --->

2. Set the right processor in:

   Host processor type and features  --->

3. Don’t include all other network support except TUN/TAP:

   UML Network Devices  --->[*] Virtual network device[ ]   Ethertap transport [*]   TUN/TAP transport[ ]   SLIP transport[ ]   Daemon transport[ ]   VDE transport[ ]   Multicast transport[ ]   pcap transport[ ]   SLiRP transport

4. Disable the support for PPP:

   [*] Network device support  --->[ ]   PPP (point-to-point protocol) support

5. Check whatever option under section:

   Kernel hacking  --->

   Author’s Note: For the sake of completeness, I’ll mention the SKAS (Separate Kernel Address Space) kernel patch, which solves some security and performance issues of UML. The standard way is called TT (Tracing Thread) mode.

6. Then finally:

   make ARCH=um linux

At the end of the compilation, I had a file called linux that was the executable file to run.

As I already explained, in the command line I can configure the hardware platform upon which I want to run UML. For the first UML, I set these switch variables:

• ubd0=/path/to/file/uml1-root?This sets the block device from which the system mounts the root filesystem.
• con=pty con0=null,fd:2 con1=fd:0,fd:1?These settings deal with the configuration of console terminals.
• eth0=tuntap,the_uml_conn1?This is the association of the UML eth0 network interface with the host TAP network interface. A random and fictitious MAC address will be assigned to eth0.
• mem=512M?With this variable you can set the physical RAM memory that the UML will have, regardless to the actual existing one in the host machine.
• umid=uml1-umid?This is a unique identification label for the first UML machine used by some host UML utilities and also by the debugger.

There are a lot of other variables to set; you can read about them on the man pages.

Now I was ready to boot, launching the command:

linux-2.6.24.2/linux      ubd0=/path/to/file/uml1-root      con=pty con0=null,fd:2      con1=fd:0,fd:1      eth0=tuntap,the_uml_conn1      mem=512M      umid=uml1-umid

See Listing 1 for the complete boot log of first UML.

Enjoying the Full Ubuntu Experience
Remember that to this point I had built only the basic root filesystem for Ubuntu 7.10. To experience the typical Ubuntu desktop, I had to install it. Nothing has ever been easier because I was already connected to my LAN and I could just run the command:

apt-get install ubuntu-desktop

This operation took a long time to complete because it required the installation of many packages. It is also a memory-consuming process, so because the host system uses tmpfs abstraction, I had to make it 1GB large at least. I accomplished this by putting the following line in the host file:

tmpfs     /tmp     tmpfs     size=5000M     0     2

In addition, I had to install Xnest X server because I needed a complete remote X session on the host:

apt-get install xnest

I also had to allow the remote X sessions on the host:

xhost +

And finally in the UML I ran these two commands:

Xnest :1 &gnome-session --display=:1 &

At this point, the Ubuntu desktop popped up in a window of my Debian host (see Figure 2).

Figure 2. UML Ubuntu Sessions Running: You can see the two UML Ubuntu sessions running under my Debian host system.

Playing in the Sandbox
The availability of a Linux kernel running in user space allows you to develop programs in complete safety, as well as hack and debug the kernel itself without any risk. In the past month, the vmsplice local root exploit alarmed the Linux community and they fixed it in kernel version 2.6.24.2. Let’s test this exploit with the code that was available in the community (see Listing 2.) What better place to try it than on a UML machine?

For this purpose, download and install a vulnerable kernel version (e.g., 2.6.24.1) and compile a UML machine following the steps above with the following two additional settings for debugging purposes:

Kernel hacking  --->     [*] Compile the kernel with debug info     [*] Compile the kernel with frame pointers

Then boot it, access it as a normal user, compile, and run the code:

uml1-user@uml1:~$ gcc -g -o test proof-of-concept.c uml1-user@uml1:~$ ./test

UML will freeze, harmlessly with regard to your host system:

----------------------------------- Linux vmsplice Local Root Exploit By qaaz-----------------------------------[+] mmap: 0x0 .. 0x1000[+] page: 0x0[+] page: 0x20[+] mmap: 0x4000 .. 0x5000[+] page: 0x4000[+] page: 0x4020[+] mmap: 0x1000 .. 0x2000[+] page: 0x1000[+] mmap: 0x40173000 .. 0x401a5000

You also can debug it from the host with gdb if you know the umid of the crashed UML. Find the pid of the UML in the filedirectory .uml/uml1-umid/pid and then run:

host-user@host:~$ gdb linux-2.6.24.1/linux pid_number(gdb) where#0  0xffffe410 in __kernel_vsyscall ()#1  0xb7dcb2c9 in sigprocmask () from /lib/i686/cmov/libc.so.6#2  0x080648c0 in change_sig (signal=11, on=1) at arch/um/os-linux/signal.c:186#3  0x08066d1a in sig_handler_common_skas (sig=11, sc_ptr=0x81f4d24) at arch/um/os-Linux/skas/trap.c:38#4  0x08064620 in sig_handler (sig=11, sc=0x81f4d24) at arch/um/os-Linux/signal.c:49#5  0x0806478f in handle_signal (sig=136268828, sc=0x81f4d24) at arch/um/os-Linux/signal.c:136#6  0x08065b07 in hard_handler (sig=11) at arch/um/os-Linux/sys-i386/signal.c:12#7  #8  0x080c01a8 in splice_to_pipe (pipe=0x27488600, spd=0x27e02ec8) at fs/splice.c:257#9  0x080c14d4 in vmsplice_to_pipe (file=0x8f95360, iov=0x0, nr_segs=4096, flags=) at fs/splice.c:1462#10 0x00001000 in ?? ()

During and after the boot of UML, you might have noticed a lot of annoying messages coming from the console similar to this:

line_ioctl: tty0: unknown ioctl: 0x541e

Digging into the kernel source code in file arch/um/drivers/line.c, I found the piece of code that originates this kind of message:

if (i == ARRAY_SIZE(tty_ioctls)) {     printk(KERN_ERR "%s: %s: unknown ioctl: 0x%x
",     __FUNCTION__, tty->name, cmd);}

It doesn’t seem to be important, so comment out the code and see what happens.

This operation allowed me to investigate a piece of kernel code without the fear of crashing the host system?a demonstration of the benefit of having a Linux kernel running in user space.