After I discovered GitLab CI, I was a huge fan of Docker and its containers. I wanted to dig deeper into the business of virtualization and look at alternative approaches. Recently, I thought I’d give Xen a try. I’m also rather new to CentOS, but I planned to setup a CentOS dom0 and a para-virtualized domU anyway. It didn’t go as planned. Whenever I tried to boot the installer for the guest OS with

$ virt-install --name centos7 \
        --connect xen:/// \
        --disk /var/xen/images/centos7.img,size=50 \
        --memory=2000 \
        -p \
        --nographics \
        --location=http://mirror.centos.org/centos-7/7.4.1708/os/x86_64/

I received an error starting with something like

[    4.586097] ------------[ cut here ]------------
[    4.586108] WARNING: CPU: 0 PID: 143 at mm/vmalloc.c:131 vmap_page_range_noflush+0x2c1/0x350

This was really puzzling to me. What I was trying to do was basic and I expected that everything works out-of-the-box. It turned out that this has been already reported as a bug, which (according to the discussion at the bug report) will be eventually fixed in CentOS 7.5. Until that version of the OS is released, the solution seems to be to use the CentOS’ kernel-plus repositories which use a fixed version of the kernel. However, this task is rather cumbersome if this is your first Xen+CentOS installation.

Using the information posted by PryMar56 on the bug report and by Johnny Hughes and PJ Welsh on the CentOS-virt mailing list, I will explain in the following how to install CentOS 7.4 with kernel-plus as a Xen PV guest. The post is intended for people who are new to Xen and CentOS.

As discussed in the threads mentioned above, the problem stems from a bug in the kernel used in CentOS 7.4. CentOS/RHEL’s policy is to not update the kernel of a released version of the OS. It is, however, possible to install the centos-plus repository which ships with a newer version of the kernel. So, the task is

  1. to modify the installer such that it boots with the kernel-plus, and
  2. to install the kernel-plus by default, such that the installed OS is able to boot with the new kernel.

Throughout this guide, I’m using mirror.certos.org. You should probably replace these parts with another mirror.

1. Setting up the installer

First, we need to make sure that lorax is installed. Lorax is a tool to build boot images. We will use it to build the anaconda (the centos installer) boot image with the kernel-plus.

$ yum install lorax
$ lorax -V    
lorax-19.6.92-1

The default build options are specified in /usr/share/lorax/runtime-install.tmpl. Open this file with your favorite text editor and change the line installpkg kernel to

installpkg kernel-plus

To build the image, disable SElinux with setenforce 0 and run

DEFAULTKERNEL=kernel-plus lorax  -p CentOSMinimal -v7 -r 7.4 \
    -s http://mirror.centos.org/centos-7/7.4.1708/os/x86_64/ \
    -s http://mirror.centos.org/centos-7/7.4.1708/centosplus/x86_64/ \
    centos7_installer

Lorax loads the required packages from the CentOS mirror, builds the installer images and puts the result in the directory centos7_installer. It is generally a bad idea to deactivate SElinux. While judging for yourself, if you can make an exception, consider that we are still in permission mode, so any violation will be logged and you can switch to enforcement mode immediately after the lorax done.

All the files in the output directory have to be available during the installation process. The easiest method is to expose them via HTTP. If you are not in a hostile network environment, you can make the files available with

$ cd centos7_installer
$ python -m SimpleHTTPServer

For later reference, let’s assume the IP address of this host where we publish the installer image is 10.1.1.100. It is not necessary to run the server on your dom0. This can be any other server on your network. Check that files are accessible by directing your browser to http://10.1.1.100:8000/. You might have to modify your firewall settings to allow traffic to go to port 8000.

2. Install guest OS with kernel-plus

If we run virt-install now with the --location argument pointing to our newly created installer image, the installer should boot. However, if we then proceeded, we would install the guest OS with the default kernel.

We can configure the installation process with a so-called kickstart file. Create a new directory centos7_kickstart which we will later also expose via HTTP. Open centos7_kickstart/kickstart.cfg and enter

url --url=http://mirror.centos.org/centos-7/7.4.1708/os/x86_64/
repo --name=centosplus --baseurl=http://mirror.centos.org/centos-7/7.4.1708/centosplus/x86_64/

%packages
@core
kernel-plus
%end

This file instructs anaconda to install the kernel-plus package.

Again, we can expose this file via HTTP on port 8001. Port 8000 might be in use by the other simple HTTP server.

$ cd centos7_kickstart
$ python -m SimpleHTTPServer 8001

Finally, we can create the guest machine with a custom --location argument to use our installer image with plus kernel and a custom --extra-args to tell anaconda about our kickstart file.

$ virt-install --name centos7 \
        --connect xen:/// \
        --disk /var/xen/images/centos7.img,size=50 \
        --memory=2000 \
        -p \
        --nographics \
        --location=http://10.1.1.100:8000/ \
        --extra-args="ks=http://10.1.1.100:8001/kickstart.cfg"

During the installation (I recommend to start VNC to have more options), make sure that the centos-plus repository is enabled. Furthermore, point the main installation source to a public mirror, e.g. http://mirror.centos.org/centos-7/7.4.1708/os/x86_64/.

Side nodes

While working on this document, I stumbled over other inconveniences

Use VNC

In all the examples, I have used --nographics, because my dom0 is not running an X server. Connecting with VNC from another computer over a local network usually doesn’t work out-of-the-box, because the packages are not forwarded to the domU. You can, however, use port forwarding

ssh -f -N -L 5901:${INTERNAL_DOM_U_IP}:5901 ${LOCAL_DOM_0_IP}

to forward any traffic to a local port to the domU via ssh of the dom0.

Boot partition

By default, the installer used xfs for the boot partition. Every time I tried to reboot after the installer finishes, pygrub complained that it could not read the machine image or could not find the boot partition. Switching to ext4 via VNC (or potentially via the kickstart file) solved the issue for me.