24 Aug 2006

notes about gentoo on ubuntu with initrd

i use ubuntu on a machine that i use for some contract work. rest assure, that is the only linux machine that is not running gentoo that i admin.

everything people say about ubuntu's installation is true, it is incredible how easy it is to install ubuntu. in fact, those people who say they accidentally fell over and installed ubuntu are not lying. i did hit one snag when i tried installing it -- i downloaded the wrong CD.

i wanted to install ubuntu on an LVM because of the nature of work i do. turns out that i downloaded and booted up with "desktop" install disc rather than downloading the "server" disc. i spent half an hour looking up documentation in which people claim that they see an LVM partitioning option that i couldn't find. i only realised this mistake after i relented and just installed it on a regular partition and left the rest of the space for the LVM.

Xen on Ubuntu

ubuntu works fine for me, and apt-get is neat until you need the latest version of something, then you have to figure out how to make your own debs. but trying to get xen-unstable to work on it was a fairly trying task. there is a lot that i can go into, but it all boils down to this fact:

"mkinitrd on ubuntu makes a cramfs image that the vanilla + xen kernel sources do not understand. instead use mkinitramfs or yaird!"

Gentoo on Ubuntu

of course, now that i have a xen-enabled ubuntu, i wanted to start some VMs to really see it in action. there are some ready made domU (the terminology for the VM) images that have gentoo on them from places such as jailtime. but where is the fun in that. so i made a LVM and extracted a gentoo amd64 stage3 into it, followed the gentoo quick install guide (it's not as quick as it used to be any more, i think there is scope for the gentoo install cheatsheet, but that is for another day.)

i hit a couple of snags while trying to get a gentoo domU working on ubuntu, so here are some tips if you too are trying to particular setup, mainly dealing with the fact that the default xen-unstable kernel compiles everything as modules, so you will need to use an initrd:

1. use mkinitramfs rather than yaird when making the domU initrd.

problem with yaird is that it hardcodes the root device in the init script rather than getting it from the kernel boot params. mkinitramfs does the right thing, but it isn't quite enough. (see below)

2. do not include every module under the sun in your initrd

when making an initrd for your domU, be careful not to use the default mkinitramfs settings on ubuntu which just includes all your modules into the initramfs, which results in a 6M ramdisk image. that is just too big and will result in a kernel panic when you boot up your domU.

so try manually listing the modules by replacing MODULES=most with MODULES=list in /etc/mkinitramfs/mkinitramfs.conf and manually list the modules you think the domU will need /etc/mkinitramfs/modules. remember it is a domU, so it hardly needs any of the drivers apart from the bare minimum.

3. ubuntu's initrd script will conflict with gentoo's startup

ubuntu's init script in initrd will mount /proc and /sys when it mounts the root partition, and gentoo's init does not like that at all. i personally think that gentoo's init should be able to cope with that, but there may be a good reason why it doesn't. the solution here is to add an extra cleanup script just before it boots into the gentoo rootfs. i had to add a script in /etc/mkinitramfs/scripts/init-bottom, which as the name suggests, runs the script at the end of the whole init process.

#!/bin/sh -e

# Output pre-requisites
prereqs() {
echo "$PREREQ"

case "$1" in
exit 0

umount /sys
umount /proc
# clean up after ubuntu's initramfs
umount ${rootmnt}/dev/.static/dev

4. ubuntu's utf-8 vs gentoo's ascii terminals

ubuntu does the right thing (in my opinion) to have utf-8 in alot of places. one snag i hit was that my terminals were all supporting utf-8 encoding while gentoo's default terminal encoding was ascii, this meant that certain punctuation were appearing as weird characters. the solution here is to get into /etc/rc.conf and change UNICODE to equal "yes".

You can reply to me about this on Twitter: