Boot on Real Hardware

This document currently describes the infrastructure we use at ETH to boot Barrelfish on real hardware. You may want to use this as a starting point and do something similar.

We use a combination of TFTP/GRUB/PXEBOOT/Multiboot to transfer binaries to a new machine and boot the Barrelfish kernel:

1. The BIOS does DHCP and TFTP to load GRUB. The file that the DHCP server instructs it to load is called tftpboot.<hostname> (this should be a symlink to a suitable GRUB image, probably the one named pxegrub-undi which uses the PXE/UNDI network stack).

2. GRUB, running its internal preset menu file (see below), switches the console to serial mode and loads the menu file menu.lst.<dot-quad-ip> from the TFTP server, where <dot-quad-ip> is substituted by the IP address of the booting machine.

3. The menu file contains entries to load the kernel for that specific machine.

Deployment

There is a standard make install target that deposits your kernel image and applications into /home/netos/tftpboot/<username>/.

Serial output

All Barrelfish output (kernel debug and userspace printf()) currently is redirected to serial. By default it uses port 0x3f8 and IRQ 3. In order to see any output, you need to have a serial connection to the machine you are booting. Also note that some BIOS may require configuration and enabling the serial port beforehand. Naming may be different from BIOS to BIOS but some of the settings that are typically important are Baud rate, Flow control and the port:

Some weird machines do not have a standard serial port 0 on 0x3f8. In that case you can override the port barrelfish uses by passing serial=<portnumber> to the kernel (binary cpu) and serial driver (binary serial) as an argument in your menu.lst

Background on Bootloaders

The following gives a little background on our decisions on bootloaders and shows how to recreate our boot environment from scratch (or source).

GRUB Legacy

GRUB has extensive support for network boot, which GRUB 2 currently does not fully support. This is why we opt for GRUB for the moment.

The standard distribution of GRUB supports the Multiboot specification, but does not yet support ELF64 image or switching to long mode. However, a patch is available to add that support. Furthermore, there is no support for using the PXE UNDI network stack, nor for Intel e1000 NICs in the standard distribution, but another patch is available to enable PXE UNDI support and fix this issue. Finally, in order to support machine-specific menu.lst files, you need the patch grub-0.97-undi-menu.lst-patch.bz2.

Building GRUB

Both the current release version 0.97 and the current CVS version of GRUB work. You have to apply the 64-bit patch grub-0.97-x86_64-fix.patch as follows:

cd /location/of/unpacked/grub/sources
patch -p1 /location/of/patch/file

Basic/QEMU version

To build GRUB for QEMU type:

./configure --enable-ns8390 CFLAGS=-O2
make

--enable-ns8390 enables standard NE2000 support. Don't forget to build with -O2! This ensures that network drivers can use __outb as a builtin. The build will stop without it.

To build a GRUB with built-in menu, use an additional --enable-preset-menu=/home/netos/projects/barrelfish/tools/grub/menu.lst. That file's contents are as follows:

#
# This GRUB menu file first tries to load a local configuration file via TFTP.
# If that fails, it automatically sets up TFTP to load the Barrelfish kernel
# over the network, with only the default Multiboot modules loaded.
#

serial
terminal --timeout=0 serial
dhcp
timeout 0
default 0
fallback 1
hiddenmenu

title           Load /menu.lst from TFTP, if it exists
root            (nd)
configfile      /menu.lst

title           Barrelfish (fallback w/ default Multiboot modules)
root            (nd)
kernel          /kernel/kernel
module          /usr/init/init

It will first try to load a local menu.lst file from the root of the TFTP hierarchy. If that fails, the kernel will be booted without any special parameters or Multiboot modules loaded.

Making a Bootfloppy Image

The subdirectory util of the source tree holds a file grub-image after the build completes. Run this as root to generate the floppy image. The final image will be called grub-0.97-i386-pc.ext2fs, of course assuming you used version 0.97 as the base.

Network boot

To build a PXE-bootable GRUB for the Intel boxes, first create a presetmenu file, with the following contents:

serial --speed=115200
terminal --timeout=0 serial
default 0
timeout 0
title Load config from DHCP
dhcp --with-configfile-mac

This tells GRUB that as soon as it is loaded, it should enable the serial port, then load the machine-specific menu.lst file from the TFTP server.

If your NIC is not supported by standard GRUB (like for our Intel boxes), apply the PXE UNDI patch grub-0.97-os.3.diff.gz as follows:

cd /location/of/unpacked/grub/sources
gunzip -c /location/of/patch/file | patch -p0

NOTE: With the PXE UNDI patch applied, GRUB might need an older GCC (version 3.4 works) version to build. GCC 4.1 produces errors during build.

To make GRUB support machine-specific menu.lst files, apply the patch grub-0.97-undi-menu.lst-patch.bz2 (after having applied the PXE UNDI patch!) as follows:

cd /location/of/unpacked/grub/sources
bunzip2 -c /location/of/patch/file | patch -p1

Now, build GRUB (using GCC 3.4) and tell it to use this as the preset menu file:

./configure --enable-preset-menu=presetmenu --enable-diskless --enable-pxe --enable-pci-direct --disable-ext2fs --disable-fat --disable-ffs --disable-ufs2 --disable-minix --disable-reiserfs --disable-vstafs --disable-jfs --disable-xfs --disable-iso9660 --disable-md5-password --enable-serial CFLAGS=-O2 CC=gcc-3.4
make

Finally, take the file stage2/pxegrub -- this is your bootloader image!

The Multiboot Specification

Our kernel supports the Multiboot specification. As such, it is a regular ELF64 binary image.