Running Barrelfish on the Xeon Phi

This page describes how to build and run Barrelfish on the Intel Xeon Phi co-processor.

Environment Setup

In order to build Barrelfish for the Xeon Phi the user must have setup the GCC toolchain for the K1OM architecture and a NFS share accessible from the machine with the Xeon Phis. In summary, the Intel MPSS SDK has to be installed on the build host.

A detailed environment setup guide can be found here: Environment Setup Guide.

Compiling for Xeon Phi architecture

The system architecture for the Xeon Phi depends on host-based components as well as software running on the Xeon Phi co-processor. Hence, we need to configure Hake with two targets:

$ ../barrelfish/hake/hake.sh -s ../barrelfish -a k1om -a x86_64
$ make

This will generate the Xeon Phi boot image and the boot loader

Installing The Xeon Phi Boot Image

During the installation the boot image will be copied to the location specified in the symbolic targets file.

# symbolic_targets.mk
BARRELFISH_NFS_DIR ?="emmentaler.ethz.ch:/local/nfs/barrelfish/xeon_phi"

Booting The Xeon Phi

In order to boot a your Xeon Phi, you will need a network card with is supported by Barrelfish. Further, your host machine needs access to the NFS share where the Xeon Phi bootimage gets copied to. Following a sample menu.lst to boot the Xeon Phi cards.

# host menu.lst
title   Barrelfish
root    (nd)
kernel  /x86_64/sbin/elver loglevel=4
module  /x86_64/sbin/cpu loglevel=4
module  /x86_64/sbin/init

# Domains spawned by init
module  /x86_64/sbin/mem_serv
module  /x86_64/sbin/monitor

# Special boot time domains spawned by monitor
module  /x86_64/sbin/ramfsd boot
module  /x86_64/sbin/skb boot
modulenounzip /skb_ramfs.cpio.gz nospawn
module  /x86_64/sbin/kaluga boot
module  /x86_64/sbin/acpi boot
module  /x86_64/sbin/spawnd boot
module  /x86_64/sbin/startd boot
module /x86_64/sbin/routing_setup boot

# Device Managers
module /x86_64/sbin/xeon_phi_mgr

# Drivers
module /x86_64/sbin/pci auto skb_bridge_program=bridge_bios
module /x86_64/sbin/corectrl auto
module /x86_64/sbin/xeon_phi auto -nfs=nfs://<IP>/path/to/barrelfish/xeon_phi/ 

## For networking
module /x86_64/sbin/netd auto
module /x86_64/sbin/e1000n auto noirq
module /x86_64/sbin/NGD_mng auto

## add your domains here

# co-processor menu.lst
title   Barrelfish
root    (nd)
kernel  /k1om/sbin/cpu loglevel=4
module  /k1om/sbin/init

# Domains spawned by init
module  /k1om/sbin/mem_serv
module  /k1om/sbin/monitor

# Special boot time domains spawned by monitor
module  /k1om/sbin/ramfsd boot
module  /k1om/sbin/skb boot
module  /k1om/sbin/xeon_phi boot
module  /k1om/sbin/spawnd boot
module  /k1om/sbin/startd boot

# drivers
# specify the number of cores you want to boot
module /k1om/sbin/corectrl boot 1:55

#add your domains here

# GDDR Memory we have 6GB on our Xeon PHi
mmap map 0x0000000000 0x00FEE00000 1
# memory hole for the APIC and the flash rom 
mmap map 0x00FEE00000 0x120000 3
mmap map 0x0100000000 0x80000000 1