Running Barrelfish on Gem5

There is support for booting Barrelfish on the gem5 simulator.

This provides a platform for experimenting with OS/architecture co-design, and for experimenting with architectural support for aspects of Barrelfish (e.g., message passing). It is not intended as a general-purpose platform for experimenting with Barrelfish -- QEMU is typically much faster.

Running ARMv7 Barrelfish on Gem5

Barrelfish for ARM on Gem5 supports up to 4 cores and caches. There are some issues with the O3 CPU model and multiple cores though. Below are instructions to set up gem5 and boot ARM Barrelfish on it.

1. Build Barrelfish for ARMv7 on Gem5

• Run <srcdir>/hake/hake.sh -s <srcdir> -a armv7 in your build directory

• Edit <builddir>/hake/Config.hs to say armv7_platform = "gem5" (change from armv7_platform = "pandaboard")

• Run make ARMv7-GEM5

2. Get the gem5 simulator with hg clone http://repo.gem5.org/gem5 -r 0fea324c832c gem5 (note that this is the commit we're currently using for the nightly builds)

3. Apply the following patch to gem5 (Thanks to Mehdi Amiri for providing it!)

diff --git a/SConstruct b/SConstruct
--- a/SConstruct
+++ b/SConstruct
@@ -510,7 +510,7 @@
 if main['GCC']:
     main.Append(CCFLAGS=['-pipe'])
     main.Append(CCFLAGS=['-fno-strict-aliasing'])
-    main.Append(CCFLAGS=['-Wall', '-Wno-sign-compare', '-Wundef'])
+    # main.Append(CCFLAGS=['-Wall', '-Wno-sign-compare', '-Wundef'])
     # Read the GCC version to check for versions with bugs
     # Note CCVERSION doesn't work here because it is run with the CC
     # before we override it from the command line
@@ -566,7 +566,7 @@
 
     main.Append(CCFLAGS=['-pipe'])
     main.Append(CCFLAGS=['-fno-strict-aliasing'])
-    main.Append(CCFLAGS=['-Wall', '-Wno-sign-compare', '-Wundef'])
+    #main.Append(CCFLAGS=['-Wall', '-Wno-sign-compare', '-Wundef'])
     main.Append(CCFLAGS=['-Wno-tautological-compare'])
     main.Append(CCFLAGS=['-Wno-self-assign'])
     # Ruby makes frequent use of extraneous parantheses in the printing
diff --git a/src/python/m5/__init__.py b/src/python/m5/__init__.py
--- a/src/python/m5/__init__.py
+++ b/src/python/m5/__init__.py
@@ -32,7 +32,7 @@
 
 try:
     # Try to import something that's generated by swig
-    import internal
+    import internal.core
 
     # Try to grab something from it in case demandimport is being used
     internal.core.curTick
diff --git a/src/python/m5/internal/__init__.py b/src/python/m5/internal/__init__.py
--- a/src/python/m5/internal/__init__.py
+++ b/src/python/m5/internal/__init__.py
@@ -26,9 +26,9 @@
 #
 # Authors: Nathan Binkert
 
-import core
-import debug
-import event
-import stats
-import trace
-from drain import DrainManager, Drainable
+#import core
+#import debug
+#import event
+#import stats
+#import trace
+#from drain import DrainManager, Drainable

4. Build gem5 with scons build/ARM/gem5.fast and add the binary to the PATH

5. Run the simulation (do not forget to add the directory build/ARM from within the gem5 directory to the PATH):

• make gem5_armv7

6. To get the output of Barrelfish you can use telnet localhost 3456. If you see (approximately) the following, you've successfully run the default configuration:

Telnet escape character is '^E'.
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^E'.
==== m5 slave terminal: Terminal 0 ====
Kernel starting at address 0x82d80000
Barrelfish CPU driver starting on ARMv7 Board id 0x000008e0
The address of paging_map_kernel_section is 0x82d86248
GIC: 3 interrupt lines detected
gic_init: done
gic_enable_interrupt for id=0x22, offset=0x4, index=0x1
gic_enable_interrupt for id=0x23, offset=0x8, index=0x1
start_free_ram = 0x2dd7000
spawn module: armv7/sbin/init
init: invoked as: init 2097152
Spawning memory server (armv7/sbin/mem_serv)...
Spawning monitor (armv7/sbin/monitor)...
monitor: invoked as: armv7/sbin/monitor 7725056
RAM allocator initialised, 397 MB (of 461 MB) available
Spawning ramfsd on core 0
ramfsd.0.0: pre-populating from boot image...
ramfsd.0.0: /armv7/sbin/cpu_arm_gem5 already exists, skipping it
ramfsd.0.0: ready
Spawning skb on core 0
Spawning /armv7/sbin/spawnd on core 0
Spawning /armv7/sbin/startd on core 0
Spawning /armv7/sbin/corectrl on core 0
spawnd.0 up.
startd.0.0: starting app /armv7/sbin/memtest on core 1
skb.0.0: waiting for: spawn.1
SYSFLAGSET_BASE is at 0x1c000000
.. mapped to 0xfea00000
.. using address 0x0xfea10030
Kernel starting at address 0x88701000
Barrelfish CPU driver starting on ARMv7 Board id 0x000008e0
The address of paging_map_kernel_section is 0x88707248
GIC: 3 interrupt lines detected
gic_init: done
spawn module: armv7/sbin/monitor
monitor: invoked as: armv7/sbin/monitor 0 chanid=0 archid=0
Spawning spawnd on core 1
spawnd.1 up.
spawnd.1.0: spawning /armv7/sbin/memtest on core 1
No bootscript
memtest passed successfully! on core 1

NOTE 1: To change the system configuration, adjust 'tools/arm_gem5/gem5script.py' accordingly or change the command to start gem5 in 'symbolic_targets.mk'

NOTE 2: If you use --cpu-type=arm_detailed (use make gem5_armv7_detailed), the simulation takes a long time (depending on your machine up to an hour just to boot Barrelfish)

NOTE 3: Don't worry about seeing the following warning as output of make gem5_armv7. warn: Trying to access SPSR in an invalid mode: 31

Old Instructions

Note: As of release2013-01-11, the patch described in step 3 is not needed anymore, and hence we are not bound to a specific version of gem5. At the time of this writting the latest known working version was 0fea324c832c.

1. Build Barrelfish for Arm on Gem5

• since release2012-10-03: use 'armv7' as a make target
• for earlier releases: use 'arm_gem5' instead

2. Get the gem5 simulator with 'hg clone http://repo.gem5.org/gem5 -r d45a02bd5391 gem5' (note you have to use this release to get a working environment out of the box)

3. Change to your gem5 directory and apply 'gem5_patches.patch', located in '/path/to/barrelfish/tools/arm_gem5' to the gem5 source code

4. Build gem5 with 'scons build/ARM/gem5.fast' and add the binary to the PATH NOTE: I had to use an older version of gcc, gcc 4.4 - 4.6 worked for me. If you have troubles, install gcc 4.4 and compile with 'CXX=g++-4.4 scons build/ARM/gem5.fast'

5. Run the simulation (do not forget to add the directory build/ARM from within the gem5 directory to the PATH):

• make arm_gem5

6. To get the output of Barrelfish you can use 'telnet localhost 3456'

NOTE 1: To change the system configuration, adjust 'tools/arm_gem5/gem5script.py' accordingly or change the command to start gem5 in 'symbolic_targets.mk'

NOTE 2: If you use --cpu-type=arm_detailed (use make arm_gem5_detailed), the simulation takes a long time (depending on your machine up to an hour just to boot Barrelfish)

Running x86-64 Barrelfish on Gem5 (Old!)

The only configuration that has been exercised is full-system x86 using the simple CPU model. Other x86 configurations may work (e.g., in-order CPU, out-of-order CPU, depending on the underlying implementation of these CPU models in gem5). I have used configurations both with and without Ruby.

To get started with Barrelfish on gem5:

1. Build a gem5 tree with x86 simple CPU. I will assume this produces a binary m5.fast, and that this is added to the path.

2. Build an x86_64 Barrelfish tree as usual.

3. In the build directory, run "make m5_kernel". This will construct a single kernel image file that combines the actual kernel along with the modules and a rudimentary boot-loader. The output file is called "m5_kernel". The file "menu.lst.m5" configures which modules are included, and what the physical memory layout of the simulated machine will be. Re-run "make m5_kernel" after changing "menu.lst.m5".

4. In the build directory, run "make m5". This will invoke m5.fast, passing it the m5_kernel and a simple machine configuration taken from "m5script.py".

NB: steps 2--4 can be combined by just running "make m5". I've listed them separately here to help troubleshooting if problems emerge.

In practice the file m5script.py (and m5.fast command line) will need to be tailored to the particular kind of system that you want to simulate. These examples are intended just to help with getting started.

Feel free to contact Tim Harris with any questions.