Harness/Scalebench is a script that tests Barrelfish.
Scalebench is a Python script in tools/harness/scalebench.py in the Barrelfish source repo. It contains a number of tests and benchmarks and can be used by anyone, but needs extra configuration when used outside of Microsoft or ETH. In particular, it needs to know about the configuration of the machines it is going to execute tests on, how to boot these machines and how to read their output. There are a number of QEMU-emulated machines preconfigured, which should work for everyone without further configuration.
Runnning a test
Harness can execute tests on a variety of hardware.
Write a new Scalebench ("harness") test
Create a new python class that inherits from TestCommon in a file in tools/harness/tests and implement the following methods (minimum):
import tests from common import TestCommon from results import PassFailResult # need this decorator for our test to show up in list of available tests @tests.add_test class MyTest(TestCommon): '''Give a short description of your test here''' # this is the name that you will later use to run your test: # scalebench.py -t mytest ... name = "mytest" def get_modules(self, build, machine): '''Here you can provide one or more modules (binaries) that should be run for your test''' # get default set of modules from our superclass modules = super(MyTest, self).get_modules(build, machine) # add our test binary, this needs to be buildable as `make mytest` modules.add_module("mytest") return modules def process_data(self, testdir, rawiter): '''Here you can process the output of your test to determine pass/fail. `rawiter` is a raw iterator over the output of the test (e.g. qemu or console). `testdir` is a directory where you can store additional processed output, if desired. ''' # iterate over all lines of output for line in rawiter: if line.startswith("mytest passed"): # found the line that `mytest` prints when the test # passes, return PASS return PassFailResult(True) # didn't find line of test passing at all, return FAIL return PassFailResult(False)