diff options
authorMilosz Wasilewski <>2017-04-18 16:36:41 +0100
committerMilosz Wasilewski <>2017-04-20 13:46:50 +0000
commit05b2d384740be747d3de6745fa7ace160d1f26c4 (patch)
parent73cd1d170cca3ffbb39421f250fb8c419c17928c (diff)
automated: doc: Added section on using test-runner
The paragraph on test-runner was contributed by Nicolas Dechesne. The section was mostly copied from: Change-Id: I217d33f2259ae16b924c6a0a8fec8c7690bcb65b Signed-off-by: Milosz Wasilewski <>
1 files changed, 58 insertions, 0 deletions
diff --git a/automated/doc/test-writing-guidelines.rst b/automated/doc/test-writing-guidelines.rst
index 6a84562..0c97bfc 100644
--- a/automated/doc/test-writing-guidelines.rst
+++ b/automated/doc/test-writing-guidelines.rst
@@ -223,6 +223,64 @@ A typical Android test case can be written with the following steps::
# Pull output from device for parsing.
pull_output "${DEVICE_OUTPUT}" "${HOST_OUTPUT}"
+6. Using test-runner
+Using test-runner to run tests locally
+The tests can be run directly on the board, assuming you have installed basic
+tools such as git, gcc, ... `test-runner` is written in Python and requires
+`pexpect` and `yaml` modules to be installed as well. To run tests directly
+on the board, get a prompt and run::
+ git clone
+ cd test-definitions
+ source automated/bin/
+ test-runner -p plans/rpb_ee/rpb_ee_functional.yaml
+By default the test output are stored in `$HOME/output/`, and the output folder
+can be configured with `-o` argument.
+Using test-runner to run tests from host PC
+It is also possible to run tests from a host PC if the board is available on
+the network. In that case `test-runner` will connect to the board over SSH, and
+you need to setup the board so that the host PC can connect to the board over
+SSH without any prompt (password less connection). To run from the host, run
+the following commands from the host command prompt::
+ git clone
+ cd test-definitions
+ source automated/bin/
+ test-runner -g root@ip -p plans/rpb_ee/rpb_ee_functional.yaml
+Where `root@ip` is the credential to connect to the board over SSH.
+By default the test output are stored in `$HOME/output/root@ip`, and the output
+folder can be configured with `-o` argument.
+Running individual tests
+Instead of running a test plan with `-p` argument, it is possible to run a single
+test only using `-d` argument.
+Test output
+At the end of the test run, the following artefact are available in the output
+ - `result.csv` and `result.json` which contain summary of test results
+ (including test name, test case ID, test results such as pass, fail, skip,
+ test measurement, if any, with the associated measurement unit, and the test
+ argument used
+ - For each test executed, there is a folder which contains the console output
+ of the test run, `stdout.log` as well as all test scripts/data
Test Contribution Checklist