diff options
| -rw-r--r-- | automated/doc/test-writing-guidelines.rst | 58 |
1 files changed, 58 insertions, 0 deletions
diff --git a/automated/doc/test-writing-guidelines.rst b/automated/doc/test-writing-guidelines.rst index 6a84562..0c97bfc 100644 --- a/automated/doc/test-writing-guidelines.rst +++ b/automated/doc/test-writing-guidelines.rst @@ -223,6 +223,64 @@ A typical Android test case can be written with the following steps:: # Pull output from device for parsing. pull_output "${DEVICE_OUTPUT}" "${HOST_OUTPUT}" + +6. Using test-runner +~~~~~~~~~~~~~~~~~~~~ + +Using test-runner to run tests locally +-------------------------------------- + +The tests can be run directly on the board, assuming you have installed basic +tools such as git, gcc, ... `test-runner` is written in Python and requires +`pexpect` and `yaml` modules to be installed as well. To run tests directly +on the board, get a prompt and run:: + + git clone http://git.linaro.org/qa/test-definitions.git + cd test-definitions + source automated/bin/setenv.sh + test-runner -p plans/rpb_ee/rpb_ee_functional.yaml + +By default the test output are stored in `$HOME/output/`, and the output folder +can be configured with `-o` argument. + +Using test-runner to run tests from host PC +------------------------------------------- + +It is also possible to run tests from a host PC if the board is available on +the network. In that case `test-runner` will connect to the board over SSH, and +you need to setup the board so that the host PC can connect to the board over +SSH without any prompt (password less connection). To run from the host, run +the following commands from the host command prompt:: + + git clone http://git.linaro.org/qa/test-definitions.git + cd test-definitions + source automated/bin/setenv.sh + test-runner -g root@ip -p plans/rpb_ee/rpb_ee_functional.yaml + +Where `root@ip` is the credential to connect to the board over SSH. + +By default the test output are stored in `$HOME/output/root@ip`, and the output +folder can be configured with `-o` argument. + +Running individual tests +------------------------ + +Instead of running a test plan with `-p` argument, it is possible to run a single +test only using `-d` argument. + +Test output +----------- + +At the end of the test run, the following artefact are available in the output +folder: + + - `result.csv` and `result.json` which contain summary of test results + (including test name, test case ID, test results such as pass, fail, skip, + test measurement, if any, with the associated measurement unit, and the test + argument used + - For each test executed, there is a folder which contains the console output + of the test run, `stdout.log` as well as all test scripts/data + Test Contribution Checklist =========================== |