summaryrefslogtreecommitdiff
path: root/automated/utils/test-runner.py
AgeCommit message (Collapse)Author
2017-04-06test-runner: set test case relative path for remoet exec onlyChase Qi
'tc_relative_dir' is used for partial file copy to target. It splits realpath to yaml file by test kind parameter and assumes tests are from 'automated' or 'manual' folder. Local exec copy the whole repo. It doesn't have the above assumption. Moving the 'tc_relative_dir' related lines to remote exec only. Change-Id: I95415fa93a94ddf654681ada704e4b2a5c12f061 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-03-29test-runner: run 'run.sh' under 'set -x'Chase Qi
When converting testdef to 'run.sh', LAVA has shell '-x' argument enabled. It can be useful for debugging. Change-Id: I1d1070b716713238e213b7140eb44526cdbcd7e1 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-03-17automated: test-runner: encapsulate the TestRun object into TestDefNicolas Dechesne
There is no need to create an additional object in main(). Instead we can create the 'runner' object in the TestDef init() and directly invoke the run() and check_result() function. Also rename TestDefinition.run() function as TestDefinition.mkrun(). This function really generates the run script, calling it run() is confusing, as this is not where the test runs. Change-Id: I4348e1e834e7083174aa60a8f4533747e49c3757 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-03-17automated: test-runner: mark test run as fail when no result foundChase Qi
Change-Id: I6cfc6f192a2392d8da13b788caaba97d08b50227 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-03-17test-runner: add remote execution featureMilosz Wasilewski
Test can be executed remotely using ssh. It is assumed that ssh communication is performed without passwords and public key is already added to authorized_keys. Most tests require root access to be executed, so there is a silent assumption that remote test execution is done using 'root' account. The account name might be different, but should be granted the same level of privileges. Change-Id: I41f1d32d20f15ac6cfa8fc9e99f527fe61d4743d Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-03-14test-runner: set PATH before running the testsNicolas Dechesne
Set PATH to a known value before starting test execution, to avoid PATH being polluted by the user environment. Change-Id: Ia7861932cd56f363d2d6cfa50ffd54c8acb1ea21 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-23test-runner: user $HOME instead of /rootNicolas Dechesne
Avoid hardcoding '/root', and use $HOME instead. A nice side effect is that it allows to use test-runner on the PC as standard user for local debug.. Change-Id: I2003565f9a9aa432dcf5e02cc49f7e87fe8ed946 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2016-12-08test-runner: fixed result parsing regexpMilosz Wasilewski
Change-Id: I59f130777f01f038899041d2647de7ba0d3d004b Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-12-05automated: test-runner: Added support for complex test namesMilosz Wasilewski
Test names can contain '=' now. The parsing is based on regex rather than simple line split with '=' as delimiter. This adresses libhugetlbfs test parsing problem. Change-Id: I1b6e03960456abf437a2720f6e5177556c766f5a Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-12-02test-runner: fix version check for manual testChase Qi
When '-k manual' specified, test-runner only copy 'manual' folder to output directory, which isn't a valid git repo. From there, 'git checkout' or 'git rev-parse HEAD' fails. Change-Id: I005501efacb46fb49729995d112f8498164d8ad0 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-12-01test-runner: save test case name to result filesChase Qi
Change-Id: If4d6e28a05e886e6036916b36749b231cd916c34 Signed-off-by: Chase Qi <chase.qi@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-29test-runner: record default parametersMilosz Wasilewski
Default parameters from test definition are now recorded in the result Change-Id: Id8d0fc88c694d158f6406586f5c79dcc3a83c164 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-29test-runner: added support for test versionMilosz Wasilewski
Change-Id: Idc255ad3e01c5ab95f259221fc901b93c2362862 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-23automated: test-runner: added support for 'parameters' in result parserChase Qi
In test plan, we can use 'params' or 'parameters'. In result parser, 'params' is already supported, adding support for 'parameters' now. Change-Id: I999565eafb72ca8cb9c56a8072fbd53e9f8e0fdd Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-18automated: test-runner: added option to record manual resultsMilosz Wasilewski
Manual test cases can now be executed using test runner. The results are recorded the same way as for automated tests. Change-Id: Ifacbc7591f81bc4468c4dbe6550be0ba1ec3fd61 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-10test-runner: save test parameters to result.csvChase Qi
Change-Id: If6759527cb0a7cc8556c06a91b680ce46d466972 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-07automated: test-runner: enhance result parserChase Qi
* Add test name and params to result.json * Collect all results in output/result.json * Add header and test name for output/result.csv Change-Id: Ie73af79be9e68780a8863e651dc30ed34bcbf4c3 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-09-02v2: test-runner: handle 3rd party python modulesChase Qi
Change-Id: I0a7b460cc891f2c4a927b11940699fde221d87e4 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-08-25v2: initial commitChase Qi
- Added utils, bin, and lib folders and related tools - Added Linux smoke test case as an example - Added test plan example that used to test test-runner Change-Id: I73693108db219cb67afc99525d80cab93f1eb9ea Signed-off-by: Chase Qi <chase.qi@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-30 22:58:24 +0000