summaryrefslogtreecommitdiff
path: root/automated/lib/sh-test-lib
AgeCommit message (Collapse)Author
2018-11-12sh-test-lib: add remove_pkgs()Chase Qi
Change-Id: I4f45cf42cb8c96687168126631e0525d32121aa0 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2018-08-23automated/lib/sh-test-lib: update detect_abi with i386Anders Roxell
Change-Id: Ice90297b025b2cbf167cce3928559f2f54bd556f Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
2018-01-30LTP: Add skipgen skipfile support to ltpDan Rue
Create a helper function in sh-test-lib to handle the mechanics of generating a skipfile, that will be re-usable by kselftest. Add the following arguments to ltp.sh: [-b board] [-g branch] [-e environment] When the skipfile argument passed via -S ends in ".yaml", a skipfile will be generated using skipgen. Board, branch, and environment are optional arguments that get passed to skipgen. The resulting skipfile is automatically printed to stdout after being generated. Change-Id: Ib45d11acfa85241dbe5dfa7173e75cb7754a6892 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2018-01-17skipgen: Import Skiplist Generator 0.2.0Dan Rue
Import skipgen 0.2.0 binaries from https://github.com/Linaro/skipgen. Skiplist Generator (skipgen) is a program that will generate a skiplist given a yaml file and optionally a board name, branch name, and environment name. Change x86 to x86_64 in detect_abi() to distinguish between 32 and 64 bit x86. Change-Id: I95c01c35dd264026860ad4525194346a9800899a Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-12-12sh-test-lib: fix fatal error reportingVishal Bhoj
Change-Id: I4972cb6ed8a8af05b3bde98c75b6899884983de5 Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-12-07Merge "sh-test-lib: Don't run apt-get update every time"Chase Qi
2017-12-07sh-test-lib: Don't run apt-get update every timeDan Rue
Run apt-get update once per 6 hours, rather than every time dependencies are installed. Change-Id: I6eaa15ac47e6a3cc7d60a6ea45fa663183547ca4 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-12-05android: stop the lava job if device is not presentVishal Bhoj
Change-Id: Ia2e69c4e480c8e461792022e55568c387a350fab Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-09-14sh-test-lib: added support for x86 in detect_abi()Chase Qi
Change-Id: Ic9bcddbd476ecc912603603a4476180cc1ee24dc Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-04Exit on install_deps errorChase Qi
If test script runs with '-e' argument, test should exit on errors. If test script runs without '-e' argument, 'if [ $? -ne 0 ]' will throw an error message and exit on installation error. Change-Id: I6365801582deaf7f5f105fe7014eadc4d0d3cdf7 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-17automated: sh-test-lib: add create_out_dir()Daniel Díaz
Some error checking and renaming can happen here for all other scripts to peruse. Change-Id: I7286cf2909c59fc41935658d0a970b9c4b7ec795 Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-02-06automated: use '*' instead of 'unknown' for unsupported distrosChase Qi
When dist_name() fails to detect distribution ID, it sets ${dist} to 'unknown'. install_deps() doesn't support 'unknown' distros, and it doesn't support some known distros like oe-rpb either. Using wildcard able to cover these two cases. Change-Id: I619e06d9dbc911c9629d6d6a7f1622c40ed1f234 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-26automated: sh-test-lib: add check_root() when neededNicolas Dechesne
In a few lib functions , we are running commands which require root. so let's call check_root() whenever it's needed, to avoid adding the check in the test scripts themselves. For example , in many test case we use check_root() mostly because we want to install packages with install_deps(). Let's add a call to check_root() in install_deps() directly, since installing packages *always* require root. Then we can remove un-needed checks from the test case themselves. Change-Id: Ifd76dacf85f21825f2e47cd224a3725c69eba899 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-26automated: use pattern matching for rpbFathi Boudra
avoid to hardcode the rpb variants. use "oe-rpb" as a distribution value instead of "oe-based". Change-Id: I8f1ccf35a1b82718f662bd79480cb2c7dc41c1b8 Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
2017-01-25automated: always convert dist to lower caseFathi Boudra
Change-Id: I5ab1e600776e83a56167e22b35b0e0520f85214a Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
2017-01-25automated: switch to using ID field for distroNicolas Dechesne
Let's standardadize all test cases to use lowercase distro ID field from /etc/os-release, instead of the 'pretty' (capitalized) name that we used so far. This is an invasive change, all test scripts were modified with this command: $ sed -i -e 's|Debian\(.*\))|debian\1)|g' \ -e 's|Ubuntu\(.*\))|ubuntu\1)|g' \ -e 's|CentOS\(.*\))|centos\1)|g' \ -e 's|Fedora\(.*\))|fedora\1)|g' \ -e 's|Unknown\(.*\))|unknown\1)|g' And the remaining instances were fixed manually. dist_name() function was updated to always report lowercase distro name. Also when /etc/os-release is not available and that we use lsb_release output, we convert the old capitalized name into the new lowercase ones. Change-Id: I1f1ce0932ab04476567f0338e902656470267a74 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: sh-test-lib: add support for /etc/os-release fileNicolas Dechesne
/etc/os-release is a standard file in most distro now, [1] and [2]. Let's use it as the default mechanism to detect the distro at run time, and keep the old mechanisms as fallback. Also * defines 'oe-based' name for any known OE based releases, so that we can adapt test cases accordingly. * convert the ID name found in /etc/os-release into the one we used to have in lsb_release (e.g. Ubuntu vs ubuntu, ... ). [1] https://www.freedesktop.org/software/systemd/man/os-release.html [2] http://0pointer.de/blog/projects/os-release.html Change-Id: I1c9e11b449f8532450c8343bc29eb0aeec58c229 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: sh-test-lib: do not source /etc/lsb-releaseNicolas Dechesne
If /etc/lsb-release file exists , then it will be sourced in dist_name() so any variable defined in this file will override/pollute our existing environment. Since dist_name() is called in the context of the test script, we might end up polluting our test environment with unneeded variable, or even worse we might override variables that we have set in our test scripts... Instead of sourcing the file in the current env, let's do it in a subshell. Change-Id: I99ff9b7bb498f47f01adfe0d21f9b1cce0ebf2cc Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-20automated: sh-test-lib: use warn_msg for step skippingChase Qi
error_msg exit with non-zero, it will kill the 'run.sh' generated by lava-dispatcher or local test-runner as the file has '-e' enabled. To print a warning message for step skipping, use warn_msg instead of error_msg. Change-Id: I20f32de217761fab8261cfa625841737cca48288 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: add support for measurements with no unitNicolas Dechesne
It is possible to report measurements with no specific units, however the helper functions currently require the unit to be specified. This patch allows test case to provide only the measurement in add_metric() , effectively making the last arg (unit) optional. When processing the output , we then need to ensure whether or not the unit was specified in the first place. I needed this when porting sysbench test case onto 'automated' method (e.g. sysbench reports 'count of events'). With this change I can get the following test results from sysbench: <TEST_CASE_ID=total-time-2-threads RESULT=pass UNITS=s MEASUREMENT=4.0010> <TEST_CASE_ID=total-number-of-events-2-threads RESULT=pass UNITS= MEASUREMENT=8827> <TEST_CASE_ID=total-time-taken-by-event-execution-2-threads RESULT=pass UNITS= MEASUREMENT=7.9996> <TEST_CASE_ID=min-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.90> <TEST_CASE_ID=avg-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.91> <TEST_CASE_ID=max-2-threads RESULT=pass UNITS=ms MEASUREMENT=3.38> <TEST_CASE_ID=approx.-95-percentile-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.93> <ENDRUN sysbench 9ce36643-e7e9-481e-a45a-464cd07d8cd9> 2017-01-19 15:45:57,557 - RUNNER.ResultParser: INFO: Result files saved to: /home/ndec/output/sysbench_9ce36643-e7e9-481e-a45a-464cd07d8cd9 --- Printing result.csv --- name,test_case_id,result,measurement,units,test_params sysbench,total-time-2-threads,pass,4.0010,s,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-number-of-events-2-threads,pass,8827,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-time-taken-by-event-execution-2-threads,pass,7.9996,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,min-2-threads,pass,0.90,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,avg-2-threads,pass,0.91,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,max-2-threads,pass,3.38,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,approx.-95-percentile-2-threads,pass,0.93,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 Change-Id: Id41207c3237f597bc3db83a5d7d54ee35f6974e3 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-18automated: sh-test-lib: error_msg: exit with non-zeroChase Qi
LAVA uses '-e' in run.sh. I though we should try to save as many results as possible by always exiting with zero so that the following 'send-to-lava.sh' step will be executed even when test script exit with non-zero. However, in this way, LAVA will record test run as pass even something went wrong and there is no result at all. Always exit with non-zero when error occurs should be the right approach, at least for error_msg() function. In the abnormally exiting cases, we will need to investigate test log in any way. Change-Id: Iaaa5d141878734dea3cd9860dd6779e5d09e10b1 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-18lib: Adding new function exit_on_skipNaresh Kamboju
exit_on_skip "testcase-name" "reason for skip" Change-Id: I361f8282c1201362cd89afb4961012d02c8b2bbe Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-18lib: sh-test-lib: remove non POSIX 'local' keywordNaresh Kamboju
To cope with shellcheck - Non-POSIX keyword 'local' removed - variable name and keyword 'test' renamed to 'test_case' Change-Id: Idb1b4064bb82ec219c9b4fed39fd6d07d5cc9249 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-10-28v2: sh-test-lib: added support for adding 3rd-party repoChase Qi
Change-Id: Icae2293f011b7ed59061452cf119e4e1b077d6cd Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-27v2: sh-test-lib: add run_test_case functionChase Qi
When shell argument "-e" set and "$?" is non-zero, test script will exit immediately. check_return and exit_on_fail would not work. run_test_case should be used instead. run_test_case runs test command within if block and able to give pass/fail respectively. "skip_list" is supported as an optional parameter. It skip the following tests and exit when skip_list isn't empty. Change-Id: I0f4040f37b6d9daa0558bac0d6bf87ba1c3bd904 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-27v2: sh-test-lib: add detect_abi functionChase Qi
Binaries are placed in folders named with abi. Test script need to detect abi type and use related binary. Change-Id: Id64bb84ba1170a52fdc2f1b485dc46eec1faae3f Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-25v2: sh-test-lib: add convert_to_mb functionChase Qi
Change-Id: I91764f7abc7ca2fb1995d9fb110c32d3b5f8c2f5 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-24v2: sh-test-lib: fix shellcheck error on validate_check_sum funtionChase Qi
Change-Id: Iaca9fd730ffd80ffaba71c0438786fd6b14034f0 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-24v2: sh-test-lib: disable shellcheck SC2086 on install_deps functionChase Qi
SC2086 - Double quote to prevent globbing and word splitting. "${pkgs}" stands for a single package, not a list of packages. When multiple packages defined in the variable, quotes on it needs to be removed. Change-Id: I972df969ba3f388fe5582a6099baacda1db24cdd Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-10-20v2: lib: add pipe0_status functionChase Qi
pipe0_status is used to return exit code of the first command when using pipe. Usage: pipe0_status "cmd1" "cmd2" Change-Id: I1549e807c2a5401fdc1435a4ac3ac06e27765663 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-09-30v2: lib: added skip_list support for exit_on_failChase Qi
When using exit_on_fail function, you may need to save "rest_tests skip" to result file. Adding optional parameter skip_list for that usage. Change-Id: I6c8165dfaae36caace5757bd90b4e19185c46199 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-09-16v2: lib: update sh-test-libNaresh Kamboju
Adding two functions - report_pass - report_fail Change-Id: Ib28c8962f6ecad0b285ff54db607b61c74a056f4 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-09-13v2: linux: add openjdk smoke testChase Qi
Change-Id: I1abb98d67746529197b5f262311e2b06a582edf8 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-09-06ui-browser-test: Adding UI browser testingNaresh Kamboju
Ui browser test developed on top of Robot frame work. Test runs chromium, chrome and firefox test cases. On LAVA we have access to serial console and we have to run ui tests from this serial console, this is possible by changing user from root to linaro and exporting the DISPLAY=:0. because linaro owns the X display process. Change-Id: I43bda407f468120839b0a4c8248777303945529e Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-08-30v2: linux: add dd write/read speed testChase Qi
- Added Linux dd write/read speed test - Optimized sh-test-lib Change-Id: I3bccfda212535e9a36b8e231b69c0dd2367a88f6 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-08-25v2: initial commitChase Qi
- Added utils, bin, and lib folders and related tools - Added Linux smoke test case as an example - Added test plan example that used to test test-runner Change-Id: I73693108db219cb67afc99525d80cab93f1eb9ea Signed-off-by: Chase Qi <chase.qi@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-19 02:53:23 +0000