Age | Commit message (Collapse) | Author |
|
Change-Id: If83af404a7b819eecf5eb6359821e42cfbbfa029
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Change-Id: I6b7eafea28b6bea74a75c31cbb60f6dba2012795
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Change-Id: I75eb65bc7de03cbe612b33c4ee69d50cfdc6608d
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Some error checking and renaming can happen here for all
other scripts to peruse.
Change-Id: I7286cf2909c59fc41935658d0a970b9c4b7ec795
Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
|
|
When dist_name() fails to detect distribution ID, it sets ${dist} to
'unknown'. install_deps() doesn't support 'unknown' distros, and it
doesn't support some known distros like oe-rpb either. Using wildcard
able to cover these two cases.
Change-Id: I619e06d9dbc911c9629d6d6a7f1622c40ed1f234
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
In a few lib functions , we are running commands which require root. so let's
call check_root() whenever it's needed, to avoid adding the check in the test
scripts themselves.
For example , in many test case we use check_root() mostly because we want to
install packages with install_deps(). Let's add a call to check_root() in
install_deps() directly, since installing packages *always* require root. Then
we can remove un-needed checks from the test case themselves.
Change-Id: Ifd76dacf85f21825f2e47cd224a3725c69eba899
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
|
|
avoid to hardcode the rpb variants.
use "oe-rpb" as a distribution value instead of "oe-based".
Change-Id: I8f1ccf35a1b82718f662bd79480cb2c7dc41c1b8
Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
|
|
Change-Id: I5ab1e600776e83a56167e22b35b0e0520f85214a
Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
|
|
Let's standardadize all test cases to use lowercase distro ID field from
/etc/os-release, instead of the 'pretty' (capitalized) name that we used so far.
This is an invasive change, all test scripts were modified with this command:
$ sed -i -e 's|Debian\(.*\))|debian\1)|g' \
-e 's|Ubuntu\(.*\))|ubuntu\1)|g' \
-e 's|CentOS\(.*\))|centos\1)|g' \
-e 's|Fedora\(.*\))|fedora\1)|g' \
-e 's|Unknown\(.*\))|unknown\1)|g'
And the remaining instances were fixed manually.
dist_name() function was updated to always report lowercase distro name. Also
when /etc/os-release is not available and that we use lsb_release output, we
convert the old capitalized name into the new lowercase ones.
Change-Id: I1f1ce0932ab04476567f0338e902656470267a74
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
|
|
/etc/os-release is a standard file in most distro now, [1] and [2].
Let's use it as the default mechanism to detect the distro at run time, and keep
the old mechanisms as fallback.
Also
* defines 'oe-based' name for any known OE based releases, so that we can
adapt test cases accordingly.
* convert the ID name found in /etc/os-release into the one we used to have in
lsb_release (e.g. Ubuntu vs ubuntu, ... ).
[1] https://www.freedesktop.org/software/systemd/man/os-release.html
[2] http://0pointer.de/blog/projects/os-release.html
Change-Id: I1c9e11b449f8532450c8343bc29eb0aeec58c229
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
|
|
If /etc/lsb-release file exists , then it will be sourced in dist_name() so any
variable defined in this file will override/pollute our existing
environment. Since dist_name() is called in the context of the test script, we
might end up polluting our test environment with unneeded variable, or even
worse we might override variables that we have set in our test scripts...
Instead of sourcing the file in the current env, let's do it in a subshell.
Change-Id: I99ff9b7bb498f47f01adfe0d21f9b1cce0ebf2cc
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
|
|
error_msg exit with non-zero, it will kill the 'run.sh' generated by
lava-dispatcher or local test-runner as the file has '-e' enabled.
To print a warning message for step skipping, use warn_msg instead
of error_msg.
Change-Id: I20f32de217761fab8261cfa625841737cca48288
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
It is possible to report measurements with no specific units, however the helper
functions currently require the unit to be specified. This patch allows test
case to provide only the measurement in add_metric() , effectively making the
last arg (unit) optional.
When processing the output , we then need to ensure whether or not the unit was
specified in the first place.
I needed this when porting sysbench test case onto 'automated' method
(e.g. sysbench reports 'count of events'). With this change I can get the
following test results from sysbench:
<TEST_CASE_ID=total-time-2-threads RESULT=pass UNITS=s MEASUREMENT=4.0010>
<TEST_CASE_ID=total-number-of-events-2-threads RESULT=pass UNITS= MEASUREMENT=8827>
<TEST_CASE_ID=total-time-taken-by-event-execution-2-threads RESULT=pass UNITS= MEASUREMENT=7.9996>
<TEST_CASE_ID=min-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.90>
<TEST_CASE_ID=avg-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.91>
<TEST_CASE_ID=max-2-threads RESULT=pass UNITS=ms MEASUREMENT=3.38>
<TEST_CASE_ID=approx.-95-percentile-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.93>
<ENDRUN sysbench 9ce36643-e7e9-481e-a45a-464cd07d8cd9>
2017-01-19 15:45:57,557 - RUNNER.ResultParser: INFO: Result files saved to: /home/ndec/output/sysbench_9ce36643-e7e9-481e-a45a-464cd07d8cd9
--- Printing result.csv ---
name,test_case_id,result,measurement,units,test_params
sysbench,total-time-2-threads,pass,4.0010,s,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,total-number-of-events-2-threads,pass,8827,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,total-time-taken-by-event-execution-2-threads,pass,7.9996,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,min-2-threads,pass,0.90,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,avg-2-threads,pass,0.91,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,max-2-threads,pass,3.38,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
sysbench,approx.-95-percentile-2-threads,pass,0.93,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1
Change-Id: Id41207c3237f597bc3db83a5d7d54ee35f6974e3
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
|
|
LAVA uses '-e' in run.sh. I though we should try to save as many results
as possible by always exiting with zero so that the following
'send-to-lava.sh' step will be executed even when test script exit with
non-zero.
However, in this way, LAVA will record test run as pass even something
went wrong and there is no result at all.
Always exit with non-zero when error occurs should be the right
approach, at least for error_msg() function. In the abnormally exiting
cases, we will need to investigate test log in any way.
Change-Id: Iaaa5d141878734dea3cd9860dd6779e5d09e10b1
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
exit_on_skip "testcase-name" "reason for skip"
Change-Id: I361f8282c1201362cd89afb4961012d02c8b2bbe
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
|
|
To cope with shellcheck
- Non-POSIX keyword 'local' removed
- variable name and keyword 'test' renamed to 'test_case'
Change-Id: Idb1b4064bb82ec219c9b4fed39fd6d07d5cc9249
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
|
|
Change-Id: Icae2293f011b7ed59061452cf119e4e1b077d6cd
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
When shell argument "-e" set and "$?" is non-zero, test script will exit
immediately. check_return and exit_on_fail would not work. run_test_case
should be used instead.
run_test_case runs test command within if block and able to give
pass/fail respectively. "skip_list" is supported as an optional
parameter. It skip the following tests and exit when skip_list isn't
empty.
Change-Id: I0f4040f37b6d9daa0558bac0d6bf87ba1c3bd904
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Binaries are placed in folders named with abi. Test script need to
detect abi type and use related binary.
Change-Id: Id64bb84ba1170a52fdc2f1b485dc46eec1faae3f
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Change-Id: I91764f7abc7ca2fb1995d9fb110c32d3b5f8c2f5
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Change-Id: Iaca9fd730ffd80ffaba71c0438786fd6b14034f0
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
SC2086 - Double quote to prevent globbing and word splitting.
"${pkgs}" stands for a single package, not a list of packages. When
multiple packages defined in the variable, quotes on it needs to be
removed.
Change-Id: I972df969ba3f388fe5582a6099baacda1db24cdd
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
pipe0_status is used to return exit code of the first command when using
pipe. Usage: pipe0_status "cmd1" "cmd2"
Change-Id: I1549e807c2a5401fdc1435a4ac3ac06e27765663
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
When using exit_on_fail function, you may need to save "rest_tests skip"
to result file. Adding optional parameter skip_list for that usage.
Change-Id: I6c8165dfaae36caace5757bd90b4e19185c46199
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Adding two functions
- report_pass
- report_fail
Change-Id: Ib28c8962f6ecad0b285ff54db607b61c74a056f4
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
|
|
Change-Id: I1abb98d67746529197b5f262311e2b06a582edf8
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
Ui browser test developed on top of Robot frame work.
Test runs chromium, chrome and firefox test cases.
On LAVA we have access to serial console and we have to run ui tests
from this serial console, this is possible by changing user from
root to linaro and exporting the DISPLAY=:0. because linaro owns the
X display process.
Change-Id: I43bda407f468120839b0a4c8248777303945529e
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
|
|
- Added Linux dd write/read speed test
- Optimized sh-test-lib
Change-Id: I3bccfda212535e9a36b8e231b69c0dd2367a88f6
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|
|
- Added utils, bin, and lib folders and related tools
- Added Linux smoke test case as an example
- Added test plan example that used to test test-runner
Change-Id: I73693108db219cb67afc99525d80cab93f1eb9ea
Signed-off-by: Chase Qi <chase.qi@linaro.org>
|