summaryrefslogtreecommitdiff
path: root/automated
AgeCommit message (Collapse)Author
2017-03-06automated: add bionic-benchmarks testDaniel Díaz
Change-Id: Ic2714900764a4d0a13864f5feb4a85c8ef54afc0 Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
2017-03-02nginx-odp-dpdk.yaml: use sh-test-lib functionsJosep Puigdemont
We will now use the add_key and add_repo functions provided by sh-test-lib, also the repository used can be specified as one of the test parameters. Installation of release-specific packages has been removed. Any extra packages from other repositories than the ones under test (currently jessie-backports) will have to be installed by the calling LAVA job. This way this test definition will work on newer versions of the distribution without having to change anything. See this lava job for an example: https://lng.validation.linaro.org/scheduler/job/17262/multinode_definition Change-Id: I6119b451c4921afde861f8b3473cdfdca140d3f5 Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-03-02automated: android: cts: use adb from official platform-tools-latestChase Qi
Change-Id: I4c4ee2bf5709edb6b0e2c8c6fdbb8099e2916db2 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-03-02automated: android-test-lib: add install_latest_adb() and adb_root() functionsChase Qi
Change-Id: If83af404a7b819eecf5eb6359821e42cfbbfa029 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-28automated: utils: improve upload-to-artifactorial.shChase Qi
* Only do attachment uploading in LAVA * Use basename of "${ATTACHMENT}" in case it includes a path Change-Id: I4893da2b16766a64152804be47ee8853a162534a Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-27automated: doc: fix typo in filenameChase Qi
Change-Id: Iaa4119406790c36215cbb84f108903342f8b3e11 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-24Merge "automated/android: add CTS result 'pre-parser'"Milosz Wasilewski
2017-02-23nginx-odp-dpdk.yaml: do not install kernel headersJosep Puigdemont
Until now the test installed kernel headers from a hardcoded source. We should require the job calling his test to install the kernel headers, whether they come from a hw-pack, a private repo, or somewhere else. At the same time define configurable parameters for the test. Change-Id: I28dbaa55d71683c23b4d59b188fcebf5fd302a6e Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-02-23nginx-server: remove dependency on private repoJosep Puigdemont
To work around some limitation on the current port of NGiNX to OFP, a private repo with specially built OFP and NGiNX was used by the test script. This is not desirable in the long run, so we will resort to installing the packages provided by LNG at deb.opendataplane.org, and add the limitation that NGiNX for OFP will only be tested with one worker for now. When support for more workers is available, the limitation should be removed. Change-Id: I0f2d5f1c4efe9efb53c77270c0bebaeb03c709b6 Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-02-23automated/android: add CTS result 'pre-parser'Milosz Wasilewski
CTS xml result file might contain invalid XML characters. For this reason the ElementTree parser fails to read the result file. The invalid characters are now removed before the result is passed to ET parser. Change-Id: I1a88627090aaad493a63dca86b4ecee8f13b45fd Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-02-23automated: android: cts: skip adding backports repo on Debian sidChase Qi
Change-Id: I167868f60463b8138115e89a0a2d397a4beb853a Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-22automated: improve android-test-libChase Qi
Change-Id: I6b7eafea28b6bea74a75c31cbb60f6dba2012795 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-20libhugetlbfs: Detect pre-builts tests on targetNaresh Kamboju
libhugetlbfs test definitions modified to detect pre-built tests on target. if not found then, download, build and run tests. Change-Id: If25d54df68f8f2abc1c67ddfa5f1a4a0ed1954fe Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-02-20automated: android: add cts testChase Qi
Change-Id: I75eb65bc7de03cbe612b33c4ee69d50cfdc6608d Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-18automated: use create_out_dir where appropriateDaniel Díaz
Change-Id: I10181a7d1ebc53491085d80d67f0cd8f9e43071d Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
2017-02-17cleanup: fix a few shellcheck errors in automated/Daniel Díaz
Change-Id: If34415c681906aefc6b5b42401ed995e5cdeda11 Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
2017-02-17automated: utils: add upload-to-artifactorial.shChase Qi
Change-Id: I969b285e1b3e092acbee0ca425f5bb1f81a97354 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-17automated: sh-test-lib: add create_out_dir()Daniel Díaz
Some error checking and renaming can happen here for all other scripts to peruse. Change-Id: I7286cf2909c59fc41935658d0a970b9c4b7ec795 Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-02-07automated: linux: add lshw testChase Qi
Change-Id: Iec4efd0a4d4e9dede50505b7284d6221b83a7433 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-06automated: linux: support sysbench compiling on oe-rpbChase Qi
Change-Id: Ibc1e125dc56c4a5b6b73da368b668bafc2c9fb97 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-06automated: use '*' instead of 'unknown' for unsupported distrosChase Qi
When dist_name() fails to detect distribution ID, it sets ${dist} to 'unknown'. install_deps() doesn't support 'unknown' distros, and it doesn't support some known distros like oe-rpb either. Using wildcard able to cover these two cases. Change-Id: I619e06d9dbc911c9629d6d6a7f1622c40ed1f234 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-02-03automated: smoke: no need to be rootNicolas Dechesne
This test case works without root privileges, so let's remove this check. Running as standard user , with this patch, I am getting: --- Printing result.csv --- name,test_case_id,result,measurement,units,test_params smoke-tests-basic,pwd,pass,,,SKIP_INSTALL=False smoke-tests-basic,lsb_release,pass,,,SKIP_INSTALL=False smoke-tests-basic,uname,pass,,,SKIP_INSTALL=False smoke-tests-basic,ip,pass,,,SKIP_INSTALL=False smoke-tests-basic,lscpu,pass,,,SKIP_INSTALL=False smoke-tests-basic,vmstat,pass,,,SKIP_INSTALL=False smoke-tests-basic,lsblk,pass,,,SKIP_INSTALL=False Change-Id: I3dfce348eadf9a9a4a4dfa8321af080984cd4447 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-26automated: sh-test-lib: add check_root() when neededNicolas Dechesne
In a few lib functions , we are running commands which require root. so let's call check_root() whenever it's needed, to avoid adding the check in the test scripts themselves. For example , in many test case we use check_root() mostly because we want to install packages with install_deps(). Let's add a call to check_root() in install_deps() directly, since installing packages *always* require root. Then we can remove un-needed checks from the test case themselves. Change-Id: Ifd76dacf85f21825f2e47cd224a3725c69eba899 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-26automated: use pattern matching for rpbFathi Boudra
avoid to hardcode the rpb variants. use "oe-rpb" as a distribution value instead of "oe-based". Change-Id: I8f1ccf35a1b82718f662bd79480cb2c7dc41c1b8 Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
2017-01-26Merge "automated: always convert dist to lower case"Fathi Boudra
2017-01-25automated/openjdk: fixed shellcheck warningsMilosz Wasilewski
Change-Id: I52acc71b65f6bbb635f498c9b8453a5676afe9b8 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-01-25automated: always convert dist to lower caseFathi Boudra
Change-Id: I5ab1e600776e83a56167e22b35b0e0520f85214a Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org>
2017-01-25automated: switch to using ID field for distroNicolas Dechesne
Let's standardadize all test cases to use lowercase distro ID field from /etc/os-release, instead of the 'pretty' (capitalized) name that we used so far. This is an invasive change, all test scripts were modified with this command: $ sed -i -e 's|Debian\(.*\))|debian\1)|g' \ -e 's|Ubuntu\(.*\))|ubuntu\1)|g' \ -e 's|CentOS\(.*\))|centos\1)|g' \ -e 's|Fedora\(.*\))|fedora\1)|g' \ -e 's|Unknown\(.*\))|unknown\1)|g' And the remaining instances were fixed manually. dist_name() function was updated to always report lowercase distro name. Also when /etc/os-release is not available and that we use lsb_release output, we convert the old capitalized name into the new lowercase ones. Change-Id: I1f1ce0932ab04476567f0338e902656470267a74 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: sh-test-lib: add support for /etc/os-release fileNicolas Dechesne
/etc/os-release is a standard file in most distro now, [1] and [2]. Let's use it as the default mechanism to detect the distro at run time, and keep the old mechanisms as fallback. Also * defines 'oe-based' name for any known OE based releases, so that we can adapt test cases accordingly. * convert the ID name found in /etc/os-release into the one we used to have in lsb_release (e.g. Ubuntu vs ubuntu, ... ). [1] https://www.freedesktop.org/software/systemd/man/os-release.html [2] http://0pointer.de/blog/projects/os-release.html Change-Id: I1c9e11b449f8532450c8343bc29eb0aeec58c229 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: test-writting-guidelines: fix typosNicolas Dechesne
Change-Id: I1db92f144f8b6ae3b3453278365416af0bcd8a89 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: sh-test-lib: do not source /etc/lsb-releaseNicolas Dechesne
If /etc/lsb-release file exists , then it will be sourced in dist_name() so any variable defined in this file will override/pollute our existing environment. Since dist_name() is called in the context of the test script, we might end up polluting our test environment with unneeded variable, or even worse we might override variables that we have set in our test scripts... Instead of sourcing the file in the current env, let's do it in a subshell. Change-Id: I99ff9b7bb498f47f01adfe0d21f9b1cce0ebf2cc Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-24automated: Adding ltp-realtimeNaresh Kamboju
LTP realtime tests is an open-source testsuite for testing real-time Linux. The testsuite contains some functional tests and a few performance and latency measurement tests. Change-Id: I0488cdb639b5bc349b7b25f85d43339417eafeb8 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-24automated: linux: add rt-migrate-testChase Qi
Change-Id: I03da01b64b5cded668c86feab952e9d5731d30da Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-23test-runner: user $HOME instead of /rootNicolas Dechesne
Avoid hardcoding '/root', and use $HOME instead. A nice side effect is that it allows to use test-runner on the PC as standard user for local debug.. Change-Id: I2003565f9a9aa432dcf5e02cc49f7e87fe8ed946 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-23automated: linux: add busybox testChase Qi
Change-Id: I4caa10b60609da218f4c825c4f166c9dbc8497d4 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-20automated: sh-test-lib: use warn_msg for step skippingChase Qi
error_msg exit with non-zero, it will kill the 'run.sh' generated by lava-dispatcher or local test-runner as the file has '-e' enabled. To print a warning message for step skipping, use warn_msg instead of error_msg. Change-Id: I20f32de217761fab8261cfa625841737cca48288 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: added Linux mmc testNaresh Kamboju
MMC is the "multi-media card" bus protocol. The MultiMediaCard (MMC) is a memory card standard used for solid-state storage. MMC test will check kernel configuration and list connected mmc and emmc devices. CONFIG_MMC=y CONFIG_MMC_SDHCI=y Change-Id: I40150d11233f23b4cf615a43bb25186e982dd680 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-19automated: add support for measurements with no unitNicolas Dechesne
It is possible to report measurements with no specific units, however the helper functions currently require the unit to be specified. This patch allows test case to provide only the measurement in add_metric() , effectively making the last arg (unit) optional. When processing the output , we then need to ensure whether or not the unit was specified in the first place. I needed this when porting sysbench test case onto 'automated' method (e.g. sysbench reports 'count of events'). With this change I can get the following test results from sysbench: <TEST_CASE_ID=total-time-2-threads RESULT=pass UNITS=s MEASUREMENT=4.0010> <TEST_CASE_ID=total-number-of-events-2-threads RESULT=pass UNITS= MEASUREMENT=8827> <TEST_CASE_ID=total-time-taken-by-event-execution-2-threads RESULT=pass UNITS= MEASUREMENT=7.9996> <TEST_CASE_ID=min-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.90> <TEST_CASE_ID=avg-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.91> <TEST_CASE_ID=max-2-threads RESULT=pass UNITS=ms MEASUREMENT=3.38> <TEST_CASE_ID=approx.-95-percentile-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.93> <ENDRUN sysbench 9ce36643-e7e9-481e-a45a-464cd07d8cd9> 2017-01-19 15:45:57,557 - RUNNER.ResultParser: INFO: Result files saved to: /home/ndec/output/sysbench_9ce36643-e7e9-481e-a45a-464cd07d8cd9 --- Printing result.csv --- name,test_case_id,result,measurement,units,test_params sysbench,total-time-2-threads,pass,4.0010,s,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-number-of-events-2-threads,pass,8827,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-time-taken-by-event-execution-2-threads,pass,7.9996,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,min-2-threads,pass,0.90,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,avg-2-threads,pass,0.91,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,max-2-threads,pass,3.38,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,approx.-95-percentile-2-threads,pass,0.93,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 Change-Id: Id41207c3237f597bc3db83a5d7d54ee35f6974e3 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-19automated: update LTP default version to 20170116Fathi Boudra
Change-Id: Ie5416e687b8f8b6fad95b24fec5abfe94d4d0424 Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-01-19automated: Improve ltp testcase runNaresh Kamboju
LTP test definitions improved to support non package manager distributions. pipe0_status () function being used for runltp and get return status. Stop adding total number of test cases in LAVA results summary. Fixed shellcheck issues. Change-Id: I8eaeaddbb7ad113604144ee563868f6e256515fb Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-19automated: linux: add rt signaltestChase Qi
Change-Id: I47c8b55f9ae6f1ab21ba5f7c46532d8784be12ee Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: linux: add rt pi stress testChase Qi
Change-Id: Ie6d3a7e0a7095d1602bc9648ed4fac9e4a539bdb Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: linux: add sysbench testChase Qi
Change-Id: Ia09a192f011124902b76523bed81acbdfb250e6c Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: device-tree test fixNaresh Kamboju
Change-Id: Ifa9e27e06396d61e4e32b26755129364f09a7c8d Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-18automated: linux: add cyclictestChase Qi
Change-Id: I2a81ff35db40935fe57d25bc35685cf6b54c1fe6 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: sh-test-lib: error_msg: exit with non-zeroChase Qi
LAVA uses '-e' in run.sh. I though we should try to save as many results as possible by always exiting with zero so that the following 'send-to-lava.sh' step will be executed even when test script exit with non-zero. However, in this way, LAVA will record test run as pass even something went wrong and there is no result at all. Always exit with non-zero when error occurs should be the right approach, at least for error_msg() function. In the abnormally exiting cases, we will need to investigate test log in any way. Change-Id: Iaaa5d141878734dea3cd9860dd6779e5d09e10b1 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: linux: add unixbench testChase Qi
Change-Id: I5d887f97943b92ab84baa315b745bbaf63eee2b3 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-13Add isolation test, verifies that an appl. is isolated.Ravineet Singh
Verify that an application is isolated and that the kernel configuraion is sane for the applcation to be run without interference from user/kernel space tasks. Change-Id: Ie1c267b6d50026a080f6a0d6d25d518f2ad6f4ab Signed-off-by: Ravineet Singh <ravineet.singh@linaro.org>
2017-01-13automated: linux: fix toolchain smoke test on OEChase Qi
Change-Id: I182c3ec136ab83407367a20a358ad15070c76b98 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-13Merge "automated: doc: improve "Installing dependence" section"Naresh Kamboju
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-24 09:26:44 +0000