summaryrefslogtreecommitdiff
path: root/automated
AgeCommit message (Collapse)Author
2017-01-25automated: test-writting-guidelines: fix typosNicolas Dechesne
Change-Id: I1db92f144f8b6ae3b3453278365416af0bcd8a89 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-25automated: sh-test-lib: do not source /etc/lsb-releaseNicolas Dechesne
If /etc/lsb-release file exists , then it will be sourced in dist_name() so any variable defined in this file will override/pollute our existing environment. Since dist_name() is called in the context of the test script, we might end up polluting our test environment with unneeded variable, or even worse we might override variables that we have set in our test scripts... Instead of sourcing the file in the current env, let's do it in a subshell. Change-Id: I99ff9b7bb498f47f01adfe0d21f9b1cce0ebf2cc Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-24automated: Adding ltp-realtimeNaresh Kamboju
LTP realtime tests is an open-source testsuite for testing real-time Linux. The testsuite contains some functional tests and a few performance and latency measurement tests. Change-Id: I0488cdb639b5bc349b7b25f85d43339417eafeb8 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-24automated: linux: add rt-migrate-testChase Qi
Change-Id: I03da01b64b5cded668c86feab952e9d5731d30da Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-23test-runner: user $HOME instead of /rootNicolas Dechesne
Avoid hardcoding '/root', and use $HOME instead. A nice side effect is that it allows to use test-runner on the PC as standard user for local debug.. Change-Id: I2003565f9a9aa432dcf5e02cc49f7e87fe8ed946 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-23automated: linux: add busybox testChase Qi
Change-Id: I4caa10b60609da218f4c825c4f166c9dbc8497d4 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-20automated: sh-test-lib: use warn_msg for step skippingChase Qi
error_msg exit with non-zero, it will kill the 'run.sh' generated by lava-dispatcher or local test-runner as the file has '-e' enabled. To print a warning message for step skipping, use warn_msg instead of error_msg. Change-Id: I20f32de217761fab8261cfa625841737cca48288 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: added Linux mmc testNaresh Kamboju
MMC is the "multi-media card" bus protocol. The MultiMediaCard (MMC) is a memory card standard used for solid-state storage. MMC test will check kernel configuration and list connected mmc and emmc devices. CONFIG_MMC=y CONFIG_MMC_SDHCI=y Change-Id: I40150d11233f23b4cf615a43bb25186e982dd680 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-19automated: add support for measurements with no unitNicolas Dechesne
It is possible to report measurements with no specific units, however the helper functions currently require the unit to be specified. This patch allows test case to provide only the measurement in add_metric() , effectively making the last arg (unit) optional. When processing the output , we then need to ensure whether or not the unit was specified in the first place. I needed this when porting sysbench test case onto 'automated' method (e.g. sysbench reports 'count of events'). With this change I can get the following test results from sysbench: <TEST_CASE_ID=total-time-2-threads RESULT=pass UNITS=s MEASUREMENT=4.0010> <TEST_CASE_ID=total-number-of-events-2-threads RESULT=pass UNITS= MEASUREMENT=8827> <TEST_CASE_ID=total-time-taken-by-event-execution-2-threads RESULT=pass UNITS= MEASUREMENT=7.9996> <TEST_CASE_ID=min-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.90> <TEST_CASE_ID=avg-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.91> <TEST_CASE_ID=max-2-threads RESULT=pass UNITS=ms MEASUREMENT=3.38> <TEST_CASE_ID=approx.-95-percentile-2-threads RESULT=pass UNITS=ms MEASUREMENT=0.93> <ENDRUN sysbench 9ce36643-e7e9-481e-a45a-464cd07d8cd9> 2017-01-19 15:45:57,557 - RUNNER.ResultParser: INFO: Result files saved to: /home/ndec/output/sysbench_9ce36643-e7e9-481e-a45a-464cd07d8cd9 --- Printing result.csv --- name,test_case_id,result,measurement,units,test_params sysbench,total-time-2-threads,pass,4.0010,s,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-number-of-events-2-threads,pass,8827,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,total-time-taken-by-event-execution-2-threads,pass,7.9996,,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,min-2-threads,pass,0.90,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,avg-2-threads,pass,0.91,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,max-2-threads,pass,3.38,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 sysbench,approx.-95-percentile-2-threads,pass,0.93,ms,DURATION=4;MAX_REQUESTS=50000;SKIP_INSTALL=False;TEST=cpu;FILE_TEST_MODE=seqrewr;TIMES=1 Change-Id: Id41207c3237f597bc3db83a5d7d54ee35f6974e3 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-01-19automated: update LTP default version to 20170116Fathi Boudra
Change-Id: Ie5416e687b8f8b6fad95b24fec5abfe94d4d0424 Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-01-19automated: Improve ltp testcase runNaresh Kamboju
LTP test definitions improved to support non package manager distributions. pipe0_status () function being used for runltp and get return status. Stop adding total number of test cases in LAVA results summary. Fixed shellcheck issues. Change-Id: I8eaeaddbb7ad113604144ee563868f6e256515fb Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-19automated: linux: add rt signaltestChase Qi
Change-Id: I47c8b55f9ae6f1ab21ba5f7c46532d8784be12ee Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: linux: add rt pi stress testChase Qi
Change-Id: Ie6d3a7e0a7095d1602bc9648ed4fac9e4a539bdb Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-19automated: linux: add sysbench testChase Qi
Change-Id: Ia09a192f011124902b76523bed81acbdfb250e6c Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: device-tree test fixNaresh Kamboju
Change-Id: Ifa9e27e06396d61e4e32b26755129364f09a7c8d Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-01-18automated: linux: add cyclictestChase Qi
Change-Id: I2a81ff35db40935fe57d25bc35685cf6b54c1fe6 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: sh-test-lib: error_msg: exit with non-zeroChase Qi
LAVA uses '-e' in run.sh. I though we should try to save as many results as possible by always exiting with zero so that the following 'send-to-lava.sh' step will be executed even when test script exit with non-zero. However, in this way, LAVA will record test run as pass even something went wrong and there is no result at all. Always exit with non-zero when error occurs should be the right approach, at least for error_msg() function. In the abnormally exiting cases, we will need to investigate test log in any way. Change-Id: Iaaa5d141878734dea3cd9860dd6779e5d09e10b1 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-18automated: linux: add unixbench testChase Qi
Change-Id: I5d887f97943b92ab84baa315b745bbaf63eee2b3 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-13Add isolation test, verifies that an appl. is isolated.Ravineet Singh
Verify that an application is isolated and that the kernel configuraion is sane for the applcation to be run without interference from user/kernel space tasks. Change-Id: Ie1c267b6d50026a080f6a0d6d25d518f2ad6f4ab Signed-off-by: Ravineet Singh <ravineet.singh@linaro.org>
2017-01-13automated: linux: fix toolchain smoke test on OEChase Qi
Change-Id: I182c3ec136ab83407367a20a358ad15070c76b98 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-13Merge "automated: doc: improve "Installing dependence" section"Naresh Kamboju
2017-01-13automated: linux: fix smoke test on OEChase Qi
Change-Id: I91da729db10c08827f285f65bc525948747797b6 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-13automated: doc: improve "Installing dependence" sectionChase Qi
Change-Id: Id05395fb8d323dff07c23c0de5d13dac7c79a5f6 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-01-04nginx-server: server side of the http performance testsJosep Puigdemont
The nginx-server.sh script implements the same protocol as the httperf-client.sh script, namely: client server | | |<----------- num_cores ---------| # server indicates num cores |---------- client_ready ------->| # server is configured for 1 core |<-- server_num_cores_1_ready ---| # client runs tests for 1 core |--- client_num_cores_1_done --->| # client is finished testing | | # server reconfigures for 2 cores |<-- server_num_cores_2_ready ---| # client runs tests for 2 cores |--- client_num_cores_2_done --->| # client is finished testing | | # repeated until num_cores in | | # increments of 2 (1, 2, 4, 6...) ... Currently it supports testing NGiNX on plain linux IP stack, and a special version of NGiNX compiled for OFP and ODP-DPDK, however the latter is not too portable at the moment and will require further work. Whether one or the other is tested is configured by the CONFIG_TYPE environment variable in the test definition: linux-ip or odp-dpdk, respectively. Also the maximum amount of cores for which to test can be limited by setting the MAX_CORES environment variable (0 meaning all cores). This relies completely on VLAND and lava-{send,wait} for synchronization. Change-Id: I775083f7183895039064141b51172a3280249848 Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-01-04httperf-client: client side of http performance testsJosep Puigdemont
Script and YAML file to setup the host to run httperf against an http server. The tests measure the performance of the server depending on the number of cores. This script is meant to run in a multi-node configuration, where it will coordinate with a server side of the test in order to determine when the tests can start, and when they are done. Change-Id: I3af563086a8fba64f1b0f32c0878761997d21158 Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-01-03get_vland_pci_dev.sh: return the PCI deviceJosep Puigdemont
Change-Id: If0ac3a702528d97f536513e35a1f65be43f15a27 Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2016-12-15automated: linux: dsdbench: only save stdout to logfileChase Qi
Because of this bug https://bugs.linaro.org/show_bug.cgi?id=2694, the error message destroys the structure of logfile, and causes parser not working. Change-Id: I76b7b0b71f5427da53e1778e0abc69ac2f180c17 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-12-08test-runner: fixed result parsing regexpMilosz Wasilewski
Change-Id: I59f130777f01f038899041d2647de7ba0d3d004b Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-12-07linux: ethernet: fix ping and get default routeNaresh Kamboju
Change-Id: Ie3b4dbc0e09078da9e9de6805e3bf5511a606c3c Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-12-07linux: ltp: add net-tools packageNaresh Kamboju
Change-Id: I78080931b6406fcd193eab24b16c4424e8b15778 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-12-06Merge "automated: test-runner: Added support for complex test names"Naresh Kamboju
2016-12-06automated: linux: kernel-compilation: install wgetChase Qi
Change-Id: I5ea518268065d3bb4161e0c7b97f97407e390558 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-12-05automated: test-runner: Added support for complex test namesMilosz Wasilewski
Test names can contain '=' now. The parsing is based on regex rather than simple line split with '=' as delimiter. This adresses libhugetlbfs test parsing problem. Change-Id: I1b6e03960456abf437a2720f6e5177556c766f5a Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-12-05automated: libhugetlbfs fix for running on distro kernelMilosz Wasilewski
Config file for distro kernel is located in /boot rather than in /proc. This patch adds support for both options Change-Id: Ie516fba4e3e37d58508f013e9287964cf5a362e3 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-12-02automated: linux: enable ltp skipfile parameterChase Qi
Change-Id: I4923d5ff672d7248d66f6ae8b7151dac89cd6fdb Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-12-02test-runner: fix version check for manual testChase Qi
When '-k manual' specified, test-runner only copy 'manual' folder to output directory, which isn't a valid git repo. From there, 'git checkout' or 'git rev-parse HEAD' fails. Change-Id: I005501efacb46fb49729995d112f8498164d8ad0 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-12-01test-runner: save test case name to result filesChase Qi
Change-Id: If4d6e28a05e886e6036916b36749b231cd916c34 Signed-off-by: Chase Qi <chase.qi@linaro.org> Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-30automated: linux: docker: fix package installation on CentOSChase Qi
On CentOS/Fedora, the package name is 'docker'. Change-Id: I1d2cfc8bd74df8481583a50f8d1b1804b9267b7b Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-29test-runner: record default parametersMilosz Wasilewski
Default parameters from test definition are now recorded in the result Change-Id: Id8d0fc88c694d158f6406586f5c79dcc3a83c164 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-29Merge "test-runner: added support for test version"Chase Qi
2016-11-29test-runner: added support for test versionMilosz Wasilewski
Change-Id: Idc255ad3e01c5ab95f259221fc901b93c2362862 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2016-11-28automated: added Linux libhugetlbfs testNaresh Kamboju
Change-Id: I9e8e75d22a99c79dae92663a384da197625e85d8 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2016-11-28automated: linux: unify mysql password for LAMP and LEMP testsChase Qi
Change-Id: I1a6d2dd4b16d1a7d2fff88af4c7e98d4e06d2636 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-28automated: linux: lamp: stop nginx to avoid port conflictChase Qi
When nginx is running, apache startup will fail as they use the same port. Change-Id: I36322ed587614b570880e27eadc67d592a683abc Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-24automated: added Linux Kernel compilation testChase Qi
Change-Id: I5d0b6423b8d6267bfc9c558eb7637335541a5055 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-23automated: test-runner: added support for 'parameters' in result parserChase Qi
In test plan, we can use 'params' or 'parameters'. In result parser, 'params' is already supported, adding support for 'parameters' now. Change-Id: I999565eafb72ca8cb9c56a8072fbd53e9f8e0fdd Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-23automated: linux: fix disk partitioning testChase Qi
Change-Id: I407b65bc0d1d6e388dbe91725e7d9407d90a8823 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-23automated: linux: lamp: bug fixesChase Qi
* During the second run, mysql root password cannot be changed as it was already changed in the first run, so throw the error to /dev/null. * Delete the database table created by test case for a clean exit. Change-Id: I383370e5a01bb0457699c5a3db5bd5ea85f0a458 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-23automated: linux: added iperf testChase Qi
Change-Id: Ibf8bebfe78d60028ed7805f5f62080a194a2e660 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2016-11-22automated: Linux network basic: improvementsNaresh Kamboju
Network basic tests improved, - Install net-tools package - Get default gateway at run time - Removed variables from test_case_id - Rename 'test' to 'test_case' Change-Id: I27347cf2bd0627dbb7708742c507f27b78787ffe Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-14 10:14:39 +0000