summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-05-25automated: android: xtest: improve result parserChase Qi
Change-Id: I08cc2a8200c35e9551612fe9d1c25362f90f544d Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-25test-runner: add support for Python3Chase Qi
* Use os.chmod('file', 0o755) to change file permission. It is supported in Python3 and higher version Python2(tested with v2.7.12 on Ubuntu 16.04). To support lower version Python2, we will need to detect Python version at runtime and use different syntax respectively. That probably isn't necessary. * Use print("string") format for printing which is supported by both Python2 and Python3. * Cope with Python3's strict separation of strings and bytes. Change-Id: I11ab8b956a750e7d6bfafe6ead37686248572854 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-25android: optee-xtest: improve test results parserNaresh Kamboju
Results print format changed on upstream optee_test tree https://github.com/OP-TEE/optee_test/commit/213ca8aaf4eafe9d26af76db7b31d9595ce57e6f Use ${TEST_SUITE} as prefix to detect testcase names. Change-Id: I5f5dc30201f61049fa0882903ee830b01acac445 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-24automated: android: add bootchart testChase Qi
* Migrate bootchart to v2. * Generate bootchart graphic. Change-Id: Iea19523d30d036a8197ee6cfc1a39a58512725f7 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-24automated: add support for workload automation test suiteChase Qi
Change-Id: Ie07d58c902e4c905c938b1aad8a7c126cdcf7ebf Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-23test-runner: fix test kind selectionChase Qi
Change-Id: I89c8760d20b6d5fb9705adb4fe237916487b3a30 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-22automated: linux: lapack: fetch upstream sourceNaresh Kamboju
Fetch upstream source tree from github git clone https://github.com/Reference-LAPACK/lapack.git Change-Id: Ib2a508b62b2e5ddce36f0872fd06e5ffdbef37b6 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-19automated: android: tjbench: add support for Android O and masterChase Qi
Change-Id: I909ec3a6285dd2a31bd60f3034c5875c3de06bc3 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-18tjbench: add support for O preview and masterYongqin Liu
in O preview, the name of tjbench command is changed to tj Change-Id: I47dac9bca6225181a2bdd5203d9577ec3115f19e Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2017-05-17automated: linux: libhugetlbfs: cleanup results summaryNaresh Kamboju
Print results status in lowercase pass/fail/skip testcase name containing /dev/full coming twice in results summary. once it is printing as pass as expected. next time the default lava results parser printing as "unknown". This issue is been fixed by changing the "/" to "-" Change-Id: Ib3f5be575e0b97c9c73b9e9e408218973d92a319 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-17automated: linux: ltp-realtime: Bump LTP version to 20170516Naresh Kamboju
Change-Id: Idafb588b44a0da4d10c29ebdad3e8986b5733b72 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-17automated: linux: ltp: Bump LTP version to 20170516Naresh Kamboju
Change-Id: I67b1d174fd1a679fd90dee85710c10a8975e2ea0 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-16automated: sysbench: run the number of processors threads by defaultChase Qi
Change-Id: I757fa6baf40d8bab674d1d931c17e9c8fab63fa3 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-15automated: sysbench: add percpu benchmarkNicolas Dechesne
On multi core platforms, running 'sysbench --test=cpu' is not deterministic, since we cannot guarantee on which CPU the test is being run. It is even more problematic on big little platforms. In order for sysbench to be useful, we need to run sysbench 'cpu' benchmarks in the following conditions: 1. with --num-thread=<number of cpu> 2. with --num-thread=1 , for each cpu , using task affinity So that we can benchmark the platform as a whole, and each CPU individually. This patch adds a specific 'percpu' mode that takes care of #2 above. For #1, I would be in favor of removing NUMBER_THREADS as a params, or at least set it to <number of CPU> by default, but this is not done in this patch. A tweak in general_parser() was needed to add the cpu ID to the test name, running the test with 'percpu cpu' as PARAMS gives the following output: sysbench,percpu0-total-time,pass,9.2697,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-total-number-of-events,pass,10000,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-total-time-taken-by-event-execution,pass,9.2674,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-response-time-min,pass,0.84,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-response-time-avg,pass,0.93,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-response-time-max,pass,6.10,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-response-time-approx,pass,1.12,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-events-avg/stddev,pass,10000.0000/0.00,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu0-execution-time-avg/stddev,pass,9.2674/0.00,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-total-time,pass,9.9461,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-total-number-of-events,pass,10000,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-total-time-taken-by-event-execution,pass,9.9425,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-response-time-min,pass,0.84,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-response-time-avg,pass,0.99,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-response-time-max,pass,4.46,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-response-time-approx,pass,1.23,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-events-avg/stddev,pass,10000.0000/0.00,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu1-execution-time-avg/stddev,pass,9.9425/0.00,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-total-time,pass,9.4700,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-total-number-of-events,pass,10000,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-total-time-taken-by-event-execution,pass,9.4663,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-response-time-min,pass,0.84,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-response-time-avg,pass,0.95,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-response-time-max,pass,1.78,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-response-time-approx,pass,1.12,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-events-avg/stddev,pass,10000.0000/0.00,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu2-execution-time-avg/stddev,pass,9.4663/0.00,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-total-time,pass,9.4429,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-total-number-of-events,pass,10000,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-total-time-taken-by-event-execution,pass,9.4406,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-response-time-min,pass,0.84,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-response-time-avg,pass,0.94,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-response-time-max,pass,3.67,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-response-time-approx,pass,1.09,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-events-avg/stddev,pass,10000.0000/0.00,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,percpu3-execution-time-avg/stddev,pass,9.4406/0.00,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-total-time,pass,9.8187,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-total-number-of-events,pass,10000,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-total-time-taken-by-event-execution,pass,9.8148,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-response-time-min,pass,0.84,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-response-time-avg,pass,0.98,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-response-time-max,pass,8.63,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-response-time-approx,pass,1.16,ms,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-events-avg/stddev,pass,10000.0000/0.00,times,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false sysbench,cpu-execution-time-avg/stddev,pass,9.8148/0.00,s,NUM_THREADS=1;TESTS=percpu cpu;SKIP_INSTALL=false Change-Id: I25b10f0b5f1f5083e82b813b10e9ddd0ae184033 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-05-15automated: kselftests download right archRiku Voipio
Unless speficically requested, download aarch64 copy when running on aarch64. Change-Id: I816b27ee9d4e501c3c37964d6ce1b64a5a719772 Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
2017-05-15plans: Added lapack tests to rpb_ee_perfomance test planNaresh Kamboju
The LAPACK project supports the package in the sense that reports of errors or poor performance will gain immediate attention from the developers. Change-Id: I8cdfb072da3ce285fc1a02b4b2c53253af614d1b Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-13boottime2.sh: ignore case for the grep commandYongqin Liu
"Starting service 'logd'" changed to "starting service 'logd'" in O preview build, so we specify -i for the grep command to search with ignoring case Change-Id: I3bc5d49d219e0646dea7be33f10d7e2d7c65caa8 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2017-05-12kvm-cloud: de-race shutdownRiku Voipio
- move to automated dir - start qemu with pidfile - call shutdown with 1m timeout in guest for clean poweroff - wait for qemu process recorded in pidfile to exit to avoid primary lava connection to poweroff the machine before guest has finished Change-Id: I105ec3bbe3246a05552b3d13123ef4a3c7ea4c82 Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
2017-05-12Merge "automated: android: add lapack test"Milosz Wasilewski
2017-05-12Add a script to populate local DNS in NFSNeil Williams
When used with a Debian initramfs (jessie, stretch or later) when the kernel reports dns0 : 10.15.0.1 when mounting the NFS, use this data to populate /etc/resolv.conf Avoids overwriting an existing /etc/resolv.conf Usage: ./automated/utils/nfs-dns.sh eth0 Change-Id: Ie61fde7081f71110d8a4bc6ebe154ca1cfc5beac
2017-05-12utils: skip file uploading on empty tokenChase Qi
Change-Id: Ib6a4dc1be433b161d665ddcf07d1eb8c3382ca9f Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-11test-runner.py: use python2, not pythonNicolas Dechesne
test-runner.py does not work with python3. The first error is the following: File "/work/linaro/test-definitions/automated/bin/test-runner", line 201 os.chmod('%s/run.sh' % self.test['test_path'], 0755) ^ SyntaxError: invalid token Until we have support for python3, let's make sure that we use python2, this is especially needed when trying to use distro where python3 is the default (e.g. archlinux). Change-Id: Id387bf0d6092c6555081eb891de12ff0a4df3bb7 Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
2017-05-11automated: android: add lapack testNaresh Kamboju
Change-Id: I636b0c0f5aeee5cbb600a1ece81328cddb0ba5a9 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-10automated: android: add pm-qa testChase Qi
Change-Id: Ibd4368cdc5ec7e0f2e0098125e9cc50f33f555c2 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-09tradefed: install vts dependenciesVishal Bhoj
Change-Id: Icfd1dcc82cfe605750d51e052930d9a69d8f985d Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-05-08android: tradefed: fix package installationChase Qi
* Added support for Ubuntu * Removed support for Debian Change-Id: I4cc4074e455b8007f8f68e33815312846c344435 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-08change disable suspendVishal Bhoj
Change-Id: Ic06f1a674a43e47847f6c82082c5b5210d79836d Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-05-08android: tradefed: Wait for 5 minutes before terminatingVishal Bhoj
tradefed itself will finish execution if device is not available anymore so lets give 5 minutes for execution to finish Change-Id: I2423ce7706ac0518792183aa4c890dfe381eed82 Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-05-05kselftest: moved to LAVA parsing in YAML fileMilosz Wasilewski
LAVA v2 is able to parse results as they come. This way timeouts and crashes don't prevent all results to be registered in LAVA. Change-Id: I75e1b9af8a5159844ae20ab97e3e1f5e89e3473d Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-05-04Automated: Android: Added support for running VTSMilosz Wasilewski
CTS test is almost compatible with VTS as they use the same test shell - tradefed. The CTS test was updated to also support executing and reporting VTS results. Change-Id: Ida3ceda733c5e74f87256cf2ebefbaf93a6fce74 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-05-04Exit on install_deps errorChase Qi
If test script runs with '-e' argument, test should exit on errors. If test script runs without '-e' argument, 'if [ $? -ne 0 ]' will throw an error message and exit on installation error. Change-Id: I6365801582deaf7f5f105fe7014eadc4d0d3cdf7 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-04automated: linux: add rcutorture testChase Qi
Change-Id: Idc1506e2025e26a6c6f361f78037a55c0672c974 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-04automated: linux: add RT pmqtestChase Qi
Change-Id: I1386f165df1c446f94b22be4253b919cd8540e0b Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-04automated: linux: add preempt-rt tag for hackbench and cyclictestChase Qi
Other tests from rt-tests already have this tag. With this patch, we will be able to filter them easily. Change-Id: Ic1e62ffe4680a09da73f5e62d136e82a79b5d37e Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-28automated: android: add refactored apk-automationMilosz Wasilewski
Change-Id: I5a81b3c4317dc56f90df37c5a8f83d2547542fab Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-04-27automated: android: add monkey test caseerin.liu
Change-Id: I3eeabe9bdeb74cc90d9f49d18359a4d7a304388e Signed-off-by: erin.liu <erin.liu@linaro.org>
2017-04-25android bootchart: add support for bootchart_parseYongqin Liu
also changed to use lava command for test result collection in script Change-Id: I0fdb8c71bd6de69e06de1bbbaae8ac5924842aee Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2017-04-21kselftest: fix results parserNaresh Kamboju
Change-Id: I591c4c3e7a6c1850d6160654418092f1cd96286a Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-04-20automated: doc: Added section on using test-runnerMilosz Wasilewski
The paragraph on test-runner was contributed by Nicolas Dechesne. The section was mostly copied from: https://github.com/96boards/documentation/blob/master/ConsumerEdition/DragonBoard-410c/Guides/Testing.md Change-Id: I217d33f2259ae16b924c6a0a8fec8c7690bcb65b Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-04-19automated: android: added disable_suspend functionMilosz Wasilewski
The function unlocks the device and sets it to be awake all the time Change-Id: I53e39a047e8983e2d08c53b650df9251a793e7ab Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-04-19nginx-server: add odp-dpdk-git testsJosep Puigdemont
Change-Id: I8d262f9e9ba0d29b902c15f6dad3e75df3b0d75c Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-04-19nginx-server: use configuration files and callbaksJosep Puigdemont
NGiNX can be installed in different places, and can be implemented using different IP stacks (Linux IP stack, OFP, ANS...). The nginx-server.sh script was meant to be used independently of he NGiNX version in use. Due to this it contains several initialization and configuration functions for each of the currently supported implementations. This could grow a bit too large for one single file, so we've opted for creating files with a set of required functions for each implementation. The file is sourced and it should define some functions used for the tests. The name of the file should be that of the CONFIG_TYPE defined for the test definition, plus the .sh suffix. The functions required to be implemented in the files are: - do_configure_system(): used to initialize the tests. Here any configuration required for the test, like NIC configuration, kernel module loading, etc, should be done. It is called once before any test starts. - do_start_nginx(): this function is used to start NGiNX. - do_stop_nginx(): this function is used to stop NGiNX. It may be called even if NGiNX is not running. - do_write_nginx_config(): this function is called before starting NGiNX, and it is meant to write a proper nginx.conf file for the current number of cores. - do_pre_test_cb(): this function is called before starting NGiNX, it can be used for anything deemed appropriate. - do_post_test_cb(): this function is called right after the test has finished, and before NGiNX is stopped. It can be used for anything deemed appropriate. All functions are called with the following parameters in this order: - cores: number of cores of the current test. For configure_sytem, this is the total number of physical cores available. - interface: this is the interface that NGiNX is expected to use. - IP: the IP that the web server is expected to use. The return value of the functions is not checked. Change-Id: I0321697509f9ac628e705c69285a7478dd993ace Signed-off-by: Josep Puigdemont <josep.puigdemont@linaro.org>
2017-04-19Rename test directory for disk partitioning testChase Qi
Change-Id: Idcd83a16e0f2f938f31eb312eccaa344b634512c Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-13utils: add support for LAVA's secrets featureChase Qi
* Use token from LAVA's secrets file if defined. * Renamed URL and TOKEN variables with ARTIFACTORIAL_ as prefix to make them more meaningful. Change-Id: I80775d0bf7492275a3f2ca73d54640ea488f4a35 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-13automated: linux: migrate pm-qa testChase Qi
Change-Id: Ic23d1690d871b8cf292f1b3c22d49b19326c3f00 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-12Don't run adb root if adbd is already run as rootVishal Bhoj
if adbd is already running as root, it will not restart adbd server when we call adb root. This will cause lxc-device-wait-add to be blocked on usb event. Here are problematic jobs: https://staging.validation.linaro.org/scheduler/job/169678#L2308 https://staging.validation.linaro.org/scheduler/job/169679 Change-Id: Iefc65d6241898877b7b26bd8f5826c16e1247f7b Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-04-11automated: android: migrate boottime2Chase Qi
Migrated from android/boottime2. Most of the code in device-script.sh remain as it was. To keep consistence and cope with v2 structure: * Removed lava-specific steps from test script. * Copied the required functions from android/script/common.sh to device-script.sh Change-Id: I3560fe5dd863a1d32ced14806e852170494ea293 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-11automated: android: add media codecs functional testsChase Qi
Change-Id: I7f0a8f2696d09c9c278bacb5d9f7baa9b1918f37 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-04-11microbenchmarks: Pass gerrit id details to post-build-reportVishal Bhoj
Change-Id: I71f4c6b0deb9acc53f2aca426e0dac639532e568 Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-04-10fix adb root issueVishal Bhoj
Based on inputs from LAVA team, the right sequence of calls is: lava-lxc-device-wait-add lava-lxc-device-add adb wait-for-device Change-Id: I345aa85910ee68c79cf87b3447104b31d26d936f Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-19 20:25:11 +0000