aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-03-26v4l2-compliance: results parser fix upHEADmasterNaresh Kamboju
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-26automated/linux/piglit: Add support to use lava-test-setAníbal Limón
The lava-test-set provides better reporting previously the results were reported as: spec@arb_arrays_of_arrays@linker@vs-to-fs-unsized pass|fail|skip In this implementation the last function is reported as test case and the previous ones as test suite, lava-test-set start spec@arb_arrays_of_arrays@linker@ ... vs-to-fs-unsized pass|fail|skip ... lava-test-set stop spec@arb_arrays_of_arrays@linker@ Change-Id: Id776646d65282f060985b3bd707f11376bf4b085 Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-03-26automated/linux/piglit: Fix pycodestyle checksAníbal Limón
Change-Id: Icac05cf665a841cd1f950dca5fe5befaa2523a33 Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-03-26automated/linux: piglit_lava_parse.py create common function to print_resultsAníbal Limón
Change-Id: I49179153d74f427ff3c5b34c875d0b93ec57efb0 Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-03-25validate.py: force check all files end with .shChase Qi
When shebang line is incorrect, python-magic returns file type as plain/text, then sanity check will be skipped. This patch treat all files end with .sh as shell script and do force shellcheck test. Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-19linux/perf: Adding perf test case for LKFT test planNaresh Kamboju
Perf is a profiler tool for Linux and is used to collect and analyze performance and trace data. This test runs following perf commands: - record(record events for later reporting) - report(break down events by process, function, etc) - stat(obtain event counts) - test(Run sanity tests) Upstream link, https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/perf.txt Ref: Test perf in LKFT https://projects.linaro.org/browse/KV-195 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-18kselftests: fix tar magicPatryk Mungai
Not all versions of tar automatically detect compressed archives (eg. BusyBox v1.22.1 version of tar). Hence the [z] option needs to be specified in order to successfully untar a gunzip archive in these cases.
2019-03-14adb_join_wifi: use new non-debuggable apkYongqin Liu
to resolve the problem reported here: https://bugs.linaro.org/show_bug.cgi?id=4236 As there is a problem to fix the cached apk in lab as mentioned here[1], we changed to use new file name to workaround the lab problem. [1]: https://projects.linaro.org/browse/LSS-432?focusedCommentId=76245&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-76245 Change-Id: Ibc51a802117cecfe1d15b288bac9b55f6e0feec4 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-03-14chroot: add kselftest_chrootAnders Roxell
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
2019-03-14add chroot with ltp for nowAnders Roxell
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
2019-03-14ltp: drop parsingAnders Roxell
No need for the parsing section since in the run step we call send-to-lava.sh with the result. Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
2019-03-14sanity-check: print installation tips when required module not installedChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-14MultiNode Tradefed: Reset userdata before retryKarsten Tausche
If supplied, flash a userdata image to the devices before triggering Tradefed retry. This helps reducing invalid failures that are caused by tests bringing devices into a state where other tests cannot pass. Issue: INFRA-137 Change-Id: I4a6510ceab6aec7ce530b9f8e244e9655dc22b81 Depends-On: Ie21cc417c78ec88da523f0d14ea85c1e9bcb912c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14MultiNode Tradefed: Fix udev access for fastbootKarsten Tausche
Execute the Tradefed runner as the current user, which is root in the context of LAVA LXC environments. This prevents getting udev permission issues in fastboot or adb, if udev rules are incomplete for specific Android devices and the device IDs they might report in different modes. Without this change, the `fastboot reboot` calls in the runner don't have any effect on affected configurations. Change-Id: Ie21cc417c78ec88da523f0d14ea85c1e9bcb912c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Tradefed MultiNode: Refactor timeout handlingKarsten Tausche
Use timeout parameter of python subprocess commands instead of the system timeout command. Issue: INFRA-137 Change-Id: Id939f31d3d047a9d2c5d18ea4dd0151f04b567e4 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Move wait-and-keep-local-device-accessible into shell scriptKarsten Tausche
Having the code in a dedicated script file rather than in YAML inline simplifies coding, testing and validation. This also fixes minor shellcheck and bashate issues. Issue: INFRA-137 Change-Id: Ia090f08eb325b3b348a4863cfde4a250f771a7e2 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Allow reusing shell libs in Python 3 codeKarsten Tausche
This simplifies code reuse in the MultiNode Tradefed python runner and in upcoming changes. Issue: INFRA-135 Issue: INFRA-137 Change-Id: I6bbf2ca96cc74c2eabf7308c57be07064d81a8e4 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14android-test-lib: Don't modify shell debug optionsKarsten Tausche
Setting the shell xtrace option (set -x/+x) within the initialize_adb function changed it also for the calling context, which was probably not intended. Leave the decision to set this flag to the caller instead. Change-Id: I00628d3679369c68d090315472a6e890f5c1af7f Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-08android: tradefed-multinode: add lava job example for hikeyChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-08android: tradefed-multinode: fix test definitions repo linkChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-08linux: fuego: add lava multinode job exampleChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-08linux: iperf: add lava multinode job exampleChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-08Merge pull request #28 from chase-qi/file-typeChase Qi
sanity-check: improve file type determination
2019-03-08Merge pull request #31 from nareshkamboju/masterChase Qi
automated/lib/sh-test-lib: Adding report_skip api
2019-03-08automated/lib/sh-test-lib: Adding report_skip apiNaresh Kamboju
Shell test library have report_pass() and report_fail() Here report_skip() also needed so this patch adding it. Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-08Merge pull request #30 from nareshkamboju/masterChase Qi
kselftest: skip kernel crash test case test_tunnel.sh on all devices
2019-03-08kselftest: skip kernel crash test case test_tunnel.sh on all devicesNaresh Kamboju
Kernel panic while running bpf: test_tunnel.sh on linux -next and mainline This is been reported upstream and waiting for the fix. till it get fixed this test case test_tunnel.sh will be skipped on all devices and all branches Ref: https://bugs.linaro.org/show_bug.cgi?id=4307 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-07sanity-check: improve file type determinationChase Qi
* Use libmagic to detect file type so that we don't have to update the file for new files without extension. * Because test plan and lava test job definition yaml file also contain 'metadata' key, use 'run' key to determine test definition yaml file. * Skip sanity check on unknown file types. Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-07Merge pull request #25 from paolo-github/masterChase Qi
automated: linux: add the ssuite benchmark collection
2019-03-05LTP: skipping dio test runs on slow running db410cNaresh Kamboju
LTP dio tests getting timedout on db410c device so skipping most of them on db410c. Ref: https://projects.linaro.org/browse/KV-171 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-04LTP and kselftests: Adding stable rc 5.0 branch in skipfilesNaresh Kamboju
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-03-04automated: linux: add the ssuite benchmark collectionPaolo Valente
The S suite is a small collection of benchmarks for storage I/O. For the moment, this test makes it possible to execute the following two benchmarks with the suite: - responsiveness, by measuring start-up times of real applications under real background workloads; - throughput with processes doing filesystem or raw I/O in parallel (figure of merit measured by many other suites too). Signed-off-by: Paolo Valente <paolo.valente@linaro.org>
2019-03-04Merge pull request #24 from chase-qi/masterChase Qi
doc: a few fixes
2019-03-04doc: a few fixesChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-02-27linux: add fuego multinodeChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-02-21LTP: skip LTP dio long running tests on all devicesNaresh Kamboju
From KV-171 investigation LTP dio long running tests have been identified and skipping those test cases on all devices. Ref: https://projects.linaro.org/browse/KV-171 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-02-18Merge pull request #15 from kateyy/tradefed_shards_with_retryChase Qi
Add MultiNode TradeFed tests for --shards and with session retry
2019-02-18noninteractive-tradefed: use grep to workaround Bad substitution problemYongqin Liu
reported like following: /lava-610173/1/tests/1_cts-focused2-armeabi-v7a/run.sh: 35: /lava-610173/1/tests/1_cts-focused2-armeabi-v7a/run.sh: Bad substitution but not sure why this problem only happens for x15 jobs, jobs for hikey builds could work without problem Change-Id: I4dbaf0670f82bec69e2146d8cf91974feaf86dc1 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-02-15Add MultiNode TradeFed tests for --shards and with session retryKarsten Tausche
This builds upon the previous test setup and runner for TradeFed (CTS/VTS) with the following major features and changes: (1) Allow for "sharded" tests accross multiple DUTs (speedup test suite execution) (2) Use session retry for stabilizing final failure counts (3) Don't require DUT root access in any of the involved operations (4) Trying to be failure tolerant: Continue as long as at least one device is available. Re-establish lost connection if possible. (5) Drop support for VTS (temporary solution, it was not tested at all) (6) Add support for GTS and STS Details related to the points above: (1) Implemented using the LAVA MultiNode API. There should be one "master" job that executes the TradeFed runner and an arbitrary number of 'workers' that host additional DUTs and make them accessible via adb tcpip. Continuous messaging between master and worker is used to recover lost devices when needed. (3) Migrated features requiring root to non-root equivalents. (5) No testing was done with VTS, therefore it was partly dropped for now. (6) Apart from adjusted paths and command names, no changes were required compared to CTS. This change would ideally be a replacement of the existing TradeFed setup, or at least share more code with it. Remaining issues on that way are: - (5) - no VTS testing done. This relates to behavior of --retry, --shards, and related shell output - MultiNode environment expected by this setup. The runner could be further generalized to be usable from a regular (non-MultiNode) test submission, so that it would in a basic case not introduce any overhead compared to the existing runner. Change-Id: Idef4a5a9aac1f3cd8fc2aa1e609f544ee15ae528 Depends-On: I23f22344b9bd758d3898d4345204157cecd7d624 Depends-On: Ie011cb2cd899dac938066ca4eee7652b83ac38d4 Depends-On: I638d6a2cf44a5569569703308172e3056030783f Depends-On: I912ab5168bbe1b4fc0a3a8112db5cbc94a812b7c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-02-15Merge pull request #19 from mwasilew/tradefed-debugChase Qi
noninteractive-tradefed: add more debug
2019-02-15noninteractive-tradefed: add more debugMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-02-13LTP: skipping memory mtest06 and ksm01Naresh Kamboju
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-02-12Add MultiNode scripts for running tests on multiple adb TCP/IP devicesKarsten Tausche
With these test shells and functions, test implementations can make use of multiple devices by connecting remote devices via adb TCP/IP connections. On the MultiNode side, one master role would execute the actual test shell, whereas multiple instances of a worker role allow for remote access to their device adb TCP/IP. The following steps are required: * Start with completely booted and network attached devices. * workers: share-local-device-over-adb-tcpip.yaml - run `adb tcpip` on their local devices and share their IP address with the master * master: connect-to-remote-adb-tcpip-devices.yaml - Connects to all devices shared by the workers via `adb connect` (while also using its own local USB attached device) * workers: wait-and-keep-local-device-accessible.yaml - Wait for commands from the master -- mostly wait and do nothing else, but may need to reset their devices/network connections to make them accessible again if they got lost. * master: remote-adb-devices-smoke-test.yaml - Dummy action for the actual test job. A proper test job would execute some potentially long running tests on all available devices, instruct the worker to reconnect on connection lost etc. * master: release-remote-adb-tcpip-devices.yaml - Test executions end with the release command, so that workers exit their event loop. * workers: wait-for-release-and-reset.yaml - Final synchronization point between worker and the master, so that the master is in control when the workers are shutting down. Brings worker devices back into adb USB mode, so that they are usable for regular local test jobs. Change-Id: I23f22344b9bd758d3898d4345204157cecd7d624 Depends-On: Icd68e9de5a349880c52ec06229cd3f8bcb8eeecc Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-02-06LTP: moved skip list to qa known issues listNaresh Kamboju
The test cases known to failed are moved from skipfile-lkft.yaml to qa known issues report repository ltp-production.yaml file. The fixed test cases entries removed from skipfile-lkft.yaml file. Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2019-02-04android/tradefed: improve reboot variable comparisonMilosz Wasilewski
Make sure that comparison is case insensitive to catch any possible reboot in LAVA. Also align the comparison condition between different tradefed implementations. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-02-01tradefed: fix comparison for case sensitive "True" valueMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-01-25doc: fix rst list and block syntaxChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-01-11Improve sanity checkChase Qi
* Use 'git diff --name-only HEAD~1' to list changed files as "git show --name-only --format=''" doesn't work for pull requests. * Add instructions on the installation of a specific version shellcheck. * Add descriptions for the skipped pycodestyle and shellcheck codes. * Skip the following shellcheck as 'which' is widely used and supported and busybox doesn't support 'command'. SC2230: which is non-standard. Use builtin 'command -v' instead. Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-01-10Revert "android-test-lib adb_join_wifi: uninstall the package"Yongqin Liu
uninstall that package would cause wifi disconnected This reverts commit 1136333f3659ab251096848444d94ae7e478d31a.
2019-01-10LTP: sendfile09: skip creating large filesNaresh Kamboju
sendfile09 and sendfile09_64 tests creates 5GB file and calls sendfile(2) with offset at 3GB. These tests are time consuming so skipping on slow devices db410c and hikey. Change-Id: I9a215e1fd531dd31f1ffa5acc1a28485dd19dcec Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-07 06:14:40 +0000