summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-07-24manual: kvm64: unify file namesHEADmasterChase Qi
Change-Id: I9766b78208e6b186e59f8107a37374dd4462048f Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-07-21readme: fix a typo on running test planChase Qi
To run test plan with test runner, the right parameter is '-p' not '-a'. Change-Id: Ia8477c661791784ce6f98c2fc78dfc4ac4574056 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-07-20test-runner: fail gracefully if SSH connection is closedMilosz Wasilewski
Environment is collected after the tests are executed. In case SSH connection is closed during test execution, environment collection should fail gracefully so at least test results can be presented. Change-Id: I55e1874dae2f20f9960bf0a12d05a35a48d321fc Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-07-20test-runner: set a proper default for $HOMEMilosz Wasilewski
In case $HOME is not set, getting the environment variable returns None. This can't be concatenated with string without casting. This patch changes the default to empty string in case $HOME is not set. Change-Id: I4ba5c0e5a4952b31d6bac8e8cf225ef2ffa131d3 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-07-13android bootstat: add android bootstat testYongqin Liu
used for record boot event statistic data Change-Id: Ie7ea8b6f7d41dea338496e7d15eccdd5b3609cfb Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2017-07-13linux: ltp: Fix su01Dan Rue
The su01 test has three problems: - It requires expect (fixed in this patch) - It requires root's password to be set in environment variable PASSWD (fixed in this patch; hard coded to 'root') - It requires a fix to su01 in LTP itself - proposed change at https://github.com/linux-test-project/ltp/pull/183 Change-Id: I1a693a62058f59446b68fe48cc5739e09dc6c70c Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-07-13linux: ltp: Change LTP_TMPDIR to /ltp-tmpNaresh Kamboju
LTP_TMPDIR=${HOME}/ltp-tmp is causing below error so fixing by changing LTP_TMPDIR=/ltp-tmp Error: chdir03 2 TBROK : chdir03.c:153: rmdir(/home/root/ltp-tmp/ltp-G4ktVVX945/chdEY97gl/3090) failed: errno=ENOENT(2): No such file or directory Change-Id: I6026372ffd1dd234e76d3667e2546d1c40a5b801 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-07-11linux: ltp: Install libaio for ltp/aio0[12]Dan Rue
Change-Id: Ifc5a2494e2734ee2dd174af68d2f23293f347cb7 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-07-11linux: ltp: use mkdir -m 777 to create LTP TMPBASE directoryNaresh Kamboju
runltp script uses -d parameter to set the tmp base directory. Behind the scene, it's using already mktemp -d to create a unique directory. Simply create a base directory as expected by runltp script, with permission set to 777 for tests that call tst_tmpdir() and try to write to it as user nobody. It's fixing also the following error: chdir03 2 TBROK : chdir03.c:153: rmdir(/home/root/ltp-tmp.ePFn/ltp-Jey9cImTdA/chdhhnMyQ/3141) failed: errno=ENOENT(2): No such file or directory Change-Id: Ic3349fdb51919d220070af5ddcf50f031a57a3b4 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-07-07linux: ltp: create temporary directoryNaresh Kamboju
Create LTP temporary files under $(mktemp -d ${HOME}/ltp.tmp.XXXX) directory. This will fix readahead02 test failure so stop skipping it. with reference to bug, LTP: readahead02: readahead failed to save any I/O https://bugs.linaro.org/show_bug.cgi?id=2961 Change-Id: Ic923234332d1a8782118ed4560f9c9eb8c4c2308 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-07-03linux: ltp: skipfile-lkft-hikeyNaresh Kamboju
Lets re-enable test cases which fixed on latest LTP and cherry-picked in OE build. - faccessat01 - fchmodat01 with reference to Bug 2963 - LTP: faccessat01 1 TBROK : tst_sig.c:233: unexpected signal SIGSEGV(11) received (pid = 2573) Bug 2964 - LTP: fchmodat01: 1 TBROK : tst_sig.c:233: unexpected signal SIGSEGV(11) received (pid = 2592) Change-Id: I121058d47ed110e315f88d3b5ab43616be543b84 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-28linux: kselftest: Download kselftest tar ball from a given URLNaresh Kamboju
Change-Id: Id81ff15eb8086d44bd8dbc75ab2ce3156d9bf55b Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-27Add unixbench to rpb_ee_performance planDan Rue
Change-Id: I1b934127870b12dae97fddc70a81f562f14d854c Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-27test-runner: fix UnicodeEncodeError with Python2Chase Qi
decode('utf-8') was added to strip off b' prefix from Python3's output. The current code also runs that with Python2, it worked in most cases some how. Howerver, once test-runner output redirectory used, it hits UnicodeEncodeError when running tests include wget download step like ltp and libhugetlbfs tests. Usage: python2 automated/utils/test-runner.py \ -d automated/linux/libhugetlbfs/libhugetlbfs.yaml \ -o /root/output/libhugetlbfs \ > test-runner-stdout.log 2> test-runner-stderr.log Error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 11: ordinal not in range(128) This patch detects Python version at runtime, then uses decode('utf-8') for on Python3 pexpect and subprocess.check_output() output to remove prefix 'b'. Change-Id: Ie52192787a8e4cd0550e1cd05b1ca53a02f4ab63 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-23Skip ltp rwtest03 and rwtest04Dan Rue
Per https://bugs.linaro.org/show_bug.cgi?id=3068 Change-Id: I12fbb90d9248eea8aa2fd513359734026d53106e Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-23Merge "linux: fix nginx installation on debian"Dan Rue
2017-06-23linux: openssl: fix line break within awk statementChase Qi
The current direct line break within single quotes causes the following strange output, only the last two lines are expected. Using backslash for line break solves this issue. rsa2048-sign pass in sign/s rsa2048-verify pass 10.00s verify/s rsa2048-sign pass in sign/s rsa2048-verify pass 10.00s verify/s rsa2048-sign pass May sign/s rsa2048-verify pass 2016 verify/s rsa2048-sign pass 00:08:40 sign/s rsa2048-verify pass 2017 verify/s rsa2048-sign pass aes(partial) sign/s rsa2048-verify pass blowfish(ptr) verify/s rsa2048-sign pass -Wa,--noexecstack sign/s rsa2048-verify pass -Wall verify/s rsa2048-sign pass sign/s sign/s rsa2048-verify pass verify/s verify/s rsa 2048 bits 0.008347s 0.000209s 119.8 4789.6 rsa2048-sign pass 119.8 sign/s rsa2048-verify pass 4789.6 verify/s Change-Id: I4d68a95f3db0ec5b2c7c67d9ff236a834f8723af Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-23linux: fix nginx installation on debianChase Qi
On Debian jessie, nginx actually can be installed from the default repository, the version is older though. We just need to make sure that apache service is not running before the installation. Once the port nginx tries to use occupied, it causes installation failure. Change-Id: Ic21d033d1f951836950d9d27312a77c4f355afef Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-22HACK: android-vts: Remove timer-suspend testcaseVishal Bhoj
The timer-suspend case breaks adb connection and stops further test execution. This is a temporary workaround for now to have vts to be functional. Options in vts don't seem to work well currently to exclude this test. Change-Id: Idcb9a99747891e71f9b6c62c74e563e78f1e6ac3 Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2017-06-21linux: jessie: install golang, nginx and openjdk8+ from backportsChase Qi
Fix https://bugs.linaro.org/show_bug.cgi?id=3044 Change-Id: I126ac676681440bfe55d7805d4206244f0200b7b Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-21linux: ltp: install sudo for utimensat01 testChase Qi
This patch fixes the below error utimensat01 1 TBROK: can't read /etc/sudoers Change-Id: I8cfc8645007f38938d0e97aab6632ba84566813f Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-20linux: libhugetlbfs: fix pre-installed pathNaresh Kamboju
Generic way to detect run_tests.py is pre-installed. Change-Id: Ib55d9584709a6db83bcce1a125b04d917b6532b2 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-20linux: ltp: install genisoimage for isofs testChase Qi
Change-Id: I10749307b18e92a7414837c220393e2acf21a8b2 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-19linux: ltp: install quota to solve test dependencyChase Qi
* quota is required by quota_remount_test01 * cope with shellchecks Change-Id: Iac605a4d75d0652e305989511f1d7369603f8e9c Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-16Merge "linux: ltp: fix a typo"Milosz Wasilewski
2017-06-16linux: ltp: fix a typoChase Qi
Change-Id: Iee890456ab6190f48f79bcc815faec4b7780289a Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-16linux: docker: replace docker-engine with docker.ioChase Qi
Replace docker-engine with docker.io to fix https://bugs.linaro.org/show_bug.cgi?id=3044 Change-Id: I3eab1c0ee7ed1e1fdd320043e9f9e0a6bf5703e7 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-16linux: ltp: skip oom01 on RPKChase Qi
oom01 tries to kill bash and python which are needed by our test script. Prevent it from doing so causes 'Kernel panic - not syncing: Out of memory and no killable processes...' The issue needs to be fixed from ltp. Refer to https://projects.linaro.org/browse/QA-1662 Change-Id: I16ab3ffc6aa7a9e356464c9f6be8ad633a344c8b Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-14utils: add post-to-squad.pyChase Qi
Change-Id: I98e58c128def865c40ca84f6652963e0757259db Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-14lkft: ltp: skipfile-lkft-hikey: skip fchmodat01 and faccessat01Naresh Kamboju
With reference to bugs 2963 and 2964 Skip till build issue get fixed in OE build or upstream LTP - faccessat01 - fchmodat01 Change-Id: If2a961e3e78437e3cd1c6c71eaaab3ebe8f5ff49 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-14automated: android: enable CTS/VTS atomic result reportingMilosz Wasilewski
This patch allows to either record the CTS/VTS results as 'atomic' or as 'aggregated'. Atomic results show each test case while aggregated only present number of passed/failed for each module. Examples 1. aggregated arm64-v8a.CtsBionicTestCases_executed pass 1853 arm64-v8a.CtsBionicTestCases_passed pass 1850 arm64-v8a.CtsBionicTestCases_failed fail 3 2. atomic arm64-v8a.CtsBionicTestCases/DlExtRelroSharingTest.ChildWritesGoodData pass arm64-v8a.CtsBionicTestCases/DlExtRelroSharingTest.ChildWritesNoRelro pass arm64-v8a.CtsBionicTestCases/DlExtRelroSharingTest.RelroFileEmpty pass arm64-v8a.CtsBionicTestCases/DlExtRelroSharingTest.VerifyMemorySaving pass ... Change-Id: I6729235607cfa51b04dd7a66e4a8e49859c40876 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-06-13automated: linux: libhugetlbfs: Replace "=" with "_" in test case nameNaresh Kamboju
Change-Id: Icb892a45539f6119f66dc5509426b87e7b5e6f6d Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-12test-runner: Report environmental informationDan Rue
Add a dictionary key named 'environment' that identifies the host running the tests, whether that host is local or remote. For example, on a HiSilicon D03 system, the following data structure will be added to each result in results.json: { "bios_version": "Hisilicon D03 UEFI 16.12 Release", "board_name": "D03", "board_vendor": "Huawei", "kernel": "4.9.0-20.gitedc2a1c.linaro.aarch64", "linux_distribution": "centos", "packages": [ "GeoIP-1.5.0-11.el7", "NetworkManager-1.4.0-20.el7_3", ... "yum-plugin-fastestmirror-1.1.31-40.el7", "zlib-1.2.7-17.el7" ], "uname": "Linux localhost.localdomain 4.9.0-20.gitedc2a1c.linaro.aarch64 #1 SMP Wed Dec 14 17:50:15 UTC 2016 aarch64 aarch64 aarch64 GNU/Linux" } Support exists for Ubuntu, Debian, CentOS, and Fedora. Other distributions will print a warning and report an empty package list. Environmental data is added automatically in each test and may be skipped by passing -e or --skip-environment. This change led to a slight refactor in the way remote commands were being handled. Instead of percolating remote vs local logic everywhere, run_command will run an arbitrary shell command either locally or on the target host, and return the output. This allows all shell commands, whether local or remote, to be handled consistently. Change-Id: Ia94695751a04a7fc862a299526d33784c57a9893 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-12linux: fix disk partitioning pathChase Qi
Change-Id: I0042ca782ecda6ae2d20c6047b333ba9b4b53242 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-12automated: android: fix tradefed-runner.pyMilosz Wasilewski
This patch fixes the high CPU utilization when running tradefed (CTS) with big modules. pexpect buffer is now flushed often to ensure the search is only performed on the recent chunk of input. Change-Id: I33d68d7709dc73eb631f1e71c765ba3cc0be84ae Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-06-09ui-browser-test: support OE-RPBDaniel Díaz
Change-Id: Idfac8bf81622e18a06150fba224d8213f326a3c9 Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
2017-06-09test.sh: Support local testing environmentDan Rue
Add Dockerfiles that can build and run test-runner locally. test.sh provides a simple command line interface to the environment. This is just a start. We should be able to start implementing more robust tests and validations on top of test.sh and the 'test' dir. Change-Id: Idc1dc9a8752b244ba0c04427d9eb43ac19863377 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-09test-runner: Fix variable overloadingDan Rue
flake8 reported the following: test_runner.py:196:25: F402 import 'cmd' from line 4 shadowed by loop variable "cmd" is actually used three different ways in test-runner.py. First, it's an imported module. Second, it's a loop variable. Third, it's re-assigned inside the loop. Change-Id: If907182b4f8229e47321d3819772213bb88791dc Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-08automated: ltp: Adding skipfile-lkft-hikeyNaresh Kamboju
LTP test cases which are not supported on LKFT HiKey Open Embedded build are listed in this file "skipfile-lkft-hikey" to skip Bugs: LTP: readahead02: readahead failed to save any I/O https://bugs.linaro.org/show_bug.cgi?id=2961 Linux-4.4: LTP: bind02: socket() failed: errno=EACCES(13): Permission denied https://bugs.linaro.org/show_bug.cgi?id=2962 msgctl10 and msgct11 tests overheats HiKey board due to large number of fork() calls and message queues read/writes. [RPB] LTP: msgctl10 fork failed https://bugs.linaro.org/show_bug.cgi?id=2355 fork13 -i 1000000 fork test case overheats Hikey board and goes auto reboots. fork13 is also a peformance test case so skip it. Change-Id: I218ec4b5bb3b6d5605969e65830b9cc4cd63510e Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-07.gitignore: Ignore build-error.txtDan Rue
Change-Id: I1b6fcb6d808d96abad80d37e25ab4314d1a54ca0 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-06-06test-runner: add support for parse patternChase Qi
LAVA V2 is able to catch and save result lines that send to stdout instantly. For tests that run for a long time and generates a plenty of results as it runs, like kselftest and ltp tests, LAVA's live result collection feature could be useful. In Linaro Test Suite, to support completely local run and reduce the complexity of log parser debugging, we parse test log in test script and save results in LAVA's format to result.txt file. Then use 'send-to-lava.sh [path]/result.txt' to send result lines to stdout after test finished. This should work for most test cases, but maybe not good enough for tests mentioned above as the tests may crash or hang or cause Kernel panic or system halt during test run, in those cases, the result sending line has no chance to get executed. Using parse pattern in test definition file is one way to enable LAVA's live result collection feature. This patch is used to add parse pattern support in test-runner so that we can run the test cases using parse pattern on the local. It also could be useful for pattern debugging on the local. Change-Id: I5756af84f653ab00fc52c62ff888b71fe8817944 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-06automated: linux: kselftest: fix duplicate resultsNaresh Kamboju
Change-Id: I344e60f52a48cb14cf20101db3525ced0daaa911 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-06-02linux: sysbench: cope with shellcheckChase Qi
Change-Id: I43f893dafb9f0d7c245d2a6a9d76b20c2df0d394 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-06-01linux: sysbench: cope with LAVA V2Chase Qi
Split sysbench's result line avg/stddev into two metrics to avoid using slash in test-case-id and measurement. Using slash in test-case-id or measurement is valid in LAVA V1, but invalid in V2. Refer to lava-dispatcher's source code 'lava_dispatcher/pipeline/actions/test/shell.py' line #389 and #420. Change-Id: Ic3c411745c88827ebff78421e9ceebaec8f94cbd Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-31automated: linux: kselftest: Adding SKIPLIST and SKIPFILE parametersNaresh Kamboju
Known unsupported test cases list can be provided by using, SKIPLIST or SKIPFILE parameters which will be skipped from the run. Change-Id: I844dd6a9cded5a18bd1a21bc63320f4a3a3e7d83 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-30test-runner: Skip duplicate testsDan Rue
This commit adds checking to ensure that duplicate tests are not added to the test plan. Change-Id: I045c16467cc17812a4559a14ac66e55a9c33a189 Signed-off-by: Dan Rue <dan.rue@linaro.org>
2017-05-26automated/linux/ltp: renamed TIMEOUT variableMilosz Wasilewski
Change-Id: I021774e1f9581df05b69d026972ba50c201855fd Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2017-05-26automated: utils: use lava-test-case for result saving in LAVAChase Qi
In LAVA, use lava-test-case for result saving makes the upload helper script more reliable. For local run, saving file upload 'fail/skip' result to 'result.txt' isn't necessary as it is always skipped. And some test cases may don't use 'result.txt' at all, in these cases, we will need to handle file creating in the upload helper script which increases complicity. Change-Id: Ie9ed79b66bab6abc861c2a3da7a3c6fd21b19c88 Signed-off-by: Chase Qi <chase.qi@linaro.org>
2017-05-26automated: linux: ltp: increase timeout for slow machinesNaresh Kamboju
The default timeout is 5min per test case. Increase timeout for slow machines with the help of variable ${MINUTES_TO_TIMEOUT} from yaml file. Change-Id: Ibd4c022d15f203cc8eedca76ff754ab030f73391 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
2017-05-25automated: linux: optee-xtest: improve result parserNaresh Kamboju
Results print format changed on upstream optee_test tree https://github.com/OP-TEE/optee_test/commit/213ca8aaf4eafe9d26af76db7b31d9595ce57e6f This patch simplify result parser to make it more reliable. Change-Id: Iacc3282b5460c0d1d8bfbb237917c087e9657108 Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-08 11:37:22 +0000