summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-08-24Increase memory allocation for build containersdiana/release/11.0.0Diana Picus
Building flang needs more than 1GB per core. Try with 2GB instead. Change-Id: Ibe1a06d5e529eaad9ea3b4a9d6c569a1171bed9c
2020-08-24tcwg-llvm-release: Deal with flang and mlirDiana Picus
Flang and MLIR are new to the monorepo. By default, MLIR is enabled and flang is disabled. This patch enables flang on aarch64 (and x86 for comparison purposes). It also disables MLIR on armv7 since we do not support it (there are many test failures since 32-bit support in general is missing / incomplete / untested in MLIR). Change-Id: I2cfa31762001fbffc0c41717d3d3792a5ac757d7
2020-07-30round-robin.sh: Compress testsuite logs to save disk spaceMaxim Kuvyrkov
Change-Id: I73e2ef1da8cb474b18a366ecccae09ab83185480
2020-07-29MakeRelease.job: Fix use of port 22 to upload binary tarballs/take #2Christophe Lyon
Sigh, the previous patch used a syntax that applies only with using an rsync daemon (with rsync: protocol). Let's use -e 'ssh -p 22', like we do in other scripts. Change-Id: I41d3cb7ff843abcc56a94775e2fd04c5f317f7fa
2020-07-29MakeRelease.job: Fix use of port 22 to upload binary tarballsChristophe Lyon
Adding "-p 22" does not work for rsync, which expects the port number as a ":port" suffix for the host specification. Change-Id: I41dbfbe11f73fd02386f90a0266488557a0009c7
2020-07-10MakeRelease.job: Use port 22 to upload binary tarballsChristophe Lyon
Our default ssh config points to the host container where the directory containing the binary tarballs is not mapped. Use port 22 to upload them to the right location. Change-Id: I5f18365f9d36541e3133ca81ed9ac2ac560bef30
2020-07-08round-robin-bisect.sh: Post jira comments for tcwg_{binutils, cross, gnu}Maxim Kuvyrkov
Change-Id: I9fd245d77848f0d8d1d2117ed117b1f17e2bcdd5
2020-07-08tcwg_bmk-build.sh: Skip builds that couldn't finish benchmarkingMaxim Kuvyrkov
Tcwg-benchmark-* jobs fail only due to infrastructure reasons. Benchmark build and run failures are assigned special 99..99 and 88..88 metrics, but the jenkins build succeed. Change-Id: I71639b6d4e1fd2a139390884c0d1eae0bb89a95d
2020-07-08tcwg-benchmark.sh: Move power-cycling code here from .yaml filesMaxim Kuvyrkov
Change-Id: I8c6ec7118f9015fa3ad3ea8c344fb66b5bf9a357
2020-07-06tcwg_bmk-build.sh: Gracefully handle benchmarking board going offlineMaxim Kuvyrkov
... which is represented by tcwg-benchmark-* build returning UNSTABLE result. Change-Id: Ifa8be8e601ef93412a3a62d9bed4a078fe38fa49
2020-07-03tcwg_gnu-build.sh: Use results comparison scripts from gcc-compare-resultsChristophe Lyon
They are based on GCC's compare_tests but support a list of unstable tests. Change-Id: Icc582846d2cdd5df895cad373e3021fc7e154fbd
2020-07-02round-robin.sh: Fix check_regressionMaxim Kuvyrkov
Don't expect ${c}_url defined when ${c}_branch is "baseline". Change-Id: I436efd471dd55655ea9a125194823ed084552ecc
2020-07-02tcwg_bmk-build.sh: Workaround missing benchmarking resultsMaxim Kuvyrkov
... which are referenced in base-artifacts repo. This problem occurred when bkp-01.tcwglab, where we store benchmarking results, ran out of disk space, so tcwg-benchmark-* jobs could not upload their results. This didn't stop tcwg_bmk-build.sh to treat this failure as regressions in various configurations, which were then bisected down to the first commit. At that point baselines for individual configurations were reset to the new failing state, and references to the missing results were committed to base-artifacts repo. We will fix this problem in several ways: 1. We have added tcwg-cleanup-stale-results.sh to parse base-artifacts repo and delete results with no references from that repo (thus preventing bkp-01.tcwglab from running out of disk space). 2. We are adding this workaround to purge broken entries from base-artifacts repo. 3. We will commit .csv summary of results into base-artifacts.git so that we won't need to rely on on-the-side storage of perf.data files. Change-Id: I922df040292c1686f4461052d21bdeb98bea0b03
2020-06-27round-robin.sh: For all .sum files also copy matching .log filesMaxim Kuvyrkov
Change-Id: Iad5af1671239484b5a55830c75c23a8158d56252
2020-06-22tcwg-start-container.sh: Handle containers stuck in restart loopMaxim Kuvyrkov
Change-Id: I6e32f290ba92449905ca0c215ec13d0384ada938
2020-06-19tcwg_gnu-build.sh: Fix order of cross-toolchain buildMaxim Kuvyrkov
Change-Id: Ibb21daee5883fade944073ef4cdb1d57ba905548
2020-06-19tcwg_gnu-build.sh: Add *-build_cross and *-check_cross configurationsMaxim Kuvyrkov
Change-Id: Ic648660f11602ebf82a34bd95469ba1018592287
2020-06-19tcwg-update-llvmbot-containers.sh: Put LLDB bots on tcwg-amp-05Maxim Kuvyrkov
Change-Id: I758566e05736de9c62ba8431843cb8c1c482b2d3
2020-06-09start-container-docker.sh: Assert that the task parameter has an acceptable ↵Christophe Lyon
value This is currently either build, bench or test. Change-Id: I96fa7adad47e0bb8d95668462218fb21d08a2cd7
2020-06-09tcwg-update-llvmbot-containers.sh: Update bot names (fixup)Maxim Kuvyrkov
Change-Id: I4a6ce34a182245a77d5bb3e7ff59ac9667906430
2020-06-09tcwg_gnu-build.sh: Fix shellcheck warningsMaxim Kuvyrkov
Change-Id: I6735863b1937995e6e4883e0a811376222219bf4
2020-06-09tcwg-update-llvmbot-containers.sh: Update bot names to match zorgMaxim Kuvyrkov
See https://reviews.llvm.org/D80573 Change-Id: I6deee117fab24fcf91a9e283fec3898124accc9b
2020-06-01tcwg-llvm-release: Use port 22 for copying binariesDiana Picus
We're doing this because our .ssh/config connects to the host container on dev-01 by default, and that doesn't have the upload directory mapped. Change-Id: I80305eb168abd62eab95ce6c627178593a52a905
2020-05-31round-robin-bisect.sh: Fix more missing instance of passing $bad_urlMaxim Kuvyrkov
Change-Id: Iffa2dff5d9a937c94e9c69a4102c4739788ddefb
2020-05-31round-robin-bisect.sh: Fix linux-next rebase workaroundMaxim Kuvyrkov
Change-Id: I472bf099af6d2ab6fb9dbb70c939634aaefafd48
2020-05-31round-robin-bisect.sh: Fix references to $linux_urlMaxim Kuvyrkov
... replace them with $bad_url. Change-Id: I73224638e852367ed8c42a178f36e9f7bf6c401e
2020-05-31round-robin-bisect.sh: Fix shellcheck warningsMaxim Kuvyrkov
Change-Id: Id9495fa4785702f155bc845f4db1f8b7691e9a48
2020-05-29round-robin*.sh: Pass component URL alongside component branchesMaxim Kuvyrkov
... when not using special "baseline" branch specifier. Change-Id: If3aabeb1a2729b8d270d842fdd0ad5809ebc990d
2020-05-29round-robin.sh: Use implicit baseline when triggering bisectMaxim Kuvyrkov
Change-Id: Ide6dbb22152a5096ba21667f497f2571384579de
2020-05-29tcwg_bmk-build.sh: Add compatibility with older baseline resultsMaxim Kuvyrkov
Change-Id: I79dc695cc2d4955516fc2ab55dfed031501d0a6b
2020-05-28tcwg_bmk-build.sh: Rework details of arm/thumb mode selectionMaxim Kuvyrkov
... while keeping general logic the same. This approach sanity checks that 2 configurations of cflags in *-vs-* configurations are using the same mode. To use different arm/thumb modes in *-vs-* configurations we would need to build GCC and Glibc 2 times, which we could do, if required. Change-Id: I3d9e6152b8b01f561e7e60557ccc680e99728103
2020-05-28round-robin.sh: Simplify "check" logic of build_abe()Maxim Kuvyrkov
Make "build_abe FOO" build FOO and make "build_abe check_FOO" run FOO's testsuites. This also makes tcwg_gnu's bisections more efficient, since we will ignore FOO's build problems in "check" configurations. Change-Id: I71c644e801828b94d880bed7fe440e4db3ffb40b
2020-05-27tcwg_bmk-build.sh: Adjust build scoresMaxim Kuvyrkov
Don't bisect benchmark build/run failures in *-vs-* configurations. These failures will also appear in non-vs configurations, and will be bisected more efficiently. Change-Id: Iad4a70a9b13e5509ff95e8dabde942a55f5c10b9
2020-05-27tcwg_bmk-build.sh: Fix loop waiting for benchmarking to finishMaxim Kuvyrkov
Remote_exec can exit before tcwg-benchmark-$hw job finishes, and that's the reason why we loop until we see "Finished: " on the job's console log. Change-Id: Ib579a48e6a1e1f20d9f6d748efcf693df27d5c20
2020-05-27tcwg_gnu-build.sh: Fix detection of build failureMaxim Kuvyrkov
Right now when bootstrap fails we get score "0" from reset_artifacts, and when bootstrap succeeds, but tests fail, we get same "0" from "build_abe bootstrap". This makes us ignore bootstrap failures. Fix by assigning reset_artifacts "-1" score. Change-Id: I021c7f2c987634cde0a8052191955ac7c8a05f2e
2020-05-27tcwg-update-llvmbot-containers.sh: Move aarch64 full and global-isel botsMaxim Kuvyrkov
... to tcwg-amp-03. This reduces load of linaro-armv8-01. Change-Id: Ie742cd2af0dc4db70744a2bb91e2025b9a8b5ebe
2020-05-26round-robin.sh: Fix one more artifact in build_abe()Maxim Kuvyrkov
Pass same fake git URL to abe in both build and check steps. Before this patch we missed ".git" in the check step, which caused ABE to try rebuild the component. Change-Id: I0912854876b0054f413d168bd9152a2f530047a0
2020-05-23tcwg-update-llvmbot-containers.sh: Move full and global-isel bots to AmperesMaxim Kuvyrkov
Move aarch64 full and global-isel bots to tcwg-amp-03 and armhf full and global-isel bots to tcwg-amp-04. Change-Id: Ie1192c3c908a540cb54494ad54ba90a6dad9a507
2020-05-22round-robin.sh (build_abe): Pass "--disable update" to ABEMaxim Kuvyrkov
... to avoid ABE updating sources. Change-Id: I82ef178a1dc1054b6642bd98335fcf013a425914
2020-05-22tcwg_bmk-build.sh/tcwg_gnu-build.sh:David Spickett
Mark expected exits It is expected that commands run by run_step can fail and return non zero. This causes the build scripts to exit early and print misleading traceback. So that we only catch unexpected exits, mark expected failures with a magic return code. Only print traceback when the code is different from this. Change-Id: I8b92ca1cb941481d4daa8ab87ee74b2b394b24ac
2020-05-22twcg_bmk-build.sh: Correct scope of build retDavid Spickett
The function runs in subshell so we were trying to return build_ret after that subshell had finished. Change-Id: I7baf15077451ca563dce09facf0131ceb38c8bc1
2020-05-20tcwg_bmk-build.sh: Always write benchmark results_id fileDavid Spickett
Before it was only written when the job failed. (though the build number was in the log if you looked hard enough) This can be used to go from run_step failing, to the log of the actual benchmark. Change-Id: I8189518f948f9ac76d090ff635209a5e69762a98
2020-05-20round-robin.sh: Fix "bootstrap check" build stepDavid Spickett
When I fixed a shellcheck warning here I changed an expression that previously would always delete the build folders. To one that would only do so when we were running a check step. Without deleting the build folders you got this in the "bootstrap check" step: 00:01:25 configure: loading cache ./config.cache 00:01:25 configure: error: `CC' has changed since the previous run: 00:01:25 configure: former value: `gcc' 00:01:25 configure: current value: `armv8l-unknown-linux-gnueabihf-gcc' (since the compiler built in "bootstrap" has been added to the PATH) The old behaviour works but only acidentally. Instead we should only update the source checkouts during non check steps. This means that ABE doesn't see a change and try to reconfigure. With that, there's no need for us to clean out the .stamp files for a check step anymore. Change-Id: Ifc5574fe422ae8a01618a1de02affceb7733ae0b
2020-05-20jenkins-helpers.sh: Fix run_step starting step logicDavid Spickett
This if was originally written with -o -a but changed to || and && to fix shellcheck warning: https://github.com/koalaman/shellcheck/wiki/SC2166 Now we find out why shellcheck warns about that because the replacement expression was incorrect despite looking the same. This: X -o Y -a Z Was converted to: X || Y && Z Which does: (X || Y) && Z But the orignal behaviour is: X || (Y && Z) Change-Id: I0bebd9c008e605953c0af8d26df15d811816ff14
2020-05-19jenkins-helpers.sh: Handle corrupted repos in clone_or_update_repo_no_checkoutMaxim Kuvyrkov
In [1] we have a binutils repo corrupted, which prevents its update and most git operations. Git fetch returns: error: object file .git/objects/32/d1f47a126567f24be18e7cadf5d2490968c986 is empty error: object file .git/objects/32/d1f47a126567f24be18e7cadf5d2490968c986 is empty fatal: loose object 32d1f47a126567f24be18e7cadf5d2490968c986 (stored in .git/objects/32/d1f47a126567f24be18e7cadf5d2490968c986) is corrupt fatal: The remote end hung up unexpectedly This kind of problem happens due to Lab power outages, jenkins restarts, jenkins node container restarts, and other infrastructure artifacts. The solution is to remove a repo where a basic "git status" command fails. [1] https://ci.linaro.org/job/tcwg_kernel-bisect-gnu-master-arm-next-allyesconfig/78/artifact/artifacts/build-baseline/03-build_abe-binutils/console.log/*view*/ Change-Id: I33cd22787a62c3759a950f3a1cc84d923f2f370c
2020-05-18tcwg_bmk-build/tcwg_gnu-build: Add traceback printingDavid Spickett
The line number you get from the previous message is confusing. For example: ERROR at run_step:524 Means that something in the function run_step, which was called from line 524, called exit. Instead, add a Python style traceback that shows the call stack with file, line and the content of that line. (though unfortunatley we can't get the line no. of the exit itself) Example traceback: ERROR Traceback (most recent call last): File: /tmp/foo.sh, line 37 foo # hey! File: /tmp/foo.sh, line 32 bar # hi there File: /tmp/foo.sh, line 1 (trap handler, 'exit' call line is unknown) Change-Id: Id6a2734e2b5ef74d05cd51cdbde9a95e5d5a4b44
2020-05-18start-container-docker.sh: Only try to remove containers that existDavid Spickett
The current code is causing docker to print "ERROR <...>", which isn't an error for us and confuses the Jenkins log. Change-Id: I902bfb7ca66d781b23726785852381e41e4a76e4
2020-05-18start-container-docker.sh: Fix cleanup of $WORKSPACE/.lockMaxim Kuvyrkov
At the moment we add cleanup of $WORKSPACE/.lock only to v1 interface due to cat <<EOF CONTAINER_CLEANUP=${CONTAINER_CLEANUP}; <lock_cleanup> EOF not updating /current/ value of CONTAINER_CLEANUP, so <lock_cleanup> doesn't get added to ${prefix}container_cleanup() routine. This patches fixes this and somewhat simplifies the whole cleanup logic. Change-Id: I6b78e835876b6c1353b2a7501e3149c81ebbeefe
2020-05-18jenkins-helpers.sh: Fix log_url unbound errorDavid Spickett
local only sets scope it needs a value to actually exist. Change-Id: I6a790da6022b4ad95e1c5534b325f4c170131b74
2020-05-15jenkins-helpers.sh: Clarify HANDLING failure messageDavid Spickett
When a step fails and we carry on you get this: RUNNING <step> CARRYING ON after failure in <step> HANDLING previous failure in <step2> RUNNING <step2> Which in context can be read to mean that <step> failed then <step2> *also* failed. (especially if you don't know to expect another RUN message) Now we'll see: HANDLING <step2> will handle previous failure RUNNING <step2> Change-Id: I52da2ce8ea976524f366fb590c230303f027c644
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-25 17:14:55 +0000