summaryrefslogtreecommitdiff
path: root/tcwg_kernel-build.sh
AgeCommit message (Collapse)Author
2023-06-08round-robin-baseline.sh: New script to host update_baseline() codeMaxim Kuvyrkov
Move code for update_baseline() from round-robin.sh to its own script -- round-robin-baseline.sh. This allows us to move update_baseline() step after notification step, which simplifies notification logic. For the main workflow round-robin-baseline.sh is now called from tcwg/round-robin.yaml.inc for successful builds. The main reason for this reorganization is to simplify re-processing of results. Change-Id: I038bfc1ac68b1d6bb6dfd6f36dfb559d8e481583
2023-05-12tcwg_*-build.sh: remove rr[start_date], as it is now uselessLaurent Alfonsi
Change-Id: I4a99ca2392270d3a7fe64bb2e18d867fa452e241
2023-03-24round-robin.sh: Unify implementations of breakup_changed_componentsMaxim Kuvyrkov
Change-Id: I046212911cdd4d50ed4590e5d1cfdce19e577422
2023-03-08Do not "<output> | grep -q", use "<output> | grep -q ... >/dev/null" insteadMaxim Kuvyrkov
When 'grep' in invoked with -q option, it exits with the first match, and closes the input pipe. This will break the pipe and scripts running with "-o pipefail" will get unexpected results. Replace "<output> | grep -q", with "<output> | grep -q ... >/dev/null" throught our scripts. Leave "grep -q ... <input_file>" intact, since pipes are not involved. Change-Id: I9438919276c0d0e235600384a979b511e1a4edff
2023-02-21tcwg_bmk-build.sh: Build LLVM for ARMv8 hostMaxim Kuvyrkov
Flang does not support cross-compiling, so switch building LLVM to arm64/armhf host containers (done in yaml files). This also simplifies LLVM benchmarking flow, since we no longer need to build GNU toolchain to get target sysroot. One restriction with this approach is that host/build and target/benchmarking containers must use the same image. Otherwise we may get library compatibility problems at benchmark runtime. We are now linking applications against Ubuntu system libraries. Change-Id: Ide1a3f8f43ad23feeebd24f0b213b5c3f7e37122
2023-02-07tcwg_kernel-build.sh: Handle a corner-case when linux commit removes filesMaxim Kuvyrkov
Change-Id: I5118abb3ba34cbafae471e4299823a272fe98cf8
2022-12-06Apply llvm-vect-metric.diff patch to LLVM only for VECT configs.Prathamesh Kulkarni
Change-Id: I975ee6570104ce7f46bdedfe1fc4c25a958cec73
2022-10-04round-robin.sh,tcwg_bmk-build.sh: Can build llvm for flangLaurent Alfonsi
Change-Id: Ic97dd373bcc46b770cb73df2ff88379549bdf5f6
2022-09-22round-robin.sh (print_results_date): New functionMaxim Kuvyrkov
... to produce a reasonable date to associate with results / artifacts. Change-Id: I8cf41270abc5b642ca7a86aab0dcc0021852306b
2022-09-01round-robin.sh, dashboard.sh: factorize all dashboard routinesLaurent Alfonsi
... and simplify. During round-robin.sh CHECK_REGRESSION stage : - Creates file artifacts/dashboard/dashboard-generate.sh Simple call to jenkins-scripts/dashboard-generate-squad.sh with the proper parameters. - And simply execute it. This creates artifacts/dashboard/squad/ that contains all json files and a scripts push to squad (dashboard-push-squad.sh) During round-robin.sh UPDATE_BASELINE stage : - Creates file artifacts/jenkins/dashboard-push.sh Simple call to dashboard-push-squad.sh During last steps of the jenkins projects : - invoke artifacts/jenkins/dashboard-push.sh After the job end : - If we want to generate a dashboard info from an artifacts dir, we would have to do : $ artifacts/dashboard/dashboard-generate.sh - If we want to push a dashboard results from an artifacts dir, we would have to do : $ artifacts/jenkins/dashboard-push.sh - If we want to iterate over a full branch of base-artifacts, and for each one generate and push the dashboard, you would ave to do : $ jenkins-scripts/dashboard-push-one-branch.sh --top_artifacts <base-artifacts> --baseline_branch <git branch> Change-Id: I84b6da3460d1b9c6676a4ad93b2894e5bac968d5
2022-08-29tcwg_kernel-build.sh: Cleanup / simplifyMaxim Kuvyrkov
Change-Id: I49b75de3d9090b19c103b2f047f0167ca9bc94e3
2022-08-15round-robin.sh: Use git revisions instead of git parametersMaxim Kuvyrkov
... to determine which components have changed, and how to proceed with reduction or bisection of regressions. Using git revisions will make detection of changes accurate, instead of conservative -- before this change we assumed that all components with "==rr[COMPONENT_git] url#sha1" settings have new revisions, even when their "sha1" matched baseline "sha1". Change-Id: I89211e5c0949b9d9b78d1d47f082a07e05c75e32
2022-08-08round-robin.sh: Rework handling of ${rr[update_baseline]}Maxim Kuvyrkov
* rename "update" to "onsuccess" * rename "reset" to "force" * merge "push" into "force" Change-Id: I3e2f960f6aaf1215e1c035a706a6ba9a8fb4450b
2022-08-08round-robin.sh: Rework naming of ${rr[mode]}Maxim Kuvyrkov
... from baseline/jenkins-full/bisect to build/bisect. Change-Id: Ia9491ff53233a4a2f2c13e8bc958a3f7bede0f43
2022-08-08round-robin.sh: Delete push_baseline() stepMaxim Kuvyrkov
... and merge functionality into update_baseline() step. Defer "git push" of baseline branches to the CI job (aka .yaml file), which prevents non-CI executions of tcwg_*-build.sh scripts from affecting state of CI loops. Change-Id: I2d0485c5283aa92d1aefdbb2cdc97786f1679377
2022-05-26tcwg_kernel-build.sh: Add extra targets to LLVMMaxim Kuvyrkov
... to support using clang as HOSTCC. Change-Id: I9d46ea6604f2c3ac48a195de8ee20f848568a3f5
2022-05-26tcwg_kernel-build.sh: Report toolchain versionsKees Cook
Report the toolchain versions after locating them. Signed-off-by: Kees Cook <keescook@chromium.org> Change-Id: Icc137db66c5d40ec0120e7874a4ccfdf4b47d7c5
2021-09-23tcwg_kernel-build.sh (no_regression_p): Simplify, use no_build_regression_pMaxim Kuvyrkov
Change-Id: I103fe5f44dc483be2279858e8a54f417d7f5831e
2021-09-20tcwg_kernel-build.sh: Fix bisection of binutils failure in LLVM ci_configsMaxim Kuvyrkov
Change-Id: I920dd4307fdbaaa45c29256c7d93d72979190dc2
2021-08-11tcwg_bmk-build.sh, tcwg_kernel-build.sh: Fix shellcheck warningsMaxim Kuvyrkov
Change-Id: I5878531b2187e0c9ad972d7fcdf643b65ee94ebc
2021-08-08jenkins-helpers.sh: Fix and improve uses of print_gnu_targetMaxim Kuvyrkov
Change-Id: I80bbc452bf0c10d25220dffe5e03252d06d4c668
2021-07-21round-robin scripts: Merge <foo>_url and <foo>_branch into <foo>_gitMaxim Kuvyrkov
Merge git "_url" and "_branch" specification into a single "_git" argument that's constructed from "URL#BRANCH". This argument also has a special value "baseline", which before was a special value of "branch". This rework makes handling of git references simpler and cleaner, especially on the jenkins side. Change-Id: I1c1c07629728c693b8350dfa1b61fe767d5b0a0b
2021-05-03tcwg_kernel-build.sh: Refactor ccache logicMaxim Kuvyrkov
Change-Id: I3a5592e85f4edede3f3854b2e9e915208c5ab15e
2021-04-30tcwg_kernel-build.sh: Speed up bisections of qemu regressionsMaxim Kuvyrkov
... by using ccache for linux builds. Change-Id: Ifc7be118f3d5a5684f001d577d5f265c5d77e12d
2021-04-30tcwg_kernel-build.sh: Fix metric setting for qemu buildMaxim Kuvyrkov
When bisecting boot_linux() failure due to QEMU change we start at build_abe-qemu step, and get result metric set to "-1" by build_llvm step. Consequently, we consider boot_linux() failure as "uninteresting" regression. Change-Id: I364abb5ff828f1555fa82285dfaf58f9819d58e5
2021-02-02tcwg_kernel-build.sh: Use LLD for all LLVM configurationsMaxim Kuvyrkov
Change-Id: I3cbc9d522a31aa1d1de02c5122f66034ea8167ca
2021-01-18tcwg_kernel-build.sh: Use GNU Binutils for LLVM buildsMaxim Kuvyrkov
... since, apparently, clang needs stand-alone assembler for cases that integrated assembler can't handle. This used to work because we had binutils-arm-linux-gnueabihf and binutils-aarch64-linux-gnu packages in the base docker images. Change-Id: I0467b6c062d2e0f090c7e8628c07b6da07cdf4c9
2021-01-04Update expected x86_64 host tuple for ABE buildsMaxim Kuvyrkov
... from x86_64-unknown-linux-gnu to x86_64-pc-linux-gnu after updating config.guess in ABE. Change-Id: I81333036fb8d183ad81e3fa5b5836645e8b6220f
2020-12-21tcwg_kernel-build.sh: Rollback building with LLD for ARM's configsMaxim Kuvyrkov
Change-Id: I11155cfde4134e6662eeeee916e1f1882591e2c1
2020-12-19tcwg_kernel-build.sh: Use LLD for all LLVM configsMaxim Kuvyrkov
Change-Id: I61ecb7d40e205a7e7a3f7fe8d2ff8f6ddce4dde1
2020-12-15Revert "tcwg_{bmk,gnu,kernel}-build.sh: Update to not override passed ↵Maxim Kuvyrkov
rr[components]" This reverts commit 5a80a7248f90c25f983f7412d2432f8f1ad26368. Reason for revert: This was a bad idea. Change-Id: Ib9deb1247ded5714cccc88baf5535892e78f5baa
2020-12-15tcwg_{bmk,gnu,kernel}-build.sh: Update to not override passed rr[components]Maxim Kuvyrkov
Change-Id: I1b2dbfb294e72bab2a14cbb91a4dec03b58a37fb
2020-12-15tcwg_kernel-build.sh: Add missing handling of QEMU componentMaxim Kuvyrkov
Change-Id: Ia67f3edddc29b2f2adb31ea31cbca4da843e3b9d
2020-12-14tcwg_kernel-build.sh: Test booting using just-built QEMUMaxim Kuvyrkov
... instead of distro-provided version of QEMU. Change-Id: I7d3d773b98c21813d07df84150a7028b8405a460
2020-12-11tcwg_kernel-build.sh: Workaround missing QEMUMaxim Kuvyrkov
... which we removed from docker images in favour of building from source. Change-Id: I828d78e2091f0b537207975bb19c23621521ad69
2020-10-27tcwg_kernel-build.sh: Print the toolchain_type when unsupportedChristophe Lyon
This is similar to what tcwg_bmk-build.sh already does. Change-Id: Ifc7876307c164324fbe91049ea3030bd00a7d801
2020-05-22tcwg_bmk-build.sh/tcwg_gnu-build.sh:David Spickett
Mark expected exits It is expected that commands run by run_step can fail and return non zero. This causes the build scripts to exit early and print misleading traceback. So that we only catch unexpected exits, mark expected failures with a magic return code. Only print traceback when the code is different from this. Change-Id: I8b92ca1cb941481d4daa8ab87ee74b2b394b24ac
2020-05-05jenkins-helpers.sh: Add pwd check functionDavid Spickett
Used in other scripts to check that pwd isnt / before deleting things. Change-Id: Id3e284a486f300c97e92f730e84183340c999dd9
2020-04-30round-robin.sh/tcwg_kernel-build.shDavid Spickett
When looking for errors, exclude other uses of grep looking for errors. Without this we get job names with the entire grep line in the title. We could still get spurious names if there were no error lines found, but at least they'll be shorter. Change-Id: Iceb0279dff2bb51ff59977efaf3f0d523c91ce7c
2020-04-28Shellcheck fixes round 3David Spickett
Now using the version that's present in the bionic images. Meaning we have more warnings but can also set the minimum level to ignore a lot of them. Fix the check for whether shellcheck supports --severity. (we have pipefail on, but we expect shellcheck to fail in this case) Change-Id: I296b8554591b7d327c75393e3924184abc6512b7
2020-04-23Shellcheck fixes round 2David Spickett
Also update the sanity check script to include .job files. Change-Id: Ibf72d9ff346ff7b012d4e282a6ac8f3b7e73590d
2020-03-22round-robin.sh: Merge info from "regressions.txt" to "results"Maxim Kuvyrkov
Change-Id: Iba2ba392d0ed6656548ead3245d4d7906bbdfbc0
2020-03-22round-robin.sh: Avoid changing top-level artifacts in no_regression_p predicateMaxim Kuvyrkov
The no_regression_p() predicate function is called from two steps: check_regression and update_baseline. While we want no_regression_p to output regression info during check_regression step, we don't want that during update_baseline (during update_baseline no_regression_p is called to compare current results against pre-baseline ones). To fix the above we generate regression information in artifacts directory of the current step, and copy it to the top-level only during check_regression step. Change-Id: Ib03d82f7afe60c406c7942f2fa086855df371171
2020-03-11Add annotations to round-robin's "results" fileMaxim Kuvyrkov
Annotations provide human-readable comments and data on regressions. For this we ignore all lines in "results" file starting with "#". Change-Id: I31328fe2c13895a7379137b00891a9d2fc0a39d7
2020-02-18Round-robin scripts: improve regression email contentsChristophe Lyon
When a regression occurs after the build phase, don't include harmless errors from the the build stage, and provide info from the test phase. Change-Id: I75ce76e81ecbda4492bb49e7d765c25cac599f12
2020-01-28tcwg_kernel-build.sh: Robustify boot testMaxim Kuvyrkov
... to reduce spurious failures. Pass --foreground to timeout to avoid TTY and signal artifacts, extend timeout (we get 30s boot time on some kernels), and use non-multiplexed qemu serial. Change-Id: I1e4574c815426e8d06d0e052e8eb4710771be26d
2019-12-05tcwg_kernel-build.sh: Implement rr[breakup_updated_components] hookMaxim Kuvyrkov
... and fix tcwg_bmk-build.sh's hook along the way. Bash's read() accepts only complete lines, so we need to add newlines (using echo). Change-Id: I62950c598872dd23bef4aea3d981f09c1fa14945
2019-08-12tcwg_kernel-build.sh: Increase timeout for qemu to boot kernelMaxim Kuvyrkov
We are hitting the timeout on some of the good builds. Change-Id: I3187d8cec3031f907b342eed65534879d6150f76
2019-07-26round-robin.sh: Stop bisecting non-interesting regressionsMaxim Kuvyrkov
Currently we bisect regressions in all components equally, even when it means running benchmarks to bisect binutils, gcc or llvm internal build failures. This patch changes that by starting bisections only when the build progressed far enough to be interesting. For tcwg_kernel this means to reach kernel build step; for tcwg_bmk this means to reach benchmarking step; and for tcwg_gnu nothing is changing for now. Change-Id: I1c7c15e84fc3fd219108b0af549594be6b381768
2019-04-27tcwg_*-build.sh: Don't require repo urls when we don't need themMaxim Kuvyrkov
We need repo URLs only when not using baseline branches (baseline branches are stored in known repos at git.l.o/toolchain/ci/*). We refactor build_abe() to not rely on repo URLs, ABE only uses last element of URLs path, so we fudge URL for it. Change-Id: I19b6b5da51c8737b7176492e04e734f618d1d606
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-26 05:07:46 +0000