summaryrefslogtreecommitdiff
path: root/tcwg_bmk-build.sh
AgeCommit message (Collapse)Author
2021-06-24tcwg_bmk-build.sh: Cleanup handling of hw_tag/hw/image_arch bmk parametersMaxim Kuvyrkov
Change-Id: Icf766e71b5afc085cc833717c2944d688dbc542c
2021-06-23tcwg_bmk-build.sh: Remove workaround for LLVM+BFD now that we use LLDMaxim Kuvyrkov
Change-Id: I52ebd8448bf66dbea7e592547b93fcf643b08d95
2021-06-18tcwg_bmk-build.sh: Update bmk job nameMaxim Kuvyrkov
Change-Id: I81eba110de87683c7a6be56855bc6e0dc4844170
2021-06-17Revert "tcwg_bmk-build.sh: Switch TK1 and TX1 benchmarking to new bmk job"Maxim Kuvyrkov
This reverts commit 4147f30f9230cb4a2daf3b21a2ce7d4102e8f3da. Reason for revert: New jobs now have same names and previous old jobs Change-Id: I26d247e264112fecdc4f599e108834bd8e609e5b
2021-06-16tcwg_bmk-build.sh: Update run_profile settingMaxim Kuvyrkov
Change-Id: I225e3db943c8d61abe083d6684419f679202d86b
2021-06-16tcwg_bmk-build.sh: Switch TK1 and TX1 benchmarking to new bmk jobMaxim Kuvyrkov
Change-Id: I72f1d422c87800820b1bee7f2a95edbf2989352f
2021-05-04Set metric from cflags.Prathamesh Kulkarni
Change-Id: Ie9ed975be89cbb39e195969ed3ef2913e28cbcd9
2021-04-21Add comparison for vect metric.Prathamesh Kulkarni
Change-Id: I1c5e4c6b0cc43fbdf2923484b8c643e2a7c9c51e
2021-04-05Refactor tcwg_bmk-build.sh and tcwg-benchmark-results.sh for metrics.Prathamesh Kulkarni
The patch adds an option to tcwg-benchmark-results.sh called "metric", and calls corresponding scripts to convert relevant metric-data to csv files. And adds metric to rr in tcwg_bmk-build.sh, which is set to "perf" by default. The rationale behind this change is to refactor the scripts to simplify adding more metrics in future. Change-Id: I3dfd2a572faeea4b0034fe6a6bb635824b3d60c3
2021-04-01tcwg_bmk-build.sh: Do not look for perf logs with target==arm_eabiChristophe Lyon
When benchmarking for arm_eabi, we do not produce perf logs, so inform tcwg-benchmark-results.sh about that. Change-Id: Ifa304bd42ca80cf5d7ba28d8bc3f7377bdbf6b77
2021-03-26Make build_abe apply an optional patch for $component specified by --patch.Prathamesh Kulkarni
The change lets build_abe take an optional argument following --patch, that specifies a patch to be applied for $component. And in tcwg_bmk-build.sh, apply linaro-local/vect-metric-branch to stage-2 gcc for applying vect metric patch. Change-Id: Ica927685e40ea0ae3988557e5cb561c83abf2f3b
2021-03-07tcwg_bmk-build.sh: Speed up toolchain builds for arm-eabiChristophe Lyon
Hardcode the cpu/float-abit/mode to speed up toolchain builds: the alternative of building rmprofile multilibs takes a very long time. Setting the values here has the drawback that we have to build one toolchain per board type. The only stm32 board we have currently has a cortex-m4 with FPU. Change-Id: Ia2626405f9a6ed42915e7b575465d82b6c44871c
2021-03-03tcwg_bmk-build.sh: Use rmprofile multilibs when targeting arm_eabiChristophe Lyon
When targeting arm_eabi, we compile for Cortex-M and need the right versions of startup code and libraries. Change-Id: I7bf15b342c7b55117c66266004e0396f2a3dccfc
2021-03-02tcwg_bmk-build.sh: Use rsync to access the toolchain when target is stm32Christophe Lyon
When running benchmarks on stm32, it's easier to rsync the toolchain towards tcwg-bmk-stm32-01 (slave to which the stm32 board is attached) rather than involving the tricky process of rsync-ing the benchmark sources towards the builder and remote compile, like we do for Spec. Change-Id: I2c87bcc7fbc80ba9d75d5c88a33a15f8e928b5bc
2021-02-12tcwg_bmk-build.sh: Prefix results_id with stm32_ for arm_eabiChristophe Lyon
The tcwg-benchmark-bare.sh script expects results_id to start with the "stm32" string, so use it as a prefix. Keep the existing STM32XXXXX as it will be useful when we have different board types. Change-Id: Iae77f6789cb85115857a906ee76168b40321228e
2021-02-01tcwg_bmk-build.sh: Fix unbound variable errorMaxim Kuvyrkov
Change-Id: Ie7402bd8ff0c5188a097171a90b853af9f0f6255
2021-01-29tcwg_bmk-build.sh: Fix failure when bisecting glibc across releasesMaxim Kuvyrkov
Change-Id: Icababf1d9c659dc7c1ce443abf30c101c5067b63
2021-01-14tcwg_bmk-build.sh: Adjust toolchain parameter passed to tcwg-benchmark jobMaxim Kuvyrkov
... so that tcwg-benchmark-stm32 doesn't fail on wrong parameter. Change-Id: I469c7aa779705b17b13e290a481382f96db5a460
2021-01-14tcwg_bmk-build.sh: Don't use --with-mode=thumb for arm-eabiMaxim Kuvyrkov
... because it conflicts with --with-multilib-list=aprofile. Setting --with-mode=<foo> is used to compile glibc for mode <foo>, and for bare-metal toolchain setting -mthumb in benchmarking cflags is sufficient. Change-Id: Iddb012650e64700565e683a2f5d915fa6ad2e3f9
2021-01-14tcwg_bmk-build.sh: Rename ${rr[target]} from "arm-eabi" to "arm_eabi"Maxim Kuvyrkov
... so that we can correctly extract it from ${rr[ci_config]}, which splits fields on "-". At the moment we detect target as "arm" and pass "--target arm-linux-gnueabihf" to abe.sh. Change-Id: Ie2fd34e6d2b627181e6134ea55f31b0c5a0763b0
2021-01-04tcwg_bmk-build.sh: Update sysroot path after sysroot changes in ABEMaxim Kuvyrkov
Change-Id: Iddb06af2a77fc3928dc5bc4950d703d34ea314cd
2021-01-04Update expected x86_64 host tuple for ABE buildsMaxim Kuvyrkov
... from x86_64-unknown-linux-gnu to x86_64-pc-linux-gnu after updating config.guess in ABE. Change-Id: I81333036fb8d183ad81e3fa5b5836645e8b6220f
2020-12-15Revert "tcwg_{bmk,gnu,kernel}-build.sh: Update to not override passed ↵Maxim Kuvyrkov
rr[components]" This reverts commit 5a80a7248f90c25f983f7412d2432f8f1ad26368. Reason for revert: This was a bad idea. Change-Id: Ib9deb1247ded5714cccc88baf5535892e78f5baa
2020-12-15tcwg_{bmk,gnu,kernel}-build.sh: Update to not override passed rr[components]Maxim Kuvyrkov
Change-Id: I1b2dbfb294e72bab2a14cbb91a4dec03b58a37fb
2020-10-27tcwg_bmk-build.sh: Add support for gnu_eabi toolchain typeChristophe Lyon
gnu_eabi toolchains need to build newlib instead of glibc and linux. In addition to adding support for "gnu_eabi" toolchain type, we also take care of building the right set of components. Change-Id: I1fdba8829d8b8609495e1a9e612701cc7d36c36b
2020-09-11jenkins-helpers.sh: Split $EXPECTED_EXIT into $INTERNAL_FAILMaxim Kuvyrkov
... and $EXTERNAL_FAIL. This allows us to distinguish between regressions that we should bisect and infra problems that we should skip bisects for. Change-Id: I8f3b82c86d95e27351b01a2b3d2467b8e15b309e
2020-09-03tcwg_bmk-build.sh, jenkins-helpers.sh: Add support for arm-eabi targetChristophe Lyon
This is useful to run benchmarks on stm32 boards (Cortex-M, bare-metal). Change-Id: I115b982effea9d0ddebe990d74f8bb6c72806778
2020-07-08tcwg_bmk-build.sh: Skip builds that couldn't finish benchmarkingMaxim Kuvyrkov
Tcwg-benchmark-* jobs fail only due to infrastructure reasons. Benchmark build and run failures are assigned special 99..99 and 88..88 metrics, but the jenkins build succeed. Change-Id: I71639b6d4e1fd2a139390884c0d1eae0bb89a95d
2020-07-06tcwg_bmk-build.sh: Gracefully handle benchmarking board going offlineMaxim Kuvyrkov
... which is represented by tcwg-benchmark-* build returning UNSTABLE result. Change-Id: Ifa8be8e601ef93412a3a62d9bed4a078fe38fa49
2020-07-02tcwg_bmk-build.sh: Workaround missing benchmarking resultsMaxim Kuvyrkov
... which are referenced in base-artifacts repo. This problem occurred when bkp-01.tcwglab, where we store benchmarking results, ran out of disk space, so tcwg-benchmark-* jobs could not upload their results. This didn't stop tcwg_bmk-build.sh to treat this failure as regressions in various configurations, which were then bisected down to the first commit. At that point baselines for individual configurations were reset to the new failing state, and references to the missing results were committed to base-artifacts repo. We will fix this problem in several ways: 1. We have added tcwg-cleanup-stale-results.sh to parse base-artifacts repo and delete results with no references from that repo (thus preventing bkp-01.tcwglab from running out of disk space). 2. We are adding this workaround to purge broken entries from base-artifacts repo. 3. We will commit .csv summary of results into base-artifacts.git so that we won't need to rely on on-the-side storage of perf.data files. Change-Id: I922df040292c1686f4461052d21bdeb98bea0b03
2020-05-29tcwg_bmk-build.sh: Add compatibility with older baseline resultsMaxim Kuvyrkov
Change-Id: I79dc695cc2d4955516fc2ab55dfed031501d0a6b
2020-05-28tcwg_bmk-build.sh: Rework details of arm/thumb mode selectionMaxim Kuvyrkov
... while keeping general logic the same. This approach sanity checks that 2 configurations of cflags in *-vs-* configurations are using the same mode. To use different arm/thumb modes in *-vs-* configurations we would need to build GCC and Glibc 2 times, which we could do, if required. Change-Id: I3d9e6152b8b01f561e7e60557ccc680e99728103
2020-05-27tcwg_bmk-build.sh: Adjust build scoresMaxim Kuvyrkov
Don't bisect benchmark build/run failures in *-vs-* configurations. These failures will also appear in non-vs configurations, and will be bisected more efficiently. Change-Id: Iad4a70a9b13e5509ff95e8dabde942a55f5c10b9
2020-05-27tcwg_bmk-build.sh: Fix loop waiting for benchmarking to finishMaxim Kuvyrkov
Remote_exec can exit before tcwg-benchmark-$hw job finishes, and that's the reason why we loop until we see "Finished: " on the job's console log. Change-Id: Ib579a48e6a1e1f20d9f6d748efcf693df27d5c20
2020-05-22twcg_bmk-build.sh: Correct scope of build retDavid Spickett
The function runs in subshell so we were trying to return build_ret after that subshell had finished. Change-Id: I7baf15077451ca563dce09facf0131ceb38c8bc1
2020-05-20tcwg_bmk-build.sh: Always write benchmark results_id fileDavid Spickett
Before it was only written when the job failed. (though the build number was in the log if you looked hard enough) This can be used to go from run_step failing, to the log of the actual benchmark. Change-Id: I8189518f948f9ac76d090ff635209a5e69762a98
2020-05-18tcwg_bmk-build/tcwg_gnu-build: Add traceback printingDavid Spickett
The line number you get from the previous message is confusing. For example: ERROR at run_step:524 Means that something in the function run_step, which was called from line 524, called exit. Instead, add a Python style traceback that shows the call stack with file, line and the content of that line. (though unfortunatley we can't get the line no. of the exit itself) Example traceback: ERROR Traceback (most recent call last): File: /tmp/foo.sh, line 37 foo # hey! File: /tmp/foo.sh, line 32 bar # hi there File: /tmp/foo.sh, line 1 (trap handler, 'exit' call line is unknown) Change-Id: Id6a2734e2b5ef74d05cd51cdbde9a95e5d5a4b44
2020-05-14tcwg_bmk/round-robin: Add assert_with_msg functionDavid Spickett
This acts like assert but prints a message if the assert fails. This will be visible in the Jenkins logs and give us a unique string to look for in the abscence of traceback for shell scripts. Two instances of returning false in check_regression have had echos added to them, instead of asserts. This is because they are not hard failures, and may be handled by the caller. (run_step) I have only converted the tcwg-bmk relevant scripts to assert_with_msg so far, as a proof of concept. Change-Id: I3b1badf09e4856c80b973ca8b23088626ed66b0a
2020-05-08Revert "tcwg_bmk-build.sh: Use parallel8x4 and parallel8x1 bmk profiles for ↵Maxim Kuvyrkov
TX1s" This reverts commit 44804d02f22685b8bc015f1d9ef854b970d5d286. Reason for revert: tcwg-tx1-03 is back online. Change-Id: I701437e6d1c91fde5f8305c5374600f6c8ad2534
2020-05-06tcwg_bmk-build.sh: Use parallel8x4 and parallel8x1 bmk profiles for TX1sMaxim Kuvyrkov
... to workaround https://projects.linaro.org/browse/LSS-1293 . Change-Id: I504ddcb1186d2d2afe2b60b12be211fb23729fc7
2020-05-05jenkins-helpers.sh: Add pwd check functionDavid Spickett
Used in other scripts to check that pwd isnt / before deleting things. Change-Id: Id3e284a486f300c97e92f730e84183340c999dd9
2020-04-30Shellcheck fixes round 5David Spickett
Quote command line args in tcwg_bmk-build. Remove the manual list of shellcheck excludes now that we've settled on a newer version that supports --severity. Change-Id: I102ea9bad1ac6245885c410edf3410b13b0727c6
2020-04-30Shellcheck fixes round 4David Spickett
Fixes for: https://github.com/koalaman/shellcheck/wiki/SC2207 Are using the bash 4.x 'mapfile' syntax where the input to the array is seperated by newlines. If it's separated by spaces we use 'read' instead. Support for selecting libc in MakeRelease.job was removed in 3fa23da28ad73d0fe4b512c4aab269b5280a6650. So remove references to "libc" which were expanding to nothing anyway. In tcwg-llvm-common.sh PROJECTS->BRANCHED_PROJECTS is not strictly needed. However shellcheck can't tell that the local PROJECTS in this file is different from the PROJECTS in tcwg-llvm-build.sh. Change-Id: Ie55968e3bf185d0bd5795fe48014636cda1009bc
2020-04-28Shellcheck fixes round 3David Spickett
Now using the version that's present in the bionic images. Meaning we have more warnings but can also set the minimum level to ignore a lot of them. Fix the check for whether shellcheck supports --severity. (we have pipefail on, but we expect shellcheck to fail in this case) Change-Id: I296b8554591b7d327c75393e3924184abc6512b7
2020-04-22tcwg_bmk-build.sh: Various shellcheck fixesDavid Spickett
Change-Id: I8b2f10c368650fcd74ae261b0da406e3bc259bb3
2020-04-20Shellcheck warning fixesDavid Spickett
(all error/style level warnings) SC1035: missing whitespace SC2068: double quote array expansions SC2145: argument mixes string and array SC1066: Don't use $ on the left side of assignments SC2126: Use grep -c instead of grep|wc Change-Id: Ia53070d3e2c9aea9f626fdb7b2b5bfc9e1f8e682
2020-04-01tcwg_bmk-build.sh: Fix spurious failuresMaxim Kuvyrkov
... due to stray results.regressions file. This file can be generated by successful no_build_regression_p() invocation and contain harmless errors from toolchain build process. Change-Id: I237e09cd612c0b96d27d765de9bcd28f5ec51573
2020-03-22tcwg_bmk-build.sh: Simplify no_regression_to_base_pMaxim Kuvyrkov
Use results.regressions generated by compare_results() to report regression or success. Change-Id: Ieda1532411a49b05a5831d9802a476e92bab152b
2020-03-22round-robin.sh: Merge info from "regressions.txt" to "results"Maxim Kuvyrkov
Change-Id: Iba2ba392d0ed6656548ead3245d4d7906bbdfbc0
2020-03-22round-robin.sh: Avoid changing top-level artifacts in no_regression_p predicateMaxim Kuvyrkov
The no_regression_p() predicate function is called from two steps: check_regression and update_baseline. While we want no_regression_p to output regression info during check_regression step, we don't want that during update_baseline (during update_baseline no_regression_p is called to compare current results against pre-baseline ones). To fix the above we generate regression information in artifacts directory of the current step, and copy it to the top-level only during check_regression step. Change-Id: Ib03d82f7afe60c406c7942f2fa086855df371171
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-28 16:14:09 +0000