summaryrefslogtreecommitdiff
path: root/tcwg_bmk-build.sh
AgeCommit message (Collapse)Author
2023-06-29round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store bare bmk ↵Laurent Alfonsi
results in artifacts round-robin.sh initialize git-annex as pointing on bkp-01:/home/tcwg-benchmark/base-artifacts, and consider the "annex/" directory as annex datas to be saved with git-annex. Right now, only tcwg_bmk is using it. Taking bare results to the artifacts & baseline simplifies a lot the compare_results routine. tcwg-benchmark-results.sh no more takes results_id string as inputs, but directly takes the directories paths of each bmk results. It takes an additional parameter hw_tag, to avoid precedent results_id parsing. This means this script currently can only treat inputs belonging to the same hw_tag. bmk results are stored in : artifacts/annex/bmk-data We are still keeping results_id file because it is useful to find the benchmark job id that were used to run the benchmarks. Change-Id: I78ef05585be8f1587257f9d8950023b70bff18bc
2023-06-23Revert "round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store ↵Laurent Alfonsi
bare bmk results in artifacts" This reverts commit 4b36216d70e60de56e321d0b403020d27c2d44fe. Reason for revert: pb with ci job deployment make it bancal Change-Id: I89f94a89a0267780c7e119d03a0ab17908625310
2023-06-23Revert "tcwg_bmk-build.sh: Simplify compare_options handling"Laurent Alfonsi
This reverts commit 45c0646915879740a9f834b13345ae685595a703. Reason for revert: pb with ci job deployment make it bancal Change-Id: I84e2b3c2bad1894535d3a4db8c427e6fb42ce524
2023-06-22tcwg_bmk-build.sh: Simplify compare_options handlingLaurent Alfonsi
Change-Id: I4baf0e2f7f900383af51a70d5954b27ab1bdc969
2023-06-22round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store bare bmk ↵Laurent Alfonsi
results in artifacts round-robin.sh initialize git-annex as pointing on bkp-01:/home/tcwg-benchmark/base-artifacts, and consider the "annex/" directory as annex datas to be saved with git-annex. Right now, only tcwg_bmk is using it. Taking bare results to the artifacts & baseline simplifies a lot the compare_results routine. tcwg-benchmark-results.sh no more takes results_id string as inputs, but directly takes the directories paths of each bmk results. It takes an additional parameter hw_tag, to avoid precedent results_id parsing. This means this script currently can only treat inputs belonging to the same hw_tag. bmk results are stored in : artifacts/annex/bmk-data We are still keeping results_id file because it is useful to find the benchmark job id that were used to run the benchmarks. Change-Id: Ibac5ed3508962f1a32ad3361e429f20c66f7eae3
2023-06-21tcwg_bmk-build.sh,round-robin.sh: Fix metric in tcwg_bmk jobsLaurent Alfonsi
Change-Id: I48c7b55797389fd73674924039b3aefdcd5a0072
2023-06-16tcwg_bmk-build.sh: give metric watched parameter to output-bmk-results.pyLaurent Alfonsi
- output-bmk-results.py will raise a regression only in the watched metric. - Also ends with an error if output-bmk-results fails, instead of creating a error-code-$? file Change-Id: I90396862a4c5f9fe0033177cd88cd9ffe5e1ed3c
2023-06-14round-robin-notify.sh,tcwg_bmk-build.sh: Generate regression details at ↵Laurent Alfonsi
notify stage Change-Id: I756d09f190ded50c10fc41ed88f70f9c2925acc4
2023-06-14tcwg_bmk-build.sh: Use output-bmk-results output for bmkLaurent Alfonsi
Details changes: 1) output-bmk-results now directly outputs in $run_step_artifacts {exe,symbol}.{regression,improvement} and extra-bisect-params files are all generated by output-bmk-results.py 2) $bmk.symbols-regression/improvement files doesn't exist anymore all details are now in symbol.regression/improvement 3) $bmk.symbols-skipped doesn't exist anymore All skipped exe/symbols are now in any.skipped. 4) results-vs-prev/compare-results.csv is not generated anymore Seemed to be useless. Change-Id: I4ef03ae52d97e42b14def96f1cc967d5ebf29123
2023-06-08round-robin-baseline.sh: New script to host update_baseline() codeMaxim Kuvyrkov
Move code for update_baseline() from round-robin.sh to its own script -- round-robin-baseline.sh. This allows us to move update_baseline() step after notification step, which simplifies notification logic. For the main workflow round-robin-baseline.sh is now called from tcwg/round-robin.yaml.inc for successful builds. The main reason for this reorganization is to simplify re-processing of results. Change-Id: I038bfc1ac68b1d6bb6dfd6f36dfb559d8e481583
2023-06-06round-robin-notify.sh,tcwg_bmk-build.sh: Remove computation of results-vs-firstLaurent Alfonsi
squad-dashboard will temporarily miss these data. The plan is to have a check_regression called twice : - while RR : minimal computations and files outputted. - while RR-notify : This one could have more computations, and could decide to compute results-vs-first if necessary for squad. Change-Id: I2722d93e1a4472515eddcba35df5ba05acecbf8a
2023-05-29tcwg_bmk-build.sh: diff-bmk-results.py calls all outputs in a single fileLaurent Alfonsi
Change-Id: Ie1f5612bc832253ba3251ce950280f274ae35e48
2023-05-22tcwg_bmk-build.sh: Add symlink to ease navigation in jenkins artifactsLaurent Alfonsi
Change-Id: Icdc9543fabdb80678d77d7822b68859f8d477ac7
2023-05-16round-robin-notify.sh: remove artifacts mail and dashboard dirLaurent Alfonsi
- mail/* infos are moved to notify/ directory - all dashboard/ directory now created and used in notify/dashboard/ Change-Id: I83e25cd2b23ebc0e82459bfe8916a3ff8093764e
2023-05-12tcwg_*-build.sh: remove rr[start_date], as it is now uselessLaurent Alfonsi
Change-Id: I4a99ca2392270d3a7fe64bb2e18d867fa452e241
2023-04-23tcwg_bmk-build.sh: Fix results.regressions for bmk jobsLaurent Alfonsi
Now starts with #. It is reported as it is in results file in round-robin.shi. Only the score can start without #. Change-Id: Ic77f94213c1be9055fdefaabd1258219306796e9
2023-04-18tcwg_bmk-build.sh: more robust for empty fileLaurent Alfonsi
Handle case where file contains only 'Failed for column' lines, and cases where files is empty Change-Id: Iee75614f28cd01a81d8ee7281e58541b9491d24e
2023-04-17tcwg_bmk-build.sh: handle samples-variability even if emtpy compare-results fileLaurent Alfonsi
Change-Id: I3a3d705d548278199bec3c5766233fc4154ecb69
2023-04-14tcwg_{aosp,bmk}-build.sh: Fix usage of readarrayMaxim Kuvyrkov
Use "-t" to strip newline from the array elements. Change-Id: I74e44741280ef205ca7d52924c93ab5671ef37a3
2023-04-14tcwg_{aosp,bmk}-build.sh: Use get_git_history() to fetch "first" resultsMaxim Kuvyrkov
Change-Id: I479f4e897da59084151a693f446613368b3182fe
2023-04-14jenkins-helpers.sh: New helper get_git_history()Maxim Kuvyrkov
... to fetch different revisions of a file from git history. Use it to fetch compare-results-internal.csv in tcwg_bmk-build.sh and flaky.xfail in round-robin.sh:build_abe(). Change-Id: Ib9fa75a76d4694c70f100ae3d6bf8ea12c2123dd
2023-04-14tcwg_bmk-build.sh: exe.regression first created in xx-check_regressionLaurent Alfonsi
Change-Id: I8cbd524cfe0a466660c09123248de5272f394c2a
2023-04-13tcwg_bmk-build.sh: Fill results.regressions with ↵Laurent Alfonsi
*.{regression,symbols-regression} .. otherwise, no_regression_p will never complain about a regression Change-Id: I1130e825ee861f5e75f75a28174ff6a32058c638
2023-04-12tcwg_bmk-config.sh: Remove workaround for noisy SPEC2k6 benchmarksMaxim Kuvyrkov
Change-Id: Ic0890155ba6783b88ce9bdc965a15d6ea0dd802b
2023-04-11round-robin.sh,round-robin-notify.sh,tcwg_bmk-build.sh: rework on notify stageLaurent Alfonsi
round-robin-notify.sh interface rework: - round-robin-notify.sh script parameters changed : now simply pass top_artifacts, components list, and baseline_revision - In addition to this, inputs are taken from : . mail/check-regression-status.txt . mail/{exe,symbol}.{regression,improvement} if any. bmk case . mail/results.regressions. (gnu case) round-robin-notify.sh structure rework: - setup_environment : setup base-artifacts env, and choose the steps to perform - compute what changed in the sources looking at artifacts and base-artifacts directory - generate 1st part of the mail : the regression (project specific) - generate 2nd part of the mail : the reproduction instructions (generic) - generate mail subject and recipients (generic) - generate jira card, and post jira card - update and post interesting_commits Change-Id: I4bd368ce6536bf18104684cd10f2b06f49e0cddd
2023-04-07Revert "round-robin-notify.sh: All mail generation in later stage"Laurent Alfonsi
This reverts commit 53661d3d960f54b1cddd23d36283645f15a71793. Reason for revert: Reverting all notification rework. Need deeper analysis. Change-Id: I3df5fb19cbb522a2b0a13cd8d80adfcff9807918
2023-04-07Revert "round-robin.sh,round-robin-notify.sh,tcwg_bmk-build.sh: rework on ↵Laurent Alfonsi
notify stage" This reverts commit 9cc425dd40deffea02e96f25fed3384c1667ab7a. Reason for revert: Reverting all notification rework. Need deeper analysis. Change-Id: I96ed60fa9e8ac8ba9931fdfb8c7c98144947496d
2023-04-07Revert "tcwg_bmk-config.sh,tcwg_bmk-build.sh: fix rr not existing"Laurent Alfonsi
This reverts commit 871f9342c26195ecd19db3fa0f904e1dc28acd42. Reason for revert: Reverting all notification rework. Need deeper analysis. Change-Id: I7d2a1fd58da589a9b22bb446d19838638f7050d5
2023-04-07tcwg_bmk-build.sh: create results-vs-xxx dirLaurent Alfonsi
Change-Id: Ide9a8f424bdb46a8857aeaca7cf463bb0bef43a4
2023-04-06tcwg_bmk-config.sh,tcwg_bmk-build.sh: fix rr not existingLaurent Alfonsi
Change-Id: I338c41418d5e64c5ae50186161e1098aed941a95
2023-04-06round-robin.sh,round-robin-notify.sh,tcwg_bmk-build.sh: rework on notify stageLaurent Alfonsi
round-robin-notify.sh interface rework: - round-robin-notify.sh script parameters changed : now simply pass top_artifacts, components list, and baseline_revision - In addition to this, inputs are taken from : . mail/check-regression-status.txt . mail/{exe,symbol}.{regression,improvement} if any. bmk case . mail/results.regressions. (gnu case) round-robin-notify.sh structure rework: - setup_environment : setup base-artifacts env, and choose the steps to perform - compute what changed in the sources looking at artifacts and base-artifacts directory - generate 1st part of the mail : the regression (project specific) - generate 2nd part of the mail : the reproduction instructions (generic) - generate mail subject and recipients (generic) - generate jira card, and post jira card - update and post interesting_commits Change-Id: I6c7e1212fac73406cfd7649056e93d83e9e468e5
2023-04-06round-robin-notify.sh: All mail generation in later stageLaurent Alfonsi
Change-Id: I038afa9440a6266cea3e1656d62cfa14ceddc5e5
2023-04-06tcwg-benchmark-results.sh,tcwg_bmk-build.sh,tcwg_aosp-build.sh: Change ↵Laurent Alfonsi
directory for csv results files ... from XX-check_regression directory to results-vs-prev directory (or results-vs-first) Change-Id: I94c20d23391b84a62a6265c12cced8891c9c1d6e
2023-04-06tcwg_bmk-build.sh: Install armhf perf workaround if one is availableMaxim Kuvyrkov
In some cases perf report crashes when run from armhf container on ARMv8 machine. Install a workaround while we are investigating the cause. The workaround is present only in armhf docker images, so will be applied only in benchmarking of LLVM armhf configurations, which use native LLVM compilers built in armhf containers on tcwg-jade-03. Change-Id: I7064a83ccb3e6c0074e286f55e9d94533cdea123
2023-04-04tcwg_bmk-build.sh: Fix assert msgLaurent Alfonsi
Change-Id: I51b2a9ebf92f09146decf777f4689e130af4457a
2023-03-25tcwg_bmk-build.sh: Fix samples-variability generationLaurent Alfonsi
Change-Id: I0faf233ea422a972b7710d289485542e843b2f5d
2023-03-25tcwg_bmk-build.sh: Fix samples-variability generationLaurent Alfonsi
.. in case of not existing compare-results csv file Change-Id: I733698df4997eac73ff597e55b37a95e98865f20
2023-03-25tcwg_bmk-build.sh: TypoLaurent Alfonsi
Change-Id: I1dcc670190216334afdf664a84eb94330f3e84b1
2023-03-24round-robin.sh: Unify implementations of breakup_changed_componentsMaxim Kuvyrkov
Change-Id: I046212911cdd4d50ed4590e5d1cfdce19e577422
2023-03-24tcwg_bmk-build.sh: Compute and use automatic threshold for each exe/functionLaurent Alfonsi
Change-Id: I010889cf2bcb82f8aa75908c4fb2e223be572aaf
2023-03-23round-robin.sh,round-robin-notify.sh,tcwg_bmk-build.sh: Generate jira file ↵Laurent Alfonsi
at notify stage For now : . jira text is built using mail content emited during check-regression stage. . print_jira_card() is still duplicated in round-robin.sh, (which is useful for mail content) Change-Id: I3262ba38b47ad94af47d8c2734bd7b55049ac4eb
2023-03-23tcwg_bmk-build.sh: move call to diff-bmk-results after symbol.regression ↵Laurent Alfonsi
file generation Change-Id: Ia0b190750c83ba1c48b3986d23f356fc44c1a58f
2023-03-14tcwg_bmk-build.sh: QC board not reboot.Laurent Alfonsi
.. Need to install & configure 96boardsctl to reboot Change-Id: I855a1f3f66bb8d448cbd3fa6d27775af873b28f3
2023-03-14tcwg_bmk-build.sh: Add project handling for QC boardLaurent Alfonsi
Change-Id: I0949daeeab3487ffb2a2c2820fb7d0e6bdc46f6f
2023-03-09tcwg_bmk-build.sh: Use bmk-specific-threshold.csv for output-bmk-results.pyLaurent Alfonsi
Change-Id: I922da1610fff2e314a3b0c91623ba0634834a31f
2023-03-09tcwg_bmk-build.sh: check if previous and new bmk results are identicalLaurent Alfonsi
Change-Id: Ia8b6bb07e1c57e68ab44098936216919750d902e
2023-03-08Do not "<output> | grep -q", use "<output> | grep -q ... >/dev/null" insteadMaxim Kuvyrkov
When 'grep' in invoked with -q option, it exits with the first match, and closes the input pipe. This will break the pipe and scripts running with "-o pipefail" will get unexpected results. Replace "<output> | grep -q", with "<output> | grep -q ... >/dev/null" throught our scripts. Leave "grep -q ... <input_file>" intact, since pipes are not involved. Change-Id: I9438919276c0d0e235600384a979b511e1a4edff
2023-03-07tcwg_bmk-build.sh: Handle new code-size and code-vect configurationsMaxim Kuvyrkov
Change-Id: Ic43ccf236d933de732c4cf63972bb0224161fbd7
2023-03-07tcwg_bmk-build.sh: Run fortran benchmarks on *--llvm-aarch64 configsMaxim Kuvyrkov
Flang is in good enough shape to run fortran benchmarks for *--llvm-aarch64 configurations, which we already do for fujitsu configurations. Flang currently does not support 32-bit ARM, so those still use *_nofortran benchmarks. Change-Id: I89e37da6ddf565bf24a3ed4ffa69cc31791ce3d9
2023-03-06tcwg_bmk-build.sh: Fix tcwg_bmk_benchs. Run only one benchmark if requested.Laurent Alfonsi
+ minor indentation changes Change-Id: I360b5e077cc92d515de2d9a0183f8a8f229be40a
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 09:24:18 +0000