summaryrefslogtreecommitdiff
path: root/tcwg_bmk-build.sh
AgeCommit message (Collapse)Author
14 daystcwg_bmk-build.sh: increase minor versionLaurent Alfonsi
Change-Id: I6d404919d281423e636fa92d47aa3614d8d252c6
14 daystcwg_bmk-build.sh: Enable new noise computation method (2-peaks-linear)Laurent Alfonsi
Change-Id: I8792ea1eef38e52c5159e02610198d6b6f24d13c
2024-02-29tcwg_bmk-build.sh: Fix typoLaurent Alfonsi
Change-Id: Id92f1e97d7e5032cdb5b91874119f11459f321d2
2024-02-28tcwg_bmk-build.sh,round-robin-notify.sh: Adjust parameters for ↵Laurent Alfonsi
output-bmk-results Change-Id: I9a2cf890425cba9e5b022462911c2ae0f9ac188f
2024-02-28tcwg_bmk-build.sh: Adapt for new variability computationLaurent Alfonsi
Change-Id: I17779d0ab850dcd4fb6041ac3dc3536671f09951
2024-02-26tcwg_bmk-build.sh: upgrade minor version for some bmk projectsLaurent Alfonsi
Change-Id: I5c3438ef570c9727e88ee4329de6ad3cfe2d34f6
2024-02-20tcwg_bmk-build.sh: Include current compare-results.csv to compute bmk variabiltyLaurent Alfonsi
Change-Id: If844d1ae1da36a0f85297a57c360b6e67e4a67c9
2023-12-21round-robin.sh,tcwg_bmk-build.sh: Remove temporary code, used to change the ↵Laurent Alfonsi
base-artifacts branch names Change-Id: I98a37ee40d482b2e546c76f64e9a66da2ac3e535
2023-12-05tcwg_bmk-build.sh: Increase version for fujitsu jobsLaurent Alfonsi
Change-Id: I54417bbd5c9f125a76aa61c23c682d970ffe0e43
2023-11-22tcwg_bmk-build.sh: upgrade version for some projectsLaurent Alfonsi
purpose of this upgrade is to update lnt results. Change-Id: I5474f960dbb087ba9adac7279a5b90cb3dc83534
2023-11-21tcwg_bmk-build.sh: Upgrade major revision for some of the bmk projectsLaurent Alfonsi
We upgrade the version because we need to regenerate the lnt_results.json The base-artifacts rewrite failed because it now newly detects some [justified] regression, and stops rewriting. To continue this rewrite process further, a upgrade of the major is necessary. Change-Id: I6843d2c700de11297bd407e308984afd18c01635
2023-11-20tcwg_bmk-build.sh: Increase artifacts format versionLaurent Alfonsi
.. for a subset of the projects (tcwg_bmk-code_speed-spec2k6*). The purpose of this is to update the lnt_reports.json, and push to lnt server Change-Id: I72ab235d7bf4be09ad940dceebcf7a8bda28295a
2023-10-15round-robin.sh, tcwg_bmk-build.sh: run UNSTABLE for regressions to a ↵Laurent Alfonsi
negative score. When encountering a score-based regression, leading to a negative score, we consider it as an EXTERNAL_FAIL (infrastructure problem) This will mark the job as unstable as long as the score is regressing to a negative score. The effect is no reduction to components in that cases. Then no attempt to update other components. Change-Id: I34ff5fb0615bdc0769d19e0c0353084715ded178
2023-09-29tcwg_bmk-build.sh: Rewrite all benchmarking historyMaxim Kuvyrkov
Change-Id: I9834fc0b53dcbc2333fd4b78a20b498178aaa42b
2023-09-20Rewrite history for VECT and SVE configs.Prathamesh Kulkarni
Change-Id: Icfefa30e2916d7c33ddd9de82066dac7d3eac42a
2023-09-02tcwg_bmk-build.sh: Test history rewrite on SPEC CPU2006 configsMaxim Kuvyrkov
Change-Id: Id411349d8a46e6c705681d5e625cc9beb5f1ee97
2023-09-02round-robin-notify.sh (bmk_generate_extra_details): FixMaxim Kuvyrkov
Pass correct metric_id to output-bmk-results.py ("sample", not "time") and do not ignore failure from the script. Also add a few FIXMEs to cleanup code. Change-Id: I8b3a1c42a6be3dca30dab5adc8d6a8484535e107
2023-08-31tcwg_bmk-build.sh: rename samples-variabilty.py scriptLaurent Alfonsi
Change-Id: Id3f00d18bcc23cfd7ebe991b0047f1cfa52b47b6
2023-08-31tcwg_bmk-build.sh: use a smaller curl request to wait for benchmark jobLaurent Alfonsi
Change-Id: Ifff6f65d5c2822852c3f4719683559aed0f2bc01
2023-08-31tcwg-bmk-build.sh (no_regression_p): Update assertMaxim Kuvyrkov
Change-Id: I009b39d6489f3e3c0b48b554d50750425b855f17
2023-08-30tcwg_bmk-build.sh,round-robin-notify.sh: Regenerate details at ↵Laurent Alfonsi
round-robin-notify stage when necessary 1) Reorganize the regeneration of the details - regenerating details are now done in xxx_generate_extra_details() - generate_extra_details() is called as soon as we know we start full mode. 2) Add generation of the details in case of bmk jobs. by directly calling output-bmk-results.py in verbose mode Change-Id: I35f226d28c1e5b9bc854b04ff3cee8137654974c
2023-08-30tcwg_bmk-build.sh: Merge no_regression_p/compare_results routinesLaurent Alfonsi
Change-Id: I061cfd24226478de89c5a8884cc2e024cb642f42
2023-08-30tcwg_bmk-build.sh: Fix init mode, failing due to a wrong assertionLaurent Alfonsi
Change-Id: Ib5e83b0f4e265c2fcd2bde1773c7364648729422
2023-08-28tcwg_bmk-build.sh (benchmark): Improve wait cycleMaxim Kuvyrkov
Use jenkins json API to detect completion of tcwg-benchmark job (instead of parsing console output). In rare cases a build can fail without printing out "Finished: FAILURE" to the console. Change-Id: Ifdd746a71c6452ba8482a0edba957a1da3b2c30c
2023-08-25jenkins-helper.sh (git_annex_upload): Make more robustMaxim Kuvyrkov
Change-Id: Iaf5a463f17f6e88d0cd2f83c54a722c68264cde0
2023-08-25tcwg_bmk-build.sh (benchmark): Reboot qc and fx machines before benchmarkingMaxim Kuvyrkov
... which fixes failure to set CPU frequency on tcwg-bmk-qc-* boards. Change-Id: Ib72407a68ba85e336a54265a3a80e2c176aa6808
2023-08-25tcwg_bmk-build.sh: Rework handling of base-artifacts/annex/bmk-dataMaxim Kuvyrkov
To avoid committing unwanted files into git (e.g., raw benchmarking data) we implement "annex" support. Files are base-artifacts/annex can be either symlinks to directories or regular files containing an rsync-able url. Here we convert directory symlinks into tarballs, upload to bkp-01 and replace symlinks with files pointing to their uploaded location. In round-robin.sh:reset_artifacts() we do the opposite: download and extract tarball into a temporary directory, and replace the file with a symlink to that directory. Change-Id: I78062a5147c944f18d6cd6eb5129f0d9fb74098c
2023-08-21tcwg_bmk-build.sh: Add --weight option to samples-variability.pyLaurent Alfonsi
Change-Id: I930622fbd2c77e65e11663583bd17135a9bfa6d4
2023-08-16tcwg_bmk-build.sh: Use all history to compute sample variabilityLaurent Alfonsi
Change-Id: I9c7bd0f100f06a520a6111a25dd41564a47f5a67
2023-08-01tcwg_bmk-build.sh: Fix bmk bisects.Laurent Alfonsi
First baseline run (successful) needs to be used as a reference. but the results, are not pushed to bkp-01. In that case, in tested runs, we want to use reference bmk results directly from base-artifacts directory Change-Id: If9500bfd9a040ee36289ca31cfa98c37e9cef7d0
2023-07-18tcwg_bmk-build.sh: redirect output-bmk-results output to a log fileLaurent Alfonsi
Change-Id: Ic9ebfea312bb05c6e78fa061ecfe617db66e4bed
2023-07-17tcwg_bmk-build.sh: Fix perf record directoryLaurent Alfonsi
The rsync was not transfering links, therefore internal perf record directory was uncomplete. (i.e, .build-id sym links). This rsync should now also copies these relative symlinks. These links pointing on the elf file are important for having funtion names in the generated perf report. Change-Id: I292947fa037f8af8b32b2c695923fed4237d6b35
2023-07-09tcwg-benchmark-results.sh: new results taken from artifactsLaurent Alfonsi
Change-Id: Ib825c79d67ed76b5e7681464ebfb069eb99201f9
2023-07-08round-robin-baseline.sh,tcwg_bmk-build.sh, tcwg-benchmark-results.sh: Fix ↵Laurent Alfonsi
bmk-results getting&storing tcwg-benchmark-results: -input/-output results are content of the results_id files round-robin-baseline: fix .gitignore Change-Id: Ia695f951fc725c6f11e30c874bb0b51b50300828
2023-07-07tcwg_bmk-build.sh: Remove wrong assertLaurent Alfonsi
Change-Id: Ia72e5a31b311bed9e44b7b317f8cd4d41f527fe3
2023-07-07tcwg_bmk-build.sh: Revert storing bmk-data in annexLaurent Alfonsi
.. now store in /home/tcwg-benchmark/results-xxx only if successful (in round-robin-baseline.sh) Change-Id: Ic700c3c8dbfefff84cb386259f54cbd70147c3f7
2023-07-05tcwg_bmk-build.sh: Add clear assert if the bmk results cannot be retrievedLaurent Alfonsi
.. and specify remote in git annex init command Change-Id: Iba4196e92e3eafe00878ca6e8668c969a060e732
2023-07-02tcwg_bmk-build.sh: Fix git-annex commands to get the resultsLaurent Alfonsi
Change-Id: I695d50628ff99f6fedc83c6bf08c0c29b436cc16
2023-07-01round-robin-baseline.sh,tcwg_bmk-build.sh: Move base-artifacts for annex in ↵Laurent Alfonsi
/home/tcwg-buildslave Change-Id: I6cb051e735d9fd1c057930ba2ec478908ce1015b
2023-06-30tcwg_bmk-build.sh: Fix permissions to allow tcwg-benchmark to write resultsLaurent Alfonsi
Change-Id: Id1da2c9d7250a681879872d4fe95c2e37df57369
2023-06-30tcwg_bmk-build.sh: Fix place where to store the bmk resultsLaurent Alfonsi
- remove ssh:// from results_dest, - use absolute path, - create a dir in the workspace where tcwg-benchmark can write, - sync it in artifacts/annex, - and remove it afterwards. Change-Id: I1831cd2a4d42ca001fa78f89805953dcd9c306b9
2023-06-29tcwg_bmk-build.sh: Simplify compare_options handlingLaurent Alfonsi
Change-Id: I9de549918eced19d8aaebc177d0122a0b3b0db8f
2023-06-29round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store bare bmk ↵Laurent Alfonsi
results in artifacts round-robin.sh initialize git-annex as pointing on bkp-01:/home/tcwg-benchmark/base-artifacts, and consider the "annex/" directory as annex datas to be saved with git-annex. Right now, only tcwg_bmk is using it. Taking bare results to the artifacts & baseline simplifies a lot the compare_results routine. tcwg-benchmark-results.sh no more takes results_id string as inputs, but directly takes the directories paths of each bmk results. It takes an additional parameter hw_tag, to avoid precedent results_id parsing. This means this script currently can only treat inputs belonging to the same hw_tag. bmk results are stored in : artifacts/annex/bmk-data We are still keeping results_id file because it is useful to find the benchmark job id that were used to run the benchmarks. Change-Id: I78ef05585be8f1587257f9d8950023b70bff18bc
2023-06-23Revert "round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store ↵Laurent Alfonsi
bare bmk results in artifacts" This reverts commit 4b36216d70e60de56e321d0b403020d27c2d44fe. Reason for revert: pb with ci job deployment make it bancal Change-Id: I89f94a89a0267780c7e119d03a0ab17908625310
2023-06-23Revert "tcwg_bmk-build.sh: Simplify compare_options handling"Laurent Alfonsi
This reverts commit 45c0646915879740a9f834b13345ae685595a703. Reason for revert: pb with ci job deployment make it bancal Change-Id: I84e2b3c2bad1894535d3a4db8c427e6fb42ce524
2023-06-22tcwg_bmk-build.sh: Simplify compare_options handlingLaurent Alfonsi
Change-Id: I4baf0e2f7f900383af51a70d5954b27ab1bdc969
2023-06-22round-robin.sh,tcwg_bmk-build.sh,tcwg-benchmark-results.sh: Store bare bmk ↵Laurent Alfonsi
results in artifacts round-robin.sh initialize git-annex as pointing on bkp-01:/home/tcwg-benchmark/base-artifacts, and consider the "annex/" directory as annex datas to be saved with git-annex. Right now, only tcwg_bmk is using it. Taking bare results to the artifacts & baseline simplifies a lot the compare_results routine. tcwg-benchmark-results.sh no more takes results_id string as inputs, but directly takes the directories paths of each bmk results. It takes an additional parameter hw_tag, to avoid precedent results_id parsing. This means this script currently can only treat inputs belonging to the same hw_tag. bmk results are stored in : artifacts/annex/bmk-data We are still keeping results_id file because it is useful to find the benchmark job id that were used to run the benchmarks. Change-Id: Ibac5ed3508962f1a32ad3361e429f20c66f7eae3
2023-06-21tcwg_bmk-build.sh,round-robin.sh: Fix metric in tcwg_bmk jobsLaurent Alfonsi
Change-Id: I48c7b55797389fd73674924039b3aefdcd5a0072
2023-06-16tcwg_bmk-build.sh: give metric watched parameter to output-bmk-results.pyLaurent Alfonsi
- output-bmk-results.py will raise a regression only in the watched metric. - Also ends with an error if output-bmk-results fails, instead of creating a error-code-$? file Change-Id: I90396862a4c5f9fe0033177cd88cd9ffe5e1ed3c
2023-06-14round-robin-notify.sh,tcwg_bmk-build.sh: Generate regression details at ↵Laurent Alfonsi
notify stage Change-Id: I756d09f190ded50c10fc41ed88f70f9c2925acc4