diff options
| author | Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> | 2020-07-02 15:36:17 +0000 |
|---|---|---|
| committer | Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org> | 2020-07-02 15:50:03 +0000 |
| commit | 9c4410c50069f36e4e232b38a201eefa7aa3dc05 (patch) | |
| tree | 3e07c197c3ba44c2fab570cc8597603f1cac688a /tcwg_bmk-build.sh | |
| parent | 20c44ae22b007917af41d540727544e5d7c0eb0e (diff) | |
tcwg_bmk-build.sh: Workaround missing benchmarking results
... which are referenced in base-artifacts repo.
This problem occurred when bkp-01.tcwglab, where we store benchmarking
results, ran out of disk space, so tcwg-benchmark-* jobs could not
upload their results. This didn't stop tcwg_bmk-build.sh to treat
this failure as regressions in various configurations, which were
then bisected down to the first commit. At that point baselines for
individual configurations were reset to the new failing state, and
references to the missing results were committed to base-artifacts repo.
We will fix this problem in several ways:
1. We have added tcwg-cleanup-stale-results.sh to parse base-artifacts
repo and delete results with no references from that repo (thus
preventing bkp-01.tcwglab from running out of disk space).
2. We are adding this workaround to purge broken entries from
base-artifacts repo.
3. We will commit .csv summary of results into base-artifacts.git so that
we won't need to rely on on-the-side storage of perf.data files.
Change-Id: I922df040292c1686f4461052d21bdeb98bea0b03
Diffstat (limited to 'tcwg_bmk-build.sh')
| -rwxr-xr-x | tcwg_bmk-build.sh | 23 |
1 files changed, 22 insertions, 1 deletions
diff --git a/tcwg_bmk-build.sh b/tcwg_bmk-build.sh index fc416314..9ce91d99 100755 --- a/tcwg_bmk-build.sh +++ b/tcwg_bmk-build.sh @@ -389,6 +389,16 @@ no_regression_vs_p () if [ ! -f "$ref_artifacts/results_id-1" ] || [ ! -f "$ref_artifacts/results_id-2" ]; then return 0 fi + # <Workaround> missing reference results, which we have listed in + # tcwg-benchmark-results.broken-list. Once all entries referencing missing + # results are discarded, we'll remove this workaround. + # Otherwise compare_results will fail while fetching baseline results, + # and we'll consider this failure as a regression. + if cat "$scripts/tcwg-benchmark-results.broken-list" \ + | grep -q "^$(cat $ref_artifacts/results_id-1)\$\|^$(cat $ref_artifacts/results_id-2)\$"; then + return 0 + fi + # </Workaround> compare_results "$ref_artifacts/results_id-1" "$ref_artifacts/results_id-2" "--num_dsos 1 --num_symbols 0" while IFS= read -r -d '' i do @@ -466,7 +476,18 @@ no_regression_to_base_p () if ! [ -f "$ref_artifacts/results_id" ]; then return 0 - elif ! [ -f "$new_artifacts/results_id" ]; then + fi + # <Workaround> missing reference results, which we have listed in + # tcwg-benchmark-results.broken-list. Once all entries referencing missing + # results are discarded, we'll remove this workaround. + # Otherwise compare_results will fail while fetching baseline results, + # and we'll consider this failure as a regression. + if cat "$scripts/tcwg-benchmark-results.broken-list" \ + | grep -q "^$(cat $ref_artifacts/results_id)\$"; then + return 0 + fi + # </Workaround> + if ! [ -f "$new_artifacts/results_id" ]; then return 1 fi |