| Age | Commit message (Collapse) | Author |
|---|
|
Specify binutils_branch and gcc_branch for gnu builds.
Change-Id: I52cd3d0d2837f1fe3c5c492dd657cc6c71408100
|
|
Rename bad_rev and baseline_rev to bad_branch and baseline_branch,
and resolve them to SHA1 revisions.
The motivation for this change is ability to gather historical data,
e.g., code size regressions from GCC/LLVM 5 to 6, to 7, to 8, and so on.
The bisects are intended to be triggered manually, which is why
branch handling is desired.
Change-Id: I9c46270e0092e72836986a4bed08799ec6d84a00
|
|
For linux-next rebase workaround we reset baseline to linux-next:stable,
and for spurious failures we re-trigger build with default parameters.
Change-Id: I448870d101b7dfd554ecc4a91e71ce74538ecec6
|
|
Now that we reliably reach check_regression (due to previous
improvements in run_step support) move creation of trigger
files here from clone_repo.
If updating multiple components -- create trigger files
for individual builds. I.e.,
trigger-build-llvm:
llvm_branch=SHA1_LLVM
linux_branch=baseline
trigger-build-linux:
llvm_branch=baseline
linux_branch=SHA1_LINUX
If updating a single component -- create trigger-bisect like before.
In the tcwg_kernel-bisect.sh script we start using trigger files
produced by the build script, thus simplifying and generalizing
bisect script.
Change-Id: Iaee02ce0f8fbb3f9f5dbee805089ff31f1859cb5
|
|
This patch series is both a major cleanup and an optimization of
round-robin jobs. We switch from testing each component
(binutils, gcc, linux) in separate builds in round-robin fashion
to testing all components in the same build, and, if there's
a failure, re-triggering builds for individual components like
we are doing now. Then if individual build fails, it triggers
bisect on that one component.
The cleanup part is that jenkins triggers are greatly simplified
and we can remove all trigger jobs (of which we had hundreds).
The optimization part is that successful builds (which is
the majority) now consume N times less resources, where N is
the number of components -- 3 in case of binutils, gcc, linux, --
because all components are updated and tested in the same build.
This patch is the first in the series, and it replaces uses of
$current_project and $current_branch with "baseline" keyword
of components' branches. I.e., for testing LLVM in LLVM+Kernel
jobs we currently use
==rr[llvm_branch] master ==rr[linux_branch] master \
==rr[current_project] llvm ==rr[current_branch] SHA1
and it will change to
==rr[llvm_branch] SHA1 ==rr[linux_branch] baseline
. IMO, the latter is significantly more intuitive.
Change-Id: Ib47dcac1571381f870150e4872ec37be5d900a07
|
|
"ERROR:" detects linker errors and "] Error " detects make errors
Change-Id: I91fe470d2cf81487945e2e55796c0dfd3f5dc74c
|
|
Replaces uses of ${rr[linux_version]} and $toolchain with
${rr[ci_config]}, which makes code more generic.
Change-Id: I27030c467b3bdb448aa2dbed36272b93726789cb
|
|
... and use ${rr[X]} instead of $X.
We make an exception for $current_project in tcwg_kernel-bisect.sh
to do a mechanical change in a separate patch.
Change-Id: Ib365552483d016e9096cc791f61a34015e50e32a
|
|
Don't send regression emails for linux breakages with GNU toolchain
to Maxim K. and Christophe L..
Change-Id: Iad009ab97e9acfd6b3d6fb89f26489aecb9b7c9f
|
|
Change-Id: If07fb20b7afef172b948b06a48771f37dcd47e23
|
|
Add an excerpt from manifest listing git URLs and branches
(and avoid entries like rr[gcc_url]="no_gcc_url").
E.g.,
===
Configuration details:
rr[llvm_url]="https://github.com/llvm/llvm-project.git"
rr[llvm_branch]="master"
rr[linux_url]="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git"
rr[linux_branch]="master"
===
Also, don't include bisection range in the email body, since it's not
really useful.
Change-Id: I3a2d54968708db0fbd98bbc724449c06694fa92f
|
|
Linux-next:master rebases almost every day on top of the Linus's tree
(aka linux-next:stable), which creates problems for bisections.
We have tried to workaround these rebases, but results are still not
ideal. This patch reworks (and simplifies!) handling of linux-next.
The goal of testing linux-next:master is to detect regressions before they
reach Linus's tree (linux-next:stable). Therefore, we are really only
interested in regressions between linux-next:stable and linux-next:master
(regressions in Linus's tree will be bisected in another job anyway).
So the new workaround is to force baseline_rev to linux-next:stable
and be done with it!
This also allows us to remove confusing distinction between
baseline_rev and good_rev.
Change-Id: I96002f1d21f68e353c0b62837c4968bf956ecbf2
|
|
Add a quick exit path to the $artifacts/test.sh script, which
simplifies checking of first_bad's parents. Remove list of
skipped_revs since we are not using it.
Change-Id: I3d8601bafcbf1501239fa5a097830181b327454c
|
|
Setting $reproduce_bisect to true has the effect of bisect being
reproduced exactly as the original one, and expects manifests of
all test builds to be present. While this is useful, the common case
is to reproduce a bisect that went wrong, so the reproduction will
diverge from the initial bisection. Therefore to reproduce the common
case use (assuming original artifacts in archive directory)
./jenkins-scripts/tcwg_kernel-bisect.sh @@ archive/artifacts/manifests/bisect.sh -- @@ archive/artifacts/manifests/build-parameters.sh
and to reproduce bisect exactly use
./jenkins-scripts/tcwg_kernel-bisect.sh @@ archive/artifacts/manifests/bisect.sh --reproduce_bisect true
Change-Id: Ie6aa842d9949fad2298b1d56d74973e61541836a
|
|
At the moment we check only first parent of first_bad to confirm
that it is good. This patch extends that to checking/finding
a good parent of merge commits.
Change-Id: I0cb38223e70b10531555615ca39f9d584948c751
|
|
... and their use in reproduction instructions.
Also
- download build-parameter.sh in instructions that test.sh relies
on
- use curl instead of wget in instuctions
- limit commit output to 1000 lines in the email body.
Change-Id: Ib3b0cb94134a588e3a8c72fd7261b70d83935450
|
|
... which is needed by test.sh
Change-Id: I1b1a2c7b8e681308679d56a92edb1c745f3b5ee8
|
|
Add $ci_project and $ci_config parameters to replace references to
tcwg_kernel-specific variables.
Note that linux-next rebase workaround doesn't depend on specifics
of tcwg_kernel project, and would work just fine in other CI
projects, if necessary.
Change-Id: I81e637126bf0622ee1b67e05974c08fc51c313dd
|
|
At the moment *_rev variables, such as current_rev, accept
both git branches and git revisions (SHA1s) as values, which is
confusing and makes it difficult to write code for manifest generation
(manifests need all GIT references resolved at the moment of
generation so that reproduction runs sees the "old" state of
world).
This change standardize the meaning of *_branch and
*_rev suffices: *_branch variables accept either SHA1s or git
branches and *_rev variables accept only SHA1s.
With the established meanings, $bad_rev and $baseline_rev parameters of
tcwg_kernel-bisect.sh script can only be SHA1s (which they are anyway
in all reasonable cases), which allowed to simplify handling of a few
things in tcwg_kernel-bisect.sh script.
Change-Id: Ifc52385e4d3aeab2a3b6945fa6e1c88bcc2c7b6d
|
|
Change-Id: I0fa117d18dcec89e2c5c7e1e65c9f32d28d51511
|
|
Move code for handling "last_good" revision to a better place,
and re-trigger bisect to find true first_bad commits.
Change-Id: I29ee1ef7c287dc9baa6f909142eb57e344cda523
|
|
Change-Id: Ie4ff62b9b296efa745d10e4d51eebac349321cf1
|
|
... in preparation to converting it into a loop. Make all 3
if-clauses follow the same template.
Change-Id: If02069b536c5846db0466281ccc88d9bf0f67dff
|
|
Make it explicit that good_rev is set to baseline_rev.
Change-Id: I61bc702048aa43794661c04726bffda20995c3db
|
|
It appears the simplest and most robust way to reproduce a build
is to follow the bisection procedure: checkout revision by hand
and run a test script to perform the build. Incidentally, this
also allows testing of random commits in the repo, not just
first_bad and last_good commits.
Therefore, we reproduce the full baseline build, which initialize
all repos, builds compilers, etc., and then use the test.sh script
to replicate individual builds.
Change-Id: I5f884b1a485f641900fe33dfec8974ef207920eb
|
|
Change-Id: I05e62cba23e84c09377faadde56eea9513957d9b
|
|
The bisect script will be used in updated instructions
to reproduce the build.
Change-Id: I67b16e4f86f55b80dea39980028b244a95e2417b
|
|
1. Move build-parameters.sh to manifests/, so it survives artifacts
cleaning.
2. Don't overwrite build-parameters.sh in reproducer runs --
use "^^ $reproduce_bisect" when generating it (see documentation
of convert_args_to_variables).
3. Finally, don't pop the manifest filename when reproducing
(since the manifest filename is not pushed when "^^ true").
Change-Id: I2c2cfc571dfc11fcee7b9720d6d30bc5ee9d6897
|
|
... for the benefit of reproducer runs.
Create all symlinks with "-f" to overwrite them in reproducer runs.
Keep baseline directory in place since otherwise the reproducer run
will try to move the symlink thinking it is a directory and we end up
with two symlinks pointing to each other.
The baseline build is special (build-baseline.sh and build-baseline/)
-- it produces pre-requisites for "test" builds. We add build-SHA1
symlinks for the baseline build because it can happen that
last_good == baseline, so when creating symlink for last_good we need
build-<SHA1 of baseline> in place for such cases.
Change-Id: If11b27fad7dc3112a4aa43bb021e1de8d3884d00
|
|
Confirm_failure option was added to quickly detect failures
due to infrastructure problems (e.g., git repos stop responding).
It is saner, simpler and quicker to rely on "baseline" build
to verify that infrastructure is OK, and reproduce "bad" build
just before starting bisection.
Change-Id: Iaf35b260f3ec0ccdf043ee8e03b33bde9ef4a649
|
|
Change-Id: I95c37245a38bd4a8c8cb59969da3a94099f1560b
|
|
Create symlinks for baseline, good, last_good, first_bad and bad
builds instead of renaming directories.
Change-Id: Id343b7167bed4f4eeeacc5093de010c89354754d
|
|
When git-bisect tests commit C as good then it assumes that
all predecessors of C are good, which may not be the case when
several unrelated failures occur.
This patch improves handling of case when parent of $first_bad commit
is bad itself.
Change-Id: I8f6f55b4f3683d0e4f34d1714eeb74a46f4d6f0c
|
|
When trying to use linux mainline, the mainline/master branch might
be ahead of linux-next repo, so the tip commit won't be present in
the git clone. Fix this by fetching mainline repo and using
merge-base of mainline and $bad_rev as tentative good commit.
Change-Id: I21eaa98b2ebd156b846b109e8bb41e6528399f2a
|
|
Make sure that manifests/ directory has build-bad.sh and
build-baseline.sh manifests, which are needed for reproducing
bisects.
Change-Id: I1e2a034286ca34aa898f69fdf726197ba63e1125
|
|
In tcwg_kernel-bisect.sh we use manifest handling to put options
for tcwg_kernel-build.sh script into build-parameters.sh file.
This overwrites __manifest_file variable of bisect job, so we
end up forwarding half of bisect manifest into build-parameters.sh.
Fix this by adding push/pop logic for __manifest_file variable.
Change-Id: I22adc9f0533024063398b5d7cb1a416cad2aa2d1
|
|
Change-Id: Id3a6201bc181fd044ce7f7d648abc15a1763481c
|
|
With updates to tcwg_kernel.yaml we no longer need to
output email recipients and body into shell variable,
and can just create files with text instead.
Change-Id: I85f944aae9fa5ccc446bb26fdf871e65a9164ba1
|
|
... to match changes in tcwg_kernel.yaml.
Change-Id: I03a63d5cfa538a9db8bcb3bf7b423f00f1e6478a
|
|
Move manifest files of individual builds to a dedicated manifests/
directory, which is preserved across runs. This fixes bisect
reproductions, which previously broke due to manifests in
[downloaded] artifacts being removed at the start of this script
while cleaning artifacts directory.
Change-Id: Ieffbb9d482943398a332cd51466f8116e905c170
|
|
Change-Id: Ie2e0f6db672207123a9903eb7f60de23f316f60a
|
|
Handle transitive assumptions of git-bisect about parent commits.
See inline comment for details.
Change-Id: I28e35b249e43325f37155059de4ec356cf140de4
|
|
Try mainline:master as last-resort baseline build.
Change-Id: I3d3b03139e6052bb9254778e1f49d45319313ca8
|
|
Change-Id: I7c68ba0d0a63c922511618350357a13089002127
|
|
We are trying to reduce the number of failed bisect and make
scripting more robust against history re-writes (which is the
primary cause of failed bisects). Re-work logic to generate
email body to include useful info about failed bisects.
Change-Id: Ia25f3311016c0175f3b59e1e4c7308485b5276a4
|
|
Double-quotes can occur in git-show and git-log output,
so replace them with single-quotes to avoid ending $CI_MAIL_BODY
prematurely.
Change-Id: I83a9786b6fdc023e302941a10d1372a479227448
|
|
Change-Id: Ie3eb61223c45bd26872402533e1162e2d9eb8106
|
|
Add Tri Vo and update settings for Maxim K.
Change-Id: I59201cae01b4e04d86d3dae89830b802e4f82dbb
|
|
This requires us to switch from "properties-file" to "script-file"
in JJB inject builder.
Change-Id: Iab56b55c9e8209696137fdf0218ec327f8d7e535
|
|
... so that convert_args_to_variables can create manifest file
when needed
Change-Id: I21e484cb88c15a2dfe1c767663faf71445dc9214
|
