Age | Commit message (Collapse) | Author |
|
Change-Id: Iec9b2ad4813707efbc97d888c31dff420265a921
|
|
It some cases of early build errors, the artifact_list files do not
exist yet, making grep fail.
This is causing trouble where the Jenkins job does not complete until
it times out.
For instance the console ends with:
ERROR (#118): perform_build_steps (Step CHECKOUT failed)
ERROR (#307): build_failure (Build process failed after 1 minutes)
+ jenkins.sh: 1: exit_handler
+ jenkins.sh: exit_handler : 111: true
+ jenkins.sh: exit_handler : 112: upload_to_logserver
+ jenkins.sh: upload_to_logserver : 44: test xdev-01.tcwglab '!=' x
++ jenkins.sh: upload_to_logserver : 45: mktemp -d
+ jenkins.sh: upload_to_logserver : 45: logs_dir=/tmp/tmp.KzE4tbrB88
+ jenkins.sh: upload_to_logserver : 46: sums=
+ jenkins.sh: upload_to_logserver : 49: ssh dev-01.tcwglab mkdir -p '/home/tcwg-buildslave/logs/tcwg-backport-65ff96d8ab4119c637655776aa3df4d3a8131ce5^/0d431ccd6cc23cb1bd5153c6d4ea2c06b039b6c2/x86_64.aarch64-linux-gnu'
+ jenkins.sh: upload_to_logserver : 52: makelog_list=($(read_var ${user_workspace}/build-artifacts.txt 'log_make_[^=]*'))
++ jenkins.sh: upload_to_logserver : 52: read_var /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt 'log_make_[^=]*'
++ jenkins.sh: read_var : 37: local artifact_list=/home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt
++ jenkins.sh: read_var : 38: local 'var=log_make_[^=]*'
++ jenkins.sh: read_var : 39: grep '^log_make_[^=]*=' /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt
++ jenkins.sh: read_var : 39: cut -d = -f 2-
grep: /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt: No such file or directory
and gets stuck here because there is still a background process
waiting for the lock.
This patch is an attempt to avoid that.
Change-Id: I38dff9877bdb6e844bb59859383f1aa0110b1e68
|
|
If the build was already performed, we have nothing to upload to
logserver, so make sure the EXIT handler does not fail in such a case.
Change-Id: I3885ed08fcfe5204b5549eca0a9aa943d879fb0b
|
|
In such a case, check-artifacts.txt does not exist, and the grep
command fails, leaving the script waiting for input on stdin.
Change-Id: I078eb8d819719240c92969690631fb73e6ec867e
|
|
Fix previous patch: after it, we dropped creation of $basedir/$dir and
just kept creation of $(dirname $basedir/$dir), which is obviously not
enough.
Change-Id: I96fa60f66e8192803c30da31d304ab5ac4265f95
|
|
We upload the logs and results at the very end of the script, but in
some cases we exit earlier (for instance when the first cross-compiler
build failed). In such cases we wouldn't upload the logs, relying on
the Jenkins artifacts to debug the problem.
This patch moves the upload code to a dedicated function, which
is called by the EXIT trap handler.
We must be careful to also remove the locks if needed, so we create an
EXIT trap handler to perform both actions: upload the
logs/results/manifest and clean the locks.
This also fixes a potential problem where a build failure would not
upload the manifest (in case the failing build went far enough to
generate the manifest).
Change-Id: I45d8db3993a986b529756454e5c022105f4c4deb
|
|
This reverts commit 95beec124ec5f9bd95d88fd6c00ecd4a0e718dab.
This patch seems to be causing problems: buildfarm jobs are stuck at "exit 0", and then get killed by a Jenkins timeout after 10h.
Change-Id: I4784c61d863d33fa43a89ebf1b39a573c48a7873
|
|
We upload the logs and results at the very end of the script, but in
some cases we exit earlier (for instance when the first cross-compiler
build failed). In such cases we wouldn't upload the logs, relying on
the Jenkins artifacts to debug the problem.
This patch moves the logs upload code to a dedicated function, which
is called as the EXIT trap handler.
Change-Id: I7106ff19d933bdf90b41cf7adb1c78e25f8bba3e
|
|
And forward it to ABE.
Change-Id: Iad877a0f00b06da3259e291974acd3a291efad08
|
|
To switch to dedicated log-providing ssh server running inside a container.
Change-Id: I584ff754dfb3cf075625dfaad70e048bed4c3bdc
|
|
Otherwise check-artifacts.txt does not exist and grep fails,
making jenkins.sh fail too. If --runtests was not used, we
do not want to fail for lack of .sum files.
Change-Id: I6cb7736cf40ec4b036d5806b9ee89a6f3d8dbb53
|
|
Since these variables are not propagated to the containers,
stop using the empty values for: NODE_NAME, JOB_NAME,
BUILD_USER_FIRST_NAME, BUILD_USER_LAST_NAME, BUILD_USER_ID.
This patch removes the use of BUILD_URL only when printing it
to the manifest, and does not remove the uses of BUILD_URL
when acquiring the build lock, which should be done in a
later patch.
Change-Id: I3f4677cb174c3d8604fb0c57a4de39e38d4187ce
|
|
Use --list-artifacts to locate tarballs, manifests Abe's logs, and
*.sum/*.log files.
This eliminates use of find from jenkins.sh.
Change-Id: I824eac39e2dbcf7f3ef3cc893b446c4964e29ad4
|
|
We now start test containers before calling jenkins.sh
Change-Id: I88799d0837af8aa74fd48899e60ca60877bf6352
|
|
Change-Id: I7aaa68e9dfbcded3995127dcd099032232720bca
|
|
Change-Id: I499c4ed5d2f1c872de200ac2072e0ba0eebcfcec
|
|
Instead of just printing an help message and continuing.
Change-Id: I6734639095589c861e72d8c0f4a444b86fefa1fc
|
|
WORKSPACE used to be set to $(pwd) if not already defined,
but if jenkins.sh is called remotely via ssh for instance,
WORKSPACE is undefined but the build succeeds in $HOME
instead, which is misleading and can confuse other Jenkins
script.
This patch removes this default value and prints an error
message in case $user_workspace (derived from $WORKSPACE,
or defined by a command-line option) is undefined.
Change-Id: I53a927a8f5e00430e2aefa22991daa42b2776b95
|
|
Change-Id: I8e66a03e2533620a6d941d2af4715502f87ef6d4
|
|
When jenkins.sh is called shortly before midnight, the 2 calls to
abe.sh can generate 2 manifest files (different name because of
different date). The code can handle one only.
Change-Id: I30105f39a4c121cec095cd4cbe1000d5b6a96817
|
|
In MakeRelease.job, rename --abepath to --abedir
for homogeneity.
In jenkins.sh, replace --abe to --abedir, remove
support for short option name (-c).
Change-Id: Ic35f1a1a73e848da4da7221017db552d73d25db0
|
|
When $runtests is false check.out and check.err are not generated.
Change-Id: Ic5b0d3de2ad2edf711805099ec48e627d048d7cb
|
|
When downloading qemu.
Change-Id: I840697b23eb58e7fa280fbca2fc569694eb6a5fa
|
|
Shellcheck warns that backticks are deprecated, so replace
them with the more modern $(...) syntax.
This patch has been generated automatically with this shell
command:
perl -i -ape 's/`\(([^()`]*)\)`/\$($1)/; s/`([^(][^`]*)`/\$($1)/' *.sh lib/*.sh
This leaves two more `...` in test.sh which will be replaced by
hand in a later patch.
Change-Id: Ia909ee0d7b2d4049c6cacf3de3babafb0f4d0f07
|
|
In fact, the option has been removed a long time ago,
but some currently unused code in jenkins.sh was still
using it.
Change-Id: I1bf2810b31704a6693bec609b4620b1ae34f4b1f
|
|
... when --testcontainerfile option is given to jenkins.sh.
It can take up to 20 minutes to start a testing container when the
testing board is under heavy load, so it is best to parallelize it
with the build of the toolchain. If starting a container exceeds
time required to build the toolchain -- then we need to investigate
and fix the bottlenecks on the tester side.
Change-Id: Iba061228a9aec90e8d7f9e71162526685a48d8b4
|
|
The purpose is two-fold. Firstly, this patch is a step towards
separating build and test stages, which we need for testing of
binary releases.
Immediatelly, however, this patch makes --testcontainerfile
jenkins.sh option possible. By splitting test from the build
we give time for $testcontainerfile to appear once the test container
is started.
Change-Id: Ie85fffa8f46498f08fc2eb2fcdd5733586826a1e
|
|
Change-Id: I27d620b217008c8bd8d7d12b0c6f98bc670a0107
|
|
Remove --fileserver from jenkins.sh, but keep it in MakeRelease.job,
since the latter is using fileserver as logserver.
Also fix path to reference snapshots.
Change-Id: Idbb77318810e5f9c0597a6182e2bc84d036e9400
|
|
The gerrit patches are in refs/changes/*, which are not fetched
by default by git clone. Since we are putting more and more stuff
to gerrit, it makes sense for us to fetch these commits in ABE.
The extra refs are going to be present in the reference git repos,
so overhead will be negligible.
Change-Id: I3043decb858231a3fcf2bf8a40ed12f4550e8739
|
|
This qemu should no longer fail sync-4 test.
Change-Id: I8f2a36fec3cd9b00d2b0b8a7eacc4e717df99859
|
|
... otherwise aarch64 and armhf hosts try to execute x86_64-hosted
qemu-* binaries.
Change-Id: I1e7399c05aa409ad9fa35dcab47d13c325403ed5
|
|
This WIP qemu binaries provide significantly better results:
800 more PASSes, 400 less FAILs -- compared to Trusty qemu-armeb.
The tarball also provides qemu-arm and qemu-aarch64, which perform
just as well as qemu-armeb.
Change-Id: I21113651a01ff4af4a569673fdb54741b41aff54
|
|
configure.
This information is more useful in the case of builds started by Jenkins than
when started via the command line.
Change-Id: I00952a4c58fb2f28113d7e3e94a81001d3184c33
|
|
Currently the changes and overrides to components are not passed to the
--checkout all command. This patch fixes this.
Change-Id: If902c3cc67b0de5790f2c91f9b7743882daf9e19
|
|
it's dash instead.
Change-Id: I2b819d18b52e35045684948084b125c35d4d43a6
|
|
Change-Id: I1ba1ae046cd600042d2c473a3afd551c6a6688e7
|
|
$revision variable is not used anywhere in jenkins.sh
Change-Id: If11fb5738bd6cee4b7a20b35fb63b0112a72797b
|
|
... to try workaround sanitizer testsuite instabilities and races.
Note that the Hetzner builders have been renamed tcwg-ex40-XX -> ex40-XX,
so the old stanza is no longer applicable. Also the change will only
affect the 32-core builders, and the old 4-core builders will remain at -j4
parallelism.
Change-Id: Ie7c1ef6a3d89264e5fbad5e06520c20d8582baee
|
|
Change-Id: Iad2016cb3692c9320555a982311d5792b1a95ac0
|
|
... to specify non-default versions for glibc and other components.
Change-Id: I7322c7fd624135d7551b1da3e2afd217e1dfebd6
|
|
If building with such options as tarbin, we get a duplicate manifest inside
destdir. Look for the manifest inside build, excluding destdir. There should
always be exactly one such manifest.
Symptom of multiple manifests is jenkins.sh exiting with error due to:
+ jenkins.sh: 405: test 'x/<some_path>/foo@bar-<date>-manifest.txt
/<some other path>/foo@bar-<date>-manifest.txt' '!=' x
+ jenkins.sh: 406: echo node=tcwg_x86_64_ex40build_07
./jenkins.sh: line 406: ${manifest}: ambiguous redirect
Change-Id: Id3db8c084622efb0f09504eec51a7b4d1594272a
|
|
Change-Id: Ib08025be14712a6bc536055e87cc71d82164d5df
|
|
There is an ongoing problem which means that expect is unable to spawn
processes. Detect the condition before running make check, and try to output
a little bit of debug info.
Change-Id: Ib9432973d70562da6d31533c64214bcac67cfdec
|
|
|
|
${change} usually contains the GCC revision to build, which is
generally only in gerrit changes. Thus, --checkout all cannot
access it. This is the purpose of the subsequent git fetch, which
brings the gerrit changes into our local repo.
The GCC revision will be checked out during the abe build process.
Change-Id: I8ac9fa45444e2e969f2ceebfa9610af4fa5246d4
|
|
Special care is needed to continue if the build returned an error:
We still want to see the logs etc....
This patch helps catching errors earlier.
Change-Id: I1614385775ca7e0d57805666a2ce79f3f9101b77
|
|
I verified that "--checkout all" works as expected with source update
enabled and disabled.
Change-Id: I2c4016db4ba51b3ec1f03ece6d26223334cbb998
|
|
Validation of ABE via Jenkins currently does not pass --logserver when calling jenkins.sh. This results in errors which are silently ignored.
This patch fixes this.
Change-Id: Ifc70d6ef6b8505a3d0bfd5b3f4ab6bd961cd5df1
|
|
Right now we test native AArchXX toolchains by bootstrapping them, but without
running testsuites (which takes 3-4 hours). Without uploading /something/
after a successfull bootstrap to the logserver, we are bound to re-build
the toolchains needlessly. This patch fixes that.
Change-Id: Ia5c2e057f625cc62c461d023b3acc4f8003deada
|