Age | Commit message (Collapse) | Author |
|
Fixes for:
https://github.com/koalaman/shellcheck/wiki/SC2207
Are using the bash 4.x 'mapfile' syntax where the
input to the array is seperated by newlines.
If it's separated by spaces we use 'read' instead.
Support for selecting libc in MakeRelease.job was
removed in 3fa23da28ad73d0fe4b512c4aab269b5280a6650.
So remove references to "libc" which were expanding
to nothing anyway.
In tcwg-llvm-common.sh PROJECTS->BRANCHED_PROJECTS
is not strictly needed. However shellcheck can't
tell that the local PROJECTS in this file is
different from the PROJECTS in tcwg-llvm-build.sh.
Change-Id: Ie55968e3bf185d0bd5795fe48014636cda1009bc
|
|
Now using the version that's
present in the bionic images.
Meaning we have more warnings
but can also set the minimum
level to ignore a lot of them.
Fix the check for whether
shellcheck supports --severity.
(we have pipefail on, but we
expect shellcheck to fail in
this case)
Change-Id: I296b8554591b7d327c75393e3924184abc6512b7
|
|
Also update the sanity check
script to include .job files.
Change-Id: Ibf72d9ff346ff7b012d4e282a6ac8f3b7e73590d
|
|
In such a case, the affectation to $sums fails, and because we are
running under 'set -e' the script exits before completing the
execution of exit_handler() and does not remove the lock.
Change-Id: Ice52951ac61a72c19c93d894aab7d6dca96e8db5
|
|
The point of this patch is to replace --enable bootstrap from our
existing scripts with --set buildconfig=bootstrap, so we can remove
--enable bootstrap option from abe. A quick grep in jenkins-scripts
repo shows that only jenkins.sh and round-robin.sh pass
"--enable bootstrap" to abe.
Change-Id: I76294650cfc388f2f218a0af482f36c4a8fbd9f1
|
|
We were overriding qemu with prebuilt binaries provided by Peter Maydell.
Apparently we were doing so as those binaries show test improvements.
As we have moved to docker containers we should update these binaries
in docker files if needed.
Change-Id: I2a95521d65682bffe2385fbc49aeb84058858f81
|
|
Change-Id: Iec9b2ad4813707efbc97d888c31dff420265a921
|
|
It some cases of early build errors, the artifact_list files do not
exist yet, making grep fail.
This is causing trouble where the Jenkins job does not complete until
it times out.
For instance the console ends with:
ERROR (#118): perform_build_steps (Step CHECKOUT failed)
ERROR (#307): build_failure (Build process failed after 1 minutes)
+ jenkins.sh: 1: exit_handler
+ jenkins.sh: exit_handler : 111: true
+ jenkins.sh: exit_handler : 112: upload_to_logserver
+ jenkins.sh: upload_to_logserver : 44: test xdev-01.tcwglab '!=' x
++ jenkins.sh: upload_to_logserver : 45: mktemp -d
+ jenkins.sh: upload_to_logserver : 45: logs_dir=/tmp/tmp.KzE4tbrB88
+ jenkins.sh: upload_to_logserver : 46: sums=
+ jenkins.sh: upload_to_logserver : 49: ssh dev-01.tcwglab mkdir -p '/home/tcwg-buildslave/logs/tcwg-backport-65ff96d8ab4119c637655776aa3df4d3a8131ce5^/0d431ccd6cc23cb1bd5153c6d4ea2c06b039b6c2/x86_64.aarch64-linux-gnu'
+ jenkins.sh: upload_to_logserver : 52: makelog_list=($(read_var ${user_workspace}/build-artifacts.txt 'log_make_[^=]*'))
++ jenkins.sh: upload_to_logserver : 52: read_var /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt 'log_make_[^=]*'
++ jenkins.sh: read_var : 37: local artifact_list=/home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt
++ jenkins.sh: read_var : 38: local 'var=log_make_[^=]*'
++ jenkins.sh: read_var : 39: grep '^log_make_[^=]*=' /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt
++ jenkins.sh: read_var : 39: cut -d = -f 2-
grep: /home/tcwg-buildslave/workspace/tcwg-buildfarm/aarch64-linux-gnu/build-artifacts.txt: No such file or directory
and gets stuck here because there is still a background process
waiting for the lock.
This patch is an attempt to avoid that.
Change-Id: I38dff9877bdb6e844bb59859383f1aa0110b1e68
|
|
If the build was already performed, we have nothing to upload to
logserver, so make sure the EXIT handler does not fail in such a case.
Change-Id: I3885ed08fcfe5204b5549eca0a9aa943d879fb0b
|
|
In such a case, check-artifacts.txt does not exist, and the grep
command fails, leaving the script waiting for input on stdin.
Change-Id: I078eb8d819719240c92969690631fb73e6ec867e
|
|
Fix previous patch: after it, we dropped creation of $basedir/$dir and
just kept creation of $(dirname $basedir/$dir), which is obviously not
enough.
Change-Id: I96fa60f66e8192803c30da31d304ab5ac4265f95
|
|
We upload the logs and results at the very end of the script, but in
some cases we exit earlier (for instance when the first cross-compiler
build failed). In such cases we wouldn't upload the logs, relying on
the Jenkins artifacts to debug the problem.
This patch moves the upload code to a dedicated function, which
is called by the EXIT trap handler.
We must be careful to also remove the locks if needed, so we create an
EXIT trap handler to perform both actions: upload the
logs/results/manifest and clean the locks.
This also fixes a potential problem where a build failure would not
upload the manifest (in case the failing build went far enough to
generate the manifest).
Change-Id: I45d8db3993a986b529756454e5c022105f4c4deb
|
|
This reverts commit 95beec124ec5f9bd95d88fd6c00ecd4a0e718dab.
This patch seems to be causing problems: buildfarm jobs are stuck at "exit 0", and then get killed by a Jenkins timeout after 10h.
Change-Id: I4784c61d863d33fa43a89ebf1b39a573c48a7873
|
|
We upload the logs and results at the very end of the script, but in
some cases we exit earlier (for instance when the first cross-compiler
build failed). In such cases we wouldn't upload the logs, relying on
the Jenkins artifacts to debug the problem.
This patch moves the logs upload code to a dedicated function, which
is called as the EXIT trap handler.
Change-Id: I7106ff19d933bdf90b41cf7adb1c78e25f8bba3e
|
|
And forward it to ABE.
Change-Id: Iad877a0f00b06da3259e291974acd3a291efad08
|
|
To switch to dedicated log-providing ssh server running inside a container.
Change-Id: I584ff754dfb3cf075625dfaad70e048bed4c3bdc
|
|
Otherwise check-artifacts.txt does not exist and grep fails,
making jenkins.sh fail too. If --runtests was not used, we
do not want to fail for lack of .sum files.
Change-Id: I6cb7736cf40ec4b036d5806b9ee89a6f3d8dbb53
|
|
Since these variables are not propagated to the containers,
stop using the empty values for: NODE_NAME, JOB_NAME,
BUILD_USER_FIRST_NAME, BUILD_USER_LAST_NAME, BUILD_USER_ID.
This patch removes the use of BUILD_URL only when printing it
to the manifest, and does not remove the uses of BUILD_URL
when acquiring the build lock, which should be done in a
later patch.
Change-Id: I3f4677cb174c3d8604fb0c57a4de39e38d4187ce
|
|
Use --list-artifacts to locate tarballs, manifests Abe's logs, and
*.sum/*.log files.
This eliminates use of find from jenkins.sh.
Change-Id: I824eac39e2dbcf7f3ef3cc893b446c4964e29ad4
|
|
We now start test containers before calling jenkins.sh
Change-Id: I88799d0837af8aa74fd48899e60ca60877bf6352
|
|
Change-Id: I7aaa68e9dfbcded3995127dcd099032232720bca
|
|
Change-Id: I499c4ed5d2f1c872de200ac2072e0ba0eebcfcec
|
|
Instead of just printing an help message and continuing.
Change-Id: I6734639095589c861e72d8c0f4a444b86fefa1fc
|
|
WORKSPACE used to be set to $(pwd) if not already defined,
but if jenkins.sh is called remotely via ssh for instance,
WORKSPACE is undefined but the build succeeds in $HOME
instead, which is misleading and can confuse other Jenkins
script.
This patch removes this default value and prints an error
message in case $user_workspace (derived from $WORKSPACE,
or defined by a command-line option) is undefined.
Change-Id: I53a927a8f5e00430e2aefa22991daa42b2776b95
|
|
Change-Id: I8e66a03e2533620a6d941d2af4715502f87ef6d4
|
|
When jenkins.sh is called shortly before midnight, the 2 calls to
abe.sh can generate 2 manifest files (different name because of
different date). The code can handle one only.
Change-Id: I30105f39a4c121cec095cd4cbe1000d5b6a96817
|
|
In MakeRelease.job, rename --abepath to --abedir
for homogeneity.
In jenkins.sh, replace --abe to --abedir, remove
support for short option name (-c).
Change-Id: Ic35f1a1a73e848da4da7221017db552d73d25db0
|
|
When $runtests is false check.out and check.err are not generated.
Change-Id: Ic5b0d3de2ad2edf711805099ec48e627d048d7cb
|
|
When downloading qemu.
Change-Id: I840697b23eb58e7fa280fbca2fc569694eb6a5fa
|
|
Shellcheck warns that backticks are deprecated, so replace
them with the more modern $(...) syntax.
This patch has been generated automatically with this shell
command:
perl -i -ape 's/`\(([^()`]*)\)`/\$($1)/; s/`([^(][^`]*)`/\$($1)/' *.sh lib/*.sh
This leaves two more `...` in test.sh which will be replaced by
hand in a later patch.
Change-Id: Ia909ee0d7b2d4049c6cacf3de3babafb0f4d0f07
|
|
In fact, the option has been removed a long time ago,
but some currently unused code in jenkins.sh was still
using it.
Change-Id: I1bf2810b31704a6693bec609b4620b1ae34f4b1f
|
|
... when --testcontainerfile option is given to jenkins.sh.
It can take up to 20 minutes to start a testing container when the
testing board is under heavy load, so it is best to parallelize it
with the build of the toolchain. If starting a container exceeds
time required to build the toolchain -- then we need to investigate
and fix the bottlenecks on the tester side.
Change-Id: Iba061228a9aec90e8d7f9e71162526685a48d8b4
|
|
The purpose is two-fold. Firstly, this patch is a step towards
separating build and test stages, which we need for testing of
binary releases.
Immediatelly, however, this patch makes --testcontainerfile
jenkins.sh option possible. By splitting test from the build
we give time for $testcontainerfile to appear once the test container
is started.
Change-Id: Ie85fffa8f46498f08fc2eb2fcdd5733586826a1e
|
|
Change-Id: I27d620b217008c8bd8d7d12b0c6f98bc670a0107
|
|
Remove --fileserver from jenkins.sh, but keep it in MakeRelease.job,
since the latter is using fileserver as logserver.
Also fix path to reference snapshots.
Change-Id: Idbb77318810e5f9c0597a6182e2bc84d036e9400
|
|
The gerrit patches are in refs/changes/*, which are not fetched
by default by git clone. Since we are putting more and more stuff
to gerrit, it makes sense for us to fetch these commits in ABE.
The extra refs are going to be present in the reference git repos,
so overhead will be negligible.
Change-Id: I3043decb858231a3fcf2bf8a40ed12f4550e8739
|
|
This qemu should no longer fail sync-4 test.
Change-Id: I8f2a36fec3cd9b00d2b0b8a7eacc4e717df99859
|
|
... otherwise aarch64 and armhf hosts try to execute x86_64-hosted
qemu-* binaries.
Change-Id: I1e7399c05aa409ad9fa35dcab47d13c325403ed5
|
|
This WIP qemu binaries provide significantly better results:
800 more PASSes, 400 less FAILs -- compared to Trusty qemu-armeb.
The tarball also provides qemu-arm and qemu-aarch64, which perform
just as well as qemu-armeb.
Change-Id: I21113651a01ff4af4a569673fdb54741b41aff54
|
|
configure.
This information is more useful in the case of builds started by Jenkins than
when started via the command line.
Change-Id: I00952a4c58fb2f28113d7e3e94a81001d3184c33
|
|
Currently the changes and overrides to components are not passed to the
--checkout all command. This patch fixes this.
Change-Id: If902c3cc67b0de5790f2c91f9b7743882daf9e19
|
|
it's dash instead.
Change-Id: I2b819d18b52e35045684948084b125c35d4d43a6
|
|
Change-Id: I1ba1ae046cd600042d2c473a3afd551c6a6688e7
|
|
$revision variable is not used anywhere in jenkins.sh
Change-Id: If11fb5738bd6cee4b7a20b35fb63b0112a72797b
|
|
... to try workaround sanitizer testsuite instabilities and races.
Note that the Hetzner builders have been renamed tcwg-ex40-XX -> ex40-XX,
so the old stanza is no longer applicable. Also the change will only
affect the 32-core builders, and the old 4-core builders will remain at -j4
parallelism.
Change-Id: Ie7c1ef6a3d89264e5fbad5e06520c20d8582baee
|
|
Change-Id: Iad2016cb3692c9320555a982311d5792b1a95ac0
|
|
... to specify non-default versions for glibc and other components.
Change-Id: I7322c7fd624135d7551b1da3e2afd217e1dfebd6
|
|
If building with such options as tarbin, we get a duplicate manifest inside
destdir. Look for the manifest inside build, excluding destdir. There should
always be exactly one such manifest.
Symptom of multiple manifests is jenkins.sh exiting with error due to:
+ jenkins.sh: 405: test 'x/<some_path>/foo@bar-<date>-manifest.txt
/<some other path>/foo@bar-<date>-manifest.txt' '!=' x
+ jenkins.sh: 406: echo node=tcwg_x86_64_ex40build_07
./jenkins.sh: line 406: ${manifest}: ambiguous redirect
Change-Id: Id3db8c084622efb0f09504eec51a7b4d1594272a
|
|
Change-Id: Ib08025be14712a6bc536055e87cc71d82164d5df
|
|
There is an ongoing problem which means that expect is unable to spawn
processes. Detect the condition before running make check, and try to output
a little bit of debug info.
Change-Id: Ib9432973d70562da6d31533c64214bcac67cfdec
|