aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-07-05test-checkout.sh: Increase length of tail-of-logHEADmasterChristophe Lyon
When manifests differ, we dump the tail of the log of the comparison build, but since we execute the build under 'set -x', this includes traces that make interesting/important information disappear from the tail of the tail. This patch simply dumps the last 20 lines, which looks sufficient. Change-Id: I4c336d9c5fabd4a49869fd6638ea000cc3b130b8
2023-02-28test-manifest2.sh: Fix typoMaxim Kuvyrkov
The only reason I see for this working before -- is that bash versions before one in Ubuntu 22.04 didn't consider it an error to read from absent file. Change-Id: I5261c7823995ebffd4314f13f7c5927973218ef0
2021-01-14Update gmp and mpfr versions in reference manifestChristophe Lyon
Since we recently unconditionally updated the versions of gmp and mpfr, we need to update the manifest we use for testing to avoid failures. Change-Id: Ibad04973d0fd38319102466ea51c56bd2c482f5e
2020-12-15Add $PWD to wrapper's error outputMaxim Kuvyrkov
Change-Id: Id1e9af44681092d1367769ad6838176819cae4a9
2020-12-07Revert "Update expected fields_for_component to not require gdbserver_configure"Maxim Kuvyrkov
This reverts commit 12c00a3039aab006db2ea977ee65ea08d6b8e0ae. Reason for revert: <INSERT REASONING HERE> Change-Id: I3a88632e1d826cd7435f28c70bfb254cf910101a
2020-12-01test-manifest2.sh: Better handle subcomponents in check_revisions_in_manifestMaxim Kuvyrkov
... now that we use gdbserver as subcomponent to itself. Change-Id: I673efcf620c98e80047884c877d421c4567cc831
2020-11-26Update expected fields_for_component to not require gdbserver_configureMaxim Kuvyrkov
... after fixing (hopefully!) sysroot handling in ABE. Change-Id: I26c672fbc6f3dfc0bc87491e94cad50725326962
2019-03-14validate-manifest.pl: Handle manifests version 1.6Christophe Lyon
They include qemu (but only for linux-hosted toolchains) Change-Id: I2d7f82bee2f6c4d3aef9278e7f6c3216a87637d2
2019-01-17test-checkout.sh: Dump tail of log in case of error.Christophe Lyon
This is to help diagnose problems when looking at the Jenkins console, without having to need to connect to the builder and manually inspect the logs (which may have been cleaned up since the job ran). Change-Id: If4f5573036808846ab6bce44a4b3fb40e96bd732
2018-08-03test-manifest2.sh: Fix sorting of file lists.Christophe Lyon
Change-Id: I362aba30e9c129d059e45ebbd0c14fee10f509f8
2018-04-24Add ':' after FAIL/FAILED/PASS/PASSED in 'echo' commands.Christophe Lyon
The purpose is to make these messages easier to match in the Jenkins console (as 'fail' or 'failed' appear in locations unrelated to the tests). Change-Id: Iece616e1877c5bba04e01ea7609f531b73ef2415
2017-07-06test-checkout.sh: Improve commit message tag detection.Charles Baylis
Scan the whole history between the selected previous Abe version and the version under test. This ensures that the tags are detected if the HEAD of the Abe branch is not the one where the manifest format is changed (eg if testing with --prev-abe-revision origin/tested) Also, don't require the tags to be placed at the start of a line in the commit message. Change-Id: Iab46c9fdbf4a0f6d1f3967f88a03bca36a6d029b
2017-07-06test-checkout.sh: Improve diagnostics when test is skipped.Charles Baylis
When using one of the commit message tags to skip the previous Abe tests, record the comparison result in the output. Change-Id: I6e9e4fbe98fca4576cceac6f56cef8190dd93b72
2017-06-27validate-manifest.pl: Support version 1.5 manifests (for libiconv)Charles Baylis
Manifest format version 1.5 introduces libiconv as a new required component. The following changes are required: . accept version 1.5 . it manifest version >= 1.5, require libiconv for all host platforms. Change-Id: I469481e647de8dc13e969d1def0fc66ceb59a56d
2017-06-26test-checkout.sh: Read test flags from Abe commit logs.Charles Baylis
The test_previous_abe_cmd_line and test_previous_abe_manifest tests report failures even when the change to the manifest was an intentional result of a change in Abe. This is particularly annoying in the tcwg-regression tests, where these failures can be reported for other changes for as long as the Abe change remains the most recent Abe commit. Therefore, introduce new tags which can be incorporated into commit messages, which indicate that the manifest comparisons should not be done. These have the format: [ABE_PREV_<MANIFEST|CMD>_<VERSION|ALL>] For example, the following tags can be used: [ABE_PREV_MANIFEST_GCC4_9] - skip the manifest comparison for the previous manifest test on gcc 4.9 [ABE_PREV_CMD_GCC5] - skip the manifest comparison for the previous command line test on gcc 5 [ABE_PREV_CMD_ALL] - skip the manifest comparison for the previous command line test on all GCC versions Change-Id: I5753adb5e917da1141ef79037333cb54f88a167e
2017-06-26test-checkout.sh: Share snapshots for the prev_abe_* tests.Charles Baylis
Avoid the cost of repeated git clones for the prev_abe_* tests. These tests are not intended to prove that clean checkouts work, which is already covered by the earlier tests. In fact, sharing the snapshots directory slightly improves testing coverage. Change-Id: I6e35b6fd5b5b9e210cef1ddb4e178e6b02e37333
2017-05-24test-checkout.sh: Automatically vary default target.Charles Baylis
Add support for --target auto, and make "auto" the default target. The "auto" target cycles around the list of all released targets. Change-Id: Ib2b79d6b07b7f663ffd7bc04b4b9543ff5f211ed
2017-04-10test-manifest2.sh: Tests for --list-artifacts.Charles Baylis
Ensure that all artifacts have absolute paths, and point to files which exist. Change-Id: Icbadd8171b8c9671403d315468972dfaa244eca0
2017-04-03test-checkout.sh: Tolerate manifest version change.Charles Baylis
When doing the prev_abe_manifest_* tests, and the previous version of Abe generates an older format manifest than the current version, it is expected that the new manifest may be incomplete (eg missing required components which were not in the previous format of manifest). Therefore, when the manifest formats differ, pass --type=generic to validate_manifest.pl, so that only the syntax of the manifest is checked. In all cases, compare_manifests.pl passes --type=generic, because its goal is comparison not validation (ideally, it would be even less strict, but validate_manifest.pl does not yet have a suitable mode) The test will still fail in these cases because the manifests differ, but this is the expected behaviour because it requires manual inspection of the changes. Change-Id: I8b4cbfc4c456c6a59fcdee0d75aebe1ce61a2181
2017-03-30Use --list-artifacts instead of find.Charles Baylis
Use --list-artifacts for the manifest tests and checkout tests in place of using find to locate the manifest and the toolchain tarballs. Change-Id: Icf88f37c0ec9512d07d12f3ca1580347bc4bac32
2017-03-27validate-manifest.pl: Support generic manifests.Charles Baylis
Currently, validate-manifest.pl simultaneous validates both the syntax of the manifest file, and that its contents match the expected output from Abe when it builds a full gcc toolchain. This can be useful for partial validation of manifests which arise from partial builds (eg abe.sh --build gcc), or from old format manifests which don't contain components which are now required. Change-Id: I735a1038eac94c9d30c9ab99361172555db6cfb7
2017-03-20test-checkout.sh: Test gcc7 checkout.Charles Baylis
Add testing of --extraconfigdir gcc7 to the set of config dir validation tests. Change-Id: Id3715a70ba280cf5bddc04f974f105a5ade8106d
2017-03-14Check behaviour of gcc=... options.Charles Baylis
Improve the test_gccver_* tests so that they ensure that the expected branch, tag, revision or URL was actually selected by Abe. Change do_test to take an explicit test name, which improves the ease of selecting the right log file in the artifacts list. Change-Id: Ida212680593da4428b86a2820357090d58c73e54
2017-03-08validate-manifest.pl: Add --verbose flagCharles Baylis
Add --verbose flag to display an explicit message if the manifest is valid. Change-Id: Ie44fa321a2b3906b54ac04d10625c71cfce2da7b
2017-03-08validate-manifest.pl: Handle multiple manifests.Charles Baylis
Accept multiple manifests on a single command line. Cache md5sums of downloaded files so that files are only downloaded once when validating multiple files. Change-Id: I718da2e6c15ac2774e9a3967724da7ff9c3ef8b9
2017-03-07Use --release so that pkgversion is independent of date.Charles Baylis
If --release is not specified Abe sets --with-pkgversion with the current date, which can cause spurious test failures if midnight occurs during the test. Change-Id: I666723933ede610281cdab1b66c97125a049980c
2017-03-03Merge "Update abe-tests for manifest format 1.4."Charles Baylis
2017-03-03Update abe-tests for manifest format 1.4.Charles Baylis
Handle mingw_only and mingw_extraconf flags. Test that mingw and linux manifests are identical. Make expat and python required components for manifest format >= 1.4 Change-Id: I60fcb8d7d5bb58ab5d3c628551817e04bcde7996
2017-01-04Ignore manifestid when comparing with --ignore-revisions.Charles Baylis
Since different revisions cause the manifest id to change, it is necessary to also ignore the manifestid field when the --ignore-revisions switch is passed to compare-manifests.pl. This fixes a problem with the abe-test-checkout test when midnight occurs while the test_previous_abe_cmd_line subtests are running. Change-Id: I4d9e3aa49119bcfca39ae854fb77e2e806aaab68
2016-12-20Add option to choose ABE revisions for comparison.Charles Baylis
Add --abe-revision and --prev-abe-revision options. It is not permissible to use --abe-revision with --abe-path. It is assumed that if --abe-path is supplied then the pre-existing abe checkout is to be tested. An error is reported if this is attempted. Example usage: $ abe-tests/test-checkout.sh --abe-revision HEAD --prev-abe-revision HEAD~1 Implements TCWG-995 (ABE: Manifest testing script needs to compare two sha1s instead of HEAD and HEAD-1) Change-Id: Iec7896107c1c9173f6c5efa176fada2241c87a21
2016-12-20Add --only-prev-abe-test switchCharles Baylis
This patch adds the --only-prev-abe-test switch which means that only the tests which compare two abe revisions are run. This is intended to be used for the validation/release integration testing. Implements TCWG-993 (ABE: move manifest tests out of checkout tests) Change-Id: I831c1b651d26ddc7b3e44658989b5bef73cbd648
2016-11-29test-checkout.sh: Keep abe's configure output in log.Christophe Lyon
Print the actual log name in case of error. Change-Id: Idb63ad0e06499140554e0ef90ea104660966fdd2
2016-11-22Remove use of the do_test_with_msg function.Charles Baylis
THis function causes 'set -e' to be supressed when doing the prev-abe tests. The message is potentially misleading when the test fails for a genuine reason, so it's best removed anyway. Change-Id: I2058375358ecd921f0cd5908b4db089207332366
2016-11-21Manifest comparison with previous ABE versionCharles Baylis
There are two tests added here: First test (implements TCWG-892): . checkout previous revision of ABE. . use that version of ABE to perform --checkout all, and generate a manifest . use the current version of ABE to perform --checkout all, with the manifest from the first run of ABE. This also generates a new manifest. . compare the two manifests for equivalence. Second test (implements TCWG-893): . checkout previous revision of ABE. . use that version of ABE to perform --checkout all, and generate a manifest . use the current version of ABE to perform --checkout all, using the same command line as the first invocation of ABE. This also generates a manifest. . compare the two manifests for equivalence. Since the repos used in the build may be updated between the two runs of ABE, the *_revision fields in the manifests are ignored for the comparison. These tests may fail when the manifest version is updated, or when the ABE commit changes the latest= field in config/*.conf. In such cases, manual review of the results will be required. Change-Id: Ib0aaa630df400ffe302ca18f36e3be39ce16a599
2016-11-15Set ABE_TESTS_PATH before changing directory.Charles Baylis
If test-checkout.sh is invoked via a relative path, then ABE_TESTS_PATH is set relative to the checkout-test directory rather than the PWD when test-checkout.sh was invoked. Therefore set ABE_TESTS_PATH before the cd. Change-Id: I1e2df38478551e3edb0cfa45286cb530805e8f47
2016-11-14Merge "Validate manifestid field for version 1.3 manifests."Charles Baylis
2016-11-14Validate manifestid field for version 1.3 manifests.Charles Baylis
For manifests with a version 1.3 or newer, require the presence of a manifestid field and check that its value is correct. Change-Id: I5b01e9359d96fe8b732cd2fab65a0b0ff39e0f5f
2016-11-14Merge "Clone abe under checkout-test directory."Charles Baylis
2016-11-14Clone abe under checkout-test directory.Charles Baylis
If --abe-path is not specified, test-checkout.sh will clone the abe repo and test the master branch. This clone was incorrectly placed outside the working directory. Change-Id: I20c1caa8f120dc21c6ca70b0996500bbacc6335c
2016-11-08Avoid warning on missing filespec field.Charles Baylis
If filespec is missing, arbitrarily treat as component as a tar file, and therefore expect md5sum field too. Change-Id: Ic2dbdb355a131bf3219731fff01f4e82be8c50a5
2016-11-06Add manifest validation to checkout tests.Charles Baylis
This patch validates the manifests generated by each checkout test, except for the native target test, where ABE the validator rejects the manifest because gdbserver is not present. This is because ABE does not build gdbserver for native builds, but it is not possible to determine from the manifest that it describes a native build. This implements TCWG-891 (except for native builds) Change-Id: I91847d4f103b43d9cf612d94455207ae5f42481a
2016-11-06Add compare_manifests.pl.Charles Baylis
This script normalizes and compares two manifests. This is part of TCWG-890. Change-Id: I4a18f4302973e1b37ceef059da82a3376016215f
2016-11-06Add --output-normalized option.Charles Baylis
Output normalized version of the manifest, containing only data fields in a predictable order. This is part of TCWG-890. Change-Id: I03fe3382cdffd8509ae35a05960bad384703659f
2016-11-06Tie error handling to manifest hash.Charles Baylis
Move manifest parser state into $manifest->{ctx}. This cleans up use of $errors, $line and $line_no. This is part of TCWG-890. Change-Id: I4a661178885852c238fc5d73ea132de9b98ea904
2016-11-06Remove global variables used to store parsed manifest.Charles Baylis
parse_manifest() now returns a reference to a hash which describes all the fields in a manifest. This means that the parser can be used to parse multiple manifests in one script. This is part of TCWG-890. Change-Id: I29282d8df99b0ccd86e61f67fe282bffe7444f93
2016-11-06Put manifest parser into separate functionCharles Baylis
This starts a process of making the manifest parser usable from other scripts. This is part of TCWG-890. Change-Id: I11c960ea100af743270fd198f7450452aa6032b7
2016-11-04Make own clone of abe.git if --abe-path is not set.Charles Baylis
Change-Id: Ic7b47e8c5c18fc17ce931e30843e446963c13513
2016-11-04Remove gdbserver-mismatch.sh fileCharles Baylis
Change-Id: I63ce83832d0c4d2266fd5c983ba32b7ba001f333
2016-11-04Add .gitreview fileCharles Baylis
2016-10-11Do checkout test for gcc4_9 configdir.Charles Baylis
Support for --extraconfigdir .../config/gcc4_9 was added to abe, so add that to test-checkout.sh.
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-06 15:08:46 +0000