summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-06-28Handle all combinations of status changes.all-combosChristophe Lyon
Change-Id: I8ad095a0a8544ac33a344fe1aa3750653d57b5e7
2016-06-28compare_dg_tests.pl: Reorder initializations for easier readability.Christophe Lyon
Change-Id: I1d6572f73ebb4cef38b3da140643e0afbdaa3edc
2016-06-28compare_dg_tests.pl: Reorder status change handlers.Christophe Lyon
Rename NEW_PASSES into PASS_APPEARS for consistency. Change-Id: Ifa31438b1c997c181e9d7fddc1d09c400e03cfa8
2016-06-28compare_dg_tests.pl: Make status change handler names more uniform.Christophe Lyon
Change-Id: I992555d21ab4406be20d07f3eccb9fb739a26979
2016-05-26unstable-tests.txt: Remove gcc.dg/cpp/_Pragma3.cChristophe Lyon
This has been fixed by calling contrib/gcc_update in abe before starting the build. Change-Id: If257e255608a439419e14b63e37be3768dabd1d6
2016-05-01unstable-tests.txt: Remove tests with stderr/stdout race.Christophe Lyon
These tests (and a few more) have been fixed in trunk, and we have backported the fix in our branch: the results should be stable now. Change-Id: Ie351fdb6356189361439c030b8096aef00b71ebd
2016-04-14compare_dg_tests.pl: Do not round pass ratio...Christophe Lyon
...when printing the diagnostic. This is to avoid printing a rounded ratio equal to the threshold and exiting with an error: this is confusing for the user. Change-Id: I4e60a00caa2d8a20c11a3bb6af91b7e03a8c039b
2016-04-14compare_dg_tests.pl: Avoid division by zero in case no test was actually ↵Christophe Lyon
executed. Change-Id: I38f7de089d7a5db9ea161024d29f84d3e902eeb0
2016-04-05Print new UNTESTED, UNRESOLVED and UNSUPPORTED tests in the report.Christophe Lyon
Such new tests used to be omitted from the report, now: - UNRESOLVED are shown as regressions - UNTESTED and UNSUPPORTED are shown as "to be checked" Change-Id: Ic715b7a1e52069746ade6cec376f02a737cce0a5
2016-03-07Merge "compare_dg_tests.pl: Print an error instead of a warning when pass ↵Yvan Roux
ratio is too low."
2016-03-04compare_dg_tests.pl: Print an error instead of a warningChristophe Lyon
when pass ratio is too low. Indeed, this results in an error return-code, and printing a warning is confusing. Change-Id: Ibd2b84173a79de39334f4b67b55513b893ba467d
2016-02-17Add unstable tests from testing of current gcc:masterMaxim Kuvyrkov
Change-Id: I65d31d871629ef4e645d969ddb58a927c9261bd4
2016-02-08Update and clean gfortran IO unstable testsYvan Roux
Change-Id: If6fb32442c6188031979daa7d6d13f12fcc96229
2016-02-04Merge "Remove useless reports:"Christophe Lyon
2016-02-04Remove useless reports:Christophe Lyon
- report1.html was meant to carry the logs, but they are now in report0.html too - report*.xml are useless as long as our Jenkins does not have the Summary Display plugin, which is able to parse XML - rename report0.html into report.html Change-Id: I234a4f1c1d4c28eb13ef1363748cba1c1bfa559c
2016-02-04Print hostname and cwd in the diff-* files.Christophe Lyon
This is to help finding the full logs when needed. Change-Id: I514bee440a910106ab55a920803e30b0552cca52
2016-02-04Warn if all execution tests either passed or failed.Christophe Lyon
This probably indicates that the execution engine is not configured correctly. Return an error if no execution test passed. Change-Id: I4f86ddd9d783fe7a089a32f7ff623ef90b7efaf1
2016-02-04Warn if all execution tests either passed or failed.Christophe Lyon
This probably indicates that the execution engine is not configured correctly. Return an error if no execution test passed. Change-Id: I4f86ddd9d783fe7a089a32f7ff623ef90b7efaf1
2016-02-02Compute pass ratio as a sanity check.Christophe Lyon
Warn if it is below 0.95. Change-Id: Id8c1ad8692020f77cfefbda9311ebdba19000317
2016-01-28Merge "compare_jobs.sh: Fix HTML."Christophe Lyon
2016-01-28Merge "compare_dg_tests.pl: Add support for all possible status changes."Christophe Lyon
2016-01-25compare_tests: Add go results in the analysis when presents.Yvan Roux
Change-Id: Idb1fb029074e22798dd1218f765e7c01830ddf2c
2016-01-18compare_jobs.sh: Fix HTML.Christophe Lyon
So that the browsers render the colors correctly. Change-Id: Iab79c96c183d685f0b6c7fab8ab557704408e7a7
2016-01-12Merge "compare_tests: Fix usage and exit codes. compare_jobs.sh: Handle the ↵Yvan Roux
new return codes."
2016-01-12compare_jobs.sh: Concatenate all diff-* reports after the main report table.Christophe Lyon
Change-Id: Ie598bde0215fec06f7036fe48ffc65919ee58d30
2016-01-12compare_tests: Fix usage and exit codes.Christophe Lyon
compare_jobs.sh: Handle the new return codes. compare_test has now 5 exit codes: 0: no change 1: improvements 2: regressions 3: no logs in common, cannot compare 4: extra logs in either previous or current Exit codes 2, 3 and 4 are reported as failures by compare_jobs.sh, with different error messages in the report table. Change-Id: I7313ed38cd4cbb5ab1c6795391a6977f3576562a
2016-01-07compare_jobs.sh: Exit with status=1 in case of regression or build failure.Christophe Lyon
Change-Id: I4542116d9903ce70a4df17280e63af8690bf28f1
2016-01-07compare_dg_tests.pl: Add support for all possible status changes.Christophe Lyon
The ouput is improved to report more combinations, that were previously silently ommitted. Other transitions are now reported correctly. compare_tests now removes srcdir path in testcase names to avoid reporting differences. Finally, a new testsuite directory contains a Makefile, 2 sample inputs and 1 expected result. Change-Id: I67e454b198d4d80c899bd4da47924350549bae63
2015-12-15Merge "compare_dg_tests.pl: Do not count unstable tests when deciding if ↵Yvan Roux
there were improvements."
2015-12-15Merge "compare_dg_tests.pl: Fix pattern matching not to be greedy."Yvan Roux
2015-12-08compare_dg_tests.pl: Do not count unstable testsChristophe Lyon
when deciding if there were improvements. Still show them in the report table. Change-Id: I7e3604ae24f80213154423eb12d019d2cfe7ca2f
2015-12-07libstdc++: Add thread related testcase for arm-sim.Yvan Roux
Change-Id: If7245f26b1882858fdc6d6097a83e706d08e1ad4
2015-12-06Merge "Bug #1919: Make it clear when logs are missing."Maxim Kuvyrkov
2015-12-04compare_dg_tests.pl: Fix pattern matching not to be greedy.Christophe Lyon
This allows for ':' to be present in the test "name". This is the reason why arm-sim:gcc.dg/tree-prof/time-profiler-2.c scan-ipa-dump-times profile "Read tp_first_run: 0" 2 did not work. Change-Id: I8160ad3ec6adfe038d22622923cd5dc6962d6860
2015-12-04Use only testcase filename for time-profiler-2.c.Yvan Roux
Change-Id: I3720f9eb2a41c5b3ac824e1e5316a913945d99f3
2015-12-03Add thread related testcase for arm-sim in gcc list.Yvan Roux
Change-Id: I9e62708f2b8d0a1011784fc0d04d3cbd5862d7f6
2015-12-03Bug #1919: Make it clear when logs are missing.Christophe Lyon
In case a build failed, print a message in the report, instead of nothing. This will hopefully make it easier to detect build failures. For HTML reports, report as BUILDFAILED. This applies when either the reference or the current build failed. Change-Id: I74eab7ddd51fecb7066bccb1b49cba0ae00adef6
2015-12-01Merge "For HTML/XML reports, show improvements as BETTER (green)."Christophe Lyon
2015-12-01Update gfortran IO / EOF unstable testsMaxim Kuvyrkov
Some of these will be fixed in https://review.linaro.org/#/c/9360/ Change-Id: Ia00a7bd6374f70c2047e8a19a2d5a79d50380a8c
2015-11-26Add thread related testcases for arm-sim in libstdc++ list.Yvan Roux
Change-Id: I3dfcdc4e6d5b4cf58a407207bd821bd478f82df4
2015-11-20For HTML/XML reports, show improvements as BETTER (green).Christophe Lyon
No difference is reported as PASSED (light green), and regressions as FAILED (red). Change-Id: I9d37bcab237701edca9310847a3f69f86b07aa85
2015-11-20The list of unstable tests is not in a different repo anymore.Christophe Lyon
2015-11-18Initial version of the comparison scripts and list of unstable tests.Christophe Lyon
2015-11-18Add .gitreview fileChristophe Lyon
2015-11-18Initial empty repositoryChristophe Lyon
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 18:45:57 +0000