The warnings:
contrib/testsuite-management/validate_failures.py:65: SyntaxWarning: invalid escape sequence '\s'
  _VALID_TEST_RESULTS_REX = re.compile('(%s):\s*(\S+)\s*(.*)'
contrib/testsuite-management/validate_failures.py:77: SyntaxWarning: invalid escape sequence '\.'
  _EXP_LINE_REX = re.compile('^Running (?:.*:)?(.*) \.\.\.\n')

contrib/ChangeLog:
	* testsuite-management/validate_failures.py: Change re.compile()
	function arguments to Python raw strings.

When parsing a summary or manifest file, if we're not either after a tool
line (e.g. "=== gdb tests ===") or before a summary line (e.g.,
"=== gdb Summary ===") then the current line can't be a valid result line
so ignore it.

This addresses a problem we're seeing when running the GDB testsuite in
our CI environment where it produces a valid summary file, but then after
the "=== gdb Summary ===" section it outputs a series of Tcl errors that
match _VALID_TEST_RESULTS_REX and thus confuse the parsing logic:

05: 14:32 .sum file seems to be broken: tool="None", exp="None", summary_line="ERROR: -------------------------------------------"
05: 14:32 Traceback (most recent call last):
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 706, in <module>
05: 14:32     retval = Main(sys.argv)
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 697, in Main
05: 14:32     retval = CheckExpectedResults()
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 572, in CheckExpectedResults
05: 14:32     actual = GetResults(sum_files)
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 447, in GetResults
05: 14:32     build_results.update(ParseSummary(sum_fname))
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 389, in ParseSummary
05: 14:32     result = result_set.MakeTestResult(line, ordinal)
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 236, in MakeTestResult
05: 14:32     return TestResult(summary_line, ordinal,
05: 14:32   File "/path/to/gcc/contrib/testsuite-management/validate_failures.py", line 148, in __init__
05: 14:32     raise

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (IsInterestingResult):
	Add result_set argument and use it.  Adjust callers.

This patch simplifies comparison of results that have filesystem
paths.  E.g., (assuming different values of <N>):
<cut>
Running /home/user/gcc-N/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp ...
ERROR: tcl error sourcing /home/user/gcc-N/gcc/testsuite/gcc.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp.
</cut>

We add "--srcpath <regex>", option, and set it by default to
"[^ ]+/testsuite/", which works well for all components of the GNU
Toolchain.  We then remove substrings matching <regex> from paths of
.exp files and from occasional "ERROR:" results.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (TestResult,)
	(ParseManifestWorker, ParseSummary, Main): Handle new option
	"--srcpath <regex>".

This option sets "today" date to compare expiration entries against.
Setting expiration date into the future allows re-detection of flaky
tests and creating fresh entries for them before the current flaky
entries expire.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (TestResult): Update.
	(Main): Handle new option "--expiry_date YYYYMMDD".

This option is used to detect flaky tests that FAILed in the clean
build (or manifest), but PASSed in the current build (or manifest).

The option inverts output logic similar to what "-v/--invert-match"
does for grep.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (ResultSet.update,)
	(ResultSet.HasTestsuite): New methods.
	(GetResults): Update.
	(ParseSummary, CompareResults, PerformComparison, Main): Handle new
	option --invert_match.

- Print message in case of broken sum file error.
- Print error messages to stderr.  The script's stdout is, usually,
  redirected to a file, and error messages shouldn't go there.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (TestResult): Improve error
	output.

This makes it easier to extract the $tool:$exp pair when iterating
over failures/flaky tests, which, in turn, simplifies re-running
testsuite parts that have unexpected failures or passes.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (_EXP_LINE_FORMAT,)
	(_EXP_LINE_REX, ResultSet): Support "$tool:" prefix in exp names.

... in the results.  Python exits with code "1" on exceptions and
internal errors, which we use to detect failure to parse results.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (Main): Use exit code "2"
	to indicate regression.

Before this patch we would identify malformed line
"UNRESOLVEDTest run by tcwg-buildslave on Mon Aug 23 10:17:50 2021"
as an interesting result, only to fail in TestResult:__init__ due
to missing ":" after UNRESOLVED.

This patch makes all places that parse result lines use a single
compiled regex.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (_VALID_TEST_RESULTS_REX):
	Update.
	(TestResult): Use _VALID_TEST_RESULTS_REX.

... to control validate_failures.py output

contrib/ChangeLog:

	* testsuite-management/validate_failures.py: Add more verbosity levels.

... and don't require a valid build directory when no data from it
is necessary.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py: Simplify GetManifestPath().

This allows comparison of two build directories with a manifest
listing known flaky tests on the side.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (GetResults): Update.
	(CompareBuilds): Read in manifest.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py (ParseManifestWorker):
	Support expiry attributes in manifests.
	(ParseSummary): Add a comment.

This patch adds tracking of current testsuite "tool" and "exp"
to the processing of .sum files.  This avoids aliasing between
tests from different testsuites with same name+description.

E.g., this is necessary for testsuite/c-c++-common, which is ran
for both gcc and g++ "tools".

This patch changes manifest format from ...
<cut>
FAIL: gcc_test
FAIL: g++_test
</cut>
... to ...
<cut>
		=== gcc tests ===
Running gcc/foo.exp ...
FAIL: gcc_test
		=== gcc Summary ==
		=== g++ tests ===
Running g++/bar.exp ...
FAIL: g++_test
		=== g++ Summary ==
</cut>.

The new format uses same formatting as DejaGnu's .sum files
to specify which "tool" and "exp" the test belongs to.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py: Avoid testsuite
	aliasing.

contrib/ChangeLog:

	* testsuite-management/validate_failures.py: 2to3

validate_failures.py: Fix handling of summary lines with "|" characters or empty description fields.

* testsuite-management/validate_failures.py: Fix handling of
summary lines with "|" characters or empty description fields.

From-SVN: r199765

From-SVN: r198074

	* validate_failures.py: Loosen check for build directory.
	State what failed if we couldn't find the source tree or
	the target triplet.

From-SVN: r198064

From-SVN: r195819

When using validate_failures.py with --manifest and --results, we
don't need a GCC build directory.  This is useful when using the
validator outside of the build tree.  We were insisting on finding
a valid build tree regardless of those options.

Tested on x86_64.  Committed to trunk.

	* testsuite-management/validate_failures.py: Update
	Copyright years.
	Request contributions not to use Python features newer
	than 2.4.
	(GetBuildData): If this is not a build directory,
	emit an error only if --results or --manifest are missing.

From-SVN: r195817

2013-02-06  Bernhard Reutner-Fischer  <aldot@gcc.gnu.org>

	* testsuite-management/validate_failures.py
	(IsInterestingResult): Fix performance regression

From-SVN: r195811

2012-12-01  Bernhard Reutner-Fischer  <aldot@gcc.gnu.org>

	* testsuite-management/validate_failures.py
	(IsInterestingResult): Only strip line a second time if we did split.
	Rephrase return statement while at it.
	(CollectSumFiles): Also ignore .git directory.

From-SVN: r194182

An earlier patch had made the command line options a global variable
_OPTIONS, but it had not renamed all the uses of the old options argument.

	* testsuite-management/validate_failures.py: Fix stale
	use of 'options'.

From-SVN: r194092

	* testsuite-management/validate_failures.py: Add support for @include,
	@remove directives in manifest files.

From-SVN: r194008

	* testsuite-management/validate_failures.py: Add function
	GetManifestPath.  New global _MANIFEST_SUBDIR.

From-SVN: r194007

	* testsuite-management/validate_failures.py: Remove pass/fail
	indicator from result of GetBuildData.

From-SVN: r193968

	* testsuite-management/validate_failures.py: Store options in global
	variable _OPTIONS.

From-SVN: r193967

	* testsuite-management/validate_failures.py: Rename variable
	manifest_name to manifest_path everywhere.

From-SVN: r193965

	* testsuite-management/validate_failures.py (CompareBuilds): Pass
	options.results to GetSumFiles for clean build.

From-SVN: r193952

	* testsuite-management/validate_failures.py: Use <target_alias> instead
	of <target>.  Minor whitespace changes.

From-SVN: r193904

	* testsuite-management/validate_failures.py: Record ordinal with
	TestResult.

From-SVN: r193903

This is useful when you have two builds of the same compiler.  One with
your changes.  The other one, a clean build at the same revision.
Instead of using a manifest file, --clean_build will compare the
results it gather from the patched build against those it gathers from
the clean build.

Usage

$ cd /top/of/patched/gcc/bld
$ validate_failures.py --clean_build=clean/bld-gcc
Source directory: /usr/local/google/home/dnovillo/gcc/trunk
Build target:     x86_64-unknown-linux-gnu
Getting actual results from build directory .
        ./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.sum
        ./x86_64-unknown-linux-gnu/libffi/testsuite/libffi.sum
        ./x86_64-unknown-linux-gnu/libgomp/testsuite/libgomp.sum
        ./x86_64-unknown-linux-gnu/libgo/libgo.sum
        ./x86_64-unknown-linux-gnu/boehm-gc/testsuite/boehm-gc.sum
        ./x86_64-unknown-linux-gnu/libatomic/testsuite/libatomic.sum
        ./x86_64-unknown-linux-gnu/libmudflap/testsuite/libmudflap.sum
        ./x86_64-unknown-linux-gnu/libitm/testsuite/libitm.sum
        ./x86_64-unknown-linux-gnu/libjava/testsuite/libjava.sum
        ./gcc/testsuite/g++/g++.sum
        ./gcc/testsuite/gnat/gnat.sum
        ./gcc/testsuite/ada/acats/acats.sum
        ./gcc/testsuite/gcc/gcc.sum
        ./gcc/testsuite/gfortran/gfortran.sum
        ./gcc/testsuite/obj-c++/obj-c++.sum
        ./gcc/testsuite/go/go.sum
        ./gcc/testsuite/objc/objc.sum
Getting actual results from build directory clean/bld-gcc
        clean/bld-gcc/x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libffi/testsuite/libffi.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libgomp/testsuite/libgomp.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libgo/libgo.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/boehm-gc/testsuite/boehm-gc.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libatomic/testsuite/libatomic.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libmudflap/testsuite/libmudflap.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libitm/testsuite/libitm.sum
        clean/bld-gcc/x86_64-unknown-linux-gnu/libjava/testsuite/libjava.sum
        clean/bld-gcc/gcc/testsuite/g++/g++.sum
        clean/bld-gcc/gcc/testsuite/gnat/gnat.sum
        clean/bld-gcc/gcc/testsuite/ada/acats/acats.sum
        clean/bld-gcc/gcc/testsuite/gcc/gcc.sum
        clean/bld-gcc/gcc/testsuite/gfortran/gfortran.sum
        clean/bld-gcc/gcc/testsuite/obj-c++/obj-c++.sum
        clean/bld-gcc/gcc/testsuite/go/go.sum
        clean/bld-gcc/gcc/testsuite/objc/objc.sum

SUCCESS: No unexpected failures.

2012-11-02  Diego Novillo  <dnovillo@google.com>

	* testsuite-management/validate_failures.py: Add option
	--clean_build to compare test results against another
	build.

From-SVN: r193105

	* testsuite-management/validate_failures.py: Fix parsing
	of summary lines.

From-SVN: r193039

	* testsuite-management/validate_failures.py (ExpirationDate): Tidy
	comment.

From-SVN: r190354

	* testsuite-management/validate_failures.py (ParseSummary): Fix
	comment.

From-SVN: r190352

I noticed recently that while the validator was accepting the
'expire=YYYYMMDD' attribute, it was not actually doing anything with
it.

This patch fixes the oversight.

2012-08-13  Diego Novillo  <dnovillo@google.com>

	* testsuite-management/validate_failures.py: Import datetime.
	(TestResult.ExpirationDate): New.
	(TestResult.HasExpired): New.
	(ParseSummary): Call it.  If it returns True, warn that the
	expected failure has expired and do not add it to the set of
	expected results.
	(GetResults): Clarify documentation.

From-SVN: r190351

Some of the hosts were we run this script are still using Python 2.4.
This patch replaces the use of 'with ... as ...' to avoid syntax errors.

2012-07-26   Diego Novillo  <dnovillo@google.com>

	* testsuite-management/validate_failures.py: Do not use
	'with ... as ...' constructs.

From-SVN: r189893