ssa-math-opts, i386: Handle most unordered values rather than just 2 [PR116896]

On Mon, Oct 07, 2024 at 10:32:57AM +0200, Richard Biener wrote:
> > They are implementation defined, -1, 0, 1, 2 is defined by libstdc++:
> >     using type = signed char;
> >     enum class _Ord : type { equivalent = 0, less = -1, greater = 1 };
> >     enum class _Ncmp : type { _Unordered = 2 };
> > https://eel.is/c++draft/cmp#categories.pre-1 documents them as
> > enum class ord { equal = 0, equivalent = equal, less = -1, greater = 1 }; // exposition only
> > enum class ncmp { unordered = -127 };                                     // exposition only
> > and now looking at it, LLVM's libc++ takes that literally and uses
> > -1, 0, 1, -127.  One can't use <=> operator without including <compare>
> > which provides the enums, so I think if all we care about is libstdc++,
> > then just hardcoding -1, 0, 1, 2 is fine, if we want to also optimize
> > libc++ when used with gcc, we could support -1, 0, 1, -127 as another
> > option.
> > Supporting arbitrary 4 values doesn't make sense, at least on x86 the
> > only reason to do the conversion to int in an optab is a good sequence
> > to turn the flag comparisons to -1, 0, 1.  So, either we do nothing
> > more than the patch, or add handle both 2 and -127 for unordered,
> > or add support for arbitrary value for the unordered case except
> > -1, 0, 1 (then -1 could mean signed int, 1 unsigned int, 0 do the jumps
> > and any other value what should be returned for unordered.

Here is an incremental patch which adds support for (almost) arbitrary
unordered constant value.  It changes the .SPACESHIP and spaceship<mode>4
optab conventions, so 0 means use branches, floating point, -1, 0, 1, 2
results consumed by tree-ssa-math-opts.cc emitted comparisons, -1
means signed int comparisons, -1, 0, 1 results, 1 means unsigned int
comparisons, -1, 0, 1 results, and for constant other than -1, 0, 1
which fit into [-128, 127] converted to the PHI type are otherwise
specified as the last argument (then it is -1, 0, 1, C results).

2024-10-08  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/116896
	* tree-ssa-math-opts.cc (optimize_spaceship): Handle unordered values
	other than 2, but they still need to be signed char range possibly
	converted to the PHI result and can't be in [-1, 1] range.  Use
	last .SPACESHIP argument of 1 for unsigned int comparisons, -1 for
	signed int, 0 for floating point branches and any other for floating
	point with that value as unordered.
	* config/i386/i386-expand.cc (ix86_expand_fp_spaceship): Use op2 rather
	const2_rtx if op2 is not const0_rtx for unordered result.
	(ix86_expand_int_spaceship): Change INTVAL (op2) == 1 tests to
	INTVAL (op2) != -1.
	* doc/md.texi (spaceship@var{m}4): Document the above changes.

	* gcc.target/i386/pr116896.c: New test.