range-op-float: Fix up -ffinite-math-only range extension and don't extend...

range-op-float: Fix up -ffinite-math-only range extension and don't extend into infinities [PR109008]

The following patch does two things (both related to range extension
around the boundaries).

The first part (in the 2 real_isfinite blocks) is to make the ranges
narrower when the old boundaries are minimum and/or maximum representable
finite number.  In that case frange_nextafter gives -Inf or +Inf,
but then the resulting computed reverse range is very far from the actually
needed range, usually extends up to infinity or could even result in NaNs.
While infinities are really the next representable numbers in the
corresponding mode, REAL_VALUE_TYPE is actually a type with wider range
for exponent and 160 bit precision, so the patch instead uses
nextafter number in a hypothetical floating point format with the same
mantissa precision but wider range of exponents.  This significantly
improves the actual ranges of the reverse operations, while still making
them conservatively correct.

The second part is a fix for miscompilation of the new testcase below.
For -ffinite-math-only, without this patch we extend the minimum and/or
maximum representable finite number to -Inf or +Inf, with the patch to
some number outside of the normal exponent range of the mode, but then
we use set which canonicalizes it and turns the boundaries back to
the minimum and/or maximum representable finite numbers, but because
in say [__DBL_MAX__, __DBL_MAX__] = op1 + [__DBL_MAX__, __DBL_MAX__]
op1 can be larger than 0, up to the largest number which rounds to even
down back to __DBL_MAX__ and there are still no infinities involved,
it needs to work even with -ffinite-math-only.  So, we really need to
widen the lhs range a little bit even in that case.  The patch does
that through temporarily clearing -ffinite-math-only, such that the
value with infinities or the outside of bounds values passes the
setting and verification (the VR_VARYING case is needed because
we get ICEs otherwise, but when lhs is VR_VARYING in -ffast-math,
i.e. minimum to maximum representable finite and both signs of NaN,
then set does all we need, we don't need to or in a NaN range).
We don't really later use the range in a way that would become a problem
that it is wider than varying, we actually just perform maths on the
two boundaries.

As I said in the PR, this doesn't fix the !MODE_HAS_INFINITIES case,
I believe we actually need to treat the boundary values as infinities
in that case because they (probably) work like that, but it is unclear
if it is just the reverse operation lhs widening that is a problem there,
or whether it is a general problem.  I have zero experience with
floating points without infinities (PDP11, some ARM half type?,
what else?).

2023-03-10  Jakub Jelinek  <jakub@redhat.com>

	PR tree-optimization/109008
	* range-op-float.cc (float_widen_lhs_range): If lb is
	minimum representable finite number or ub is maximum
	representable finite number, instead of widening it to
	-inf or inf widen it to negative or positive 0x0.8p+(EMAX+1).
	Temporarily clear flag_finite_math_only when canonicalizing
	the widened range.

	* gcc.dg/pr109008.c: New test.