This patch adds similar optimizations to the C++ FE as have been
implemented earlier in the C FE.
The libcpp hunk enables use of CPP_EMBED token even for C++, not just
C; the preprocessor guarantees there is always a CPP_NUMBER CPP_COMMA
before CPP_EMBED and CPP_COMMA CPP_NUMBER after it which simplifies
parsing (unless #embed is more than 2GB, in that case it could be
CPP_NUMBER CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED
CPP_COMMA CPP_NUMBER etc. with each CPP_EMBED covering at most INT_MAX
bytes).
Similarly to the C patch, this patch parses it into RAW_DATA_CST tree
in the braced initializers (and from there peels into INTEGER_CSTs unless
it is an initializer of an std::byte array or integral array with CHAR_BIT
element precision), parses CPP_EMBED in cp_parser_expression into just
the last INTEGER_CST in it because I think users don't need millions of
-Wunused-value warnings because they did useless
  int a = (
  #embed "megabyte.dat"
  );
and so most of the inner INTEGER_CSTs would be there just for the warning,
and in the rest of contexts like template argument list, function argument
list, attribute argument list, ...) parse it into a sequence of INTEGER_CSTs
(I wrote a range/iterator classes to simplify that).

My dumb
cat embed-11.c
constexpr unsigned char a[] = {
  #embed "cc1plus"
};
const unsigned char *b = a;
testcase where cc1plus is 492329008 bytes long when configured
--enable-checking=yes,rtl,extra against recent binutils with .base64 gas
support results in:
time ./xg++ -B ./ -S -O2 embed-11.c

real    0m4.350s
user    0m2.427s
sys     0m0.830s
time ./xg++ -B ./ -c -O2 embed-11.c

real    0m6.932s
user    0m6.034s
sys     0m0.888s
(compared to running out of memory or very long compilation).
On a shorter inclusion,
cat embed-12.c
constexpr unsigned char a[] = {
  #embed "xg++"
};
const unsigned char *b = a;
where xg++ is 15225904 bytes long, this takes using GCC with the #embed
patchset except for this patch:
time ~/src/gcc/obj36/gcc/xg++ -B ~/src/gcc/obj36/gcc/ -S -O2 embed-12.c

real    0m33.190s
user    0m32.327s
sys     0m0.790s
and with this patch:
time ./xg++ -B ./ -S -O2 embed-12.c

real    0m0.118s
user    0m0.090s
sys     0m0.028s

The patch doesn't change anything on what the first patch in the series
introduces even for C++, namely that #embed is expanded (actually or as if)
into a sequence of literals like
127,69,76,70,2,1,1,3,0,0,0,0,0,0,0,0,2,0,62,0,1,0,0,0,80,211,64,0,0,0,0,0,64,0,0,0,0,0,0,0,8,253
and so each element has int type.
That is how I believe it is in C23, and the different versions of the
C++ P1967 paper specified there some casts, P1967R12 in particular
"Otherwise, the integral constant expression is the value of std::fgetc’s return is cast
to unsigned char."
but please see
https://github.com/llvm/llvm-project/pull/97274#issuecomment-2230929277
comment and whether we really want the preprocessor to preprocess it for
C++ as (or as-if)
static_cast<unsigned char>(127),static_cast<unsigned char>(69),static_cast<unsigned char>(76),static_cast<unsigned char>(70),static_cast<unsigned char>(2),...
i.e. 9 tokens per byte rather than 2, or
(unsigned char)127,(unsigned char)69,...
or
((unsigned char)127),((unsigned char)69),...
etc.
Without a literal suffix for unsigned char constant literals it is horrible,
plus the incompatibility between C and C++.  Sure, we could use the magic
form more often for C++ to save the size and do the 9 or how many tokens
form only for the boundary constants and use #embed "." __gnu__::__base64__("...")
for what is in between if there are at least 2 tokens inside of it.
E.g. (unsigned char)127 vs. static_cast<unsigned char>(127) behaves
differently if there is constexpr long long p[] = { ... };
...
  #embed __FILE__
[p]

2024-12-06  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* files.cc (finish_embed): Use CPP_EMBED even for C++.
gcc/
	* tree.h (RAW_DATA_UCHAR_ELT, RAW_DATA_SCHAR_ELT): Define.
gcc/cp/ChangeLog:
	* cp-tree.h (class raw_data_iterator): New type.
	(class raw_data_range): New type.
	* parser.cc (cp_parser_postfix_open_square_expression): Handle
	parsing of CPP_EMBED.
	(cp_parser_parenthesized_expression_list): Likewise.  Use
	cp_lexer_next_token_is.
	(cp_parser_expression): Handle parsing of CPP_EMBED.
	(cp_parser_template_argument_list): Likewise.
	(cp_parser_initializer_list): Likewise.
	(cp_parser_oacc_clause_tile): Likewise.
	(cp_parser_omp_tile_sizes): Likewise.
	* pt.cc (tsubst_expr): Handle RAW_DATA_CST.
	* constexpr.cc (reduced_constant_expression_p): Likewise.
	(raw_data_cst_elt): New function.
	(find_array_ctor_elt): Handle RAW_DATA_CST.
	(cxx_eval_array_reference): Likewise.
	* typeck2.cc (digest_init_r): Emit -Wnarrowing and/or -Wconversion
	diagnostics.
	(process_init_constructor_array): Handle RAW_DATA_CST.
	* decl.cc (maybe_deduce_size_from_array_init): Likewise.
	(is_direct_enum_init): Fail for RAW_DATA_CST.
	(cp_maybe_split_raw_data): New function.
	(consume_init): New function.
	(reshape_init_array_1): Add VECTOR_P argument.  Handle RAW_DATA_CST.
	(reshape_init_array): Adjust reshape_init_array_1 caller.
	(reshape_init_vector): Likewise.
	(reshape_init_class): Handle RAW_DATA_CST.
	(reshape_init_r): Likewise.
gcc/testsuite/
	* c-c++-common/cpp/embed-22.c: New test.
	* c-c++-common/cpp/embed-23.c: New test.
	* g++.dg/cpp/embed-4.C: New test.
	* g++.dg/cpp/embed-5.C: New test.
	* g++.dg/cpp/embed-6.C: New test.
	* g++.dg/cpp/embed-7.C: New test.
	* g++.dg/cpp/embed-8.C: New test.
	* g++.dg/cpp/embed-9.C: New test.
	* g++.dg/cpp/embed-10.C: New test.
	* g++.dg/cpp/embed-11.C: New test.
	* g++.dg/cpp/embed-12.C: New test.
	* g++.dg/cpp/embed-13.C: New test.
	* g++.dg/cpp/embed-14.C: New test.

As noted in bug 117162, C23 changed some rules on UCNs to match C++
(this was a late change agreed in the resolution to CD2 comment
US-032, implementing changes from N3124), which we need to implement.

Allow UCNs below 0xa0 outside identifiers for C, with a
pedwarn-if-pedantic before C23 (and a warning with -Wc11-c23-compat)
except for the always-allowed cases of UCNs for $ @ `.  Also as part
of that change, do not allow \u0024 in identifiers as equivalent to $
for C23.

Bootstrapped with no regressions for x86_64-pc-linux-gnu.

	PR c/117162

libcpp/
	* include/cpplib.h (struct cpp_options): Add low_ucns.
	* init.cc (struct lang_flags, lang_defaults): Add low_ucns.
	(cpp_set_lang): Set low_ucns
	* charset.cc (_cpp_valid_ucn): For C, allow UCNs below 0xa0
	outside identifiers, with a pedwarn if pedantic before C23 or a
	warning with -Wc11-c23-compat.  Do not allow \u0024 in identifiers
	for C23.

gcc/testsuite/
	* gcc.dg/cpp/c17-ucn-1.c, gcc.dg/cpp/c17-ucn-2.c,
	gcc.dg/cpp/c17-ucn-3.c, gcc.dg/cpp/c17-ucn-4.c,
	gcc.dg/cpp/c23-ucn-2.c, gcc.dg/cpp/c23-ucnid-2.c: New tests.
	* c-c++-common/cpp/delimited-escape-seq-3.c,
	c-c++-common/cpp/named-universal-char-escape-3.c,
	gcc.dg/cpp/c23-ucn-1.c, gcc.dg/cpp/c2y-delimited-escape-seq-3.c:
	Update expected messages
	* gcc.dg/cpp/ucs.c: Use -pedantic-errors.  Update expected
	messages.

No functional change intended.

gcc/analyzer/ChangeLog:
	PR c/104896
	* sm-malloc.cc: Replace "%<%s%>" with "%qs" in message wording.

gcc/c-family/ChangeLog:
	PR c/104896
	* c-lex.cc (c_common_lex_availability_macro): Replace "%<%s%>"
	with "%qs" in message wording.
	* c-opts.cc (c_common_handle_option): Likewise.
	* c-warn.cc (warn_parm_array_mismatch): Likewise.

gcc/ChangeLog:
	PR c/104896
	* common/config/ia64/ia64-common.cc (ia64_handle_option): Replace
	"%<%s%>" with "%qs" in message wording.
	* common/config/rs6000/rs6000-common.cc (rs6000_handle_option):
	Likewise.
	* config/aarch64/aarch64.cc (aarch64_validate_sls_mitigation):
	Likewise.
	(aarch64_override_options): Likewise.
	(aarch64_process_target_attr): Likewise.
	* config/arm/aarch-common.cc (aarch_validate_mbranch_protection):
	Likewise.
	* config/pru/pru.cc (pru_insert_attributes): Likewise.
	* config/riscv/riscv-target-attr.cc
	(riscv_target_attr_parser::parse_arch): Likewise.
	* omp-general.cc (oacc_verify_routine_clauses): Likewise.
	* tree-ssa-uninit.cc (maybe_warn_read_write_only): Likewise.
	(maybe_warn_pass_by_reference): Likewise.

gcc/cp/ChangeLog:
	PR c/104896
	* cvt.cc (maybe_warn_nodiscard): Replace "%<%s%>" with "%qs" in
	message wording.

gcc/fortran/ChangeLog:
	PR c/104896
	* resolve.cc (resolve_operator): Replace "%<%s%>" with "%qs" in
	message wording.

gcc/go/ChangeLog:
	PR c/104896
	* gofrontend/embed.cc (Gogo::initializer_for_embeds): Replace
	"%<%s%>" with "%qs" in message wording.
	* gofrontend/expressions.cc
	(Selector_expression::lower_method_expression): Likewise.
	* gofrontend/gogo.cc (Gogo::set_package_name): Likewise.
	(Named_object::export_named_object): Likewise.
	* gofrontend/parse.cc (Parse::struct_type): Likewise.
	(Parse::parameter_list): Likewise.

gcc/rust/ChangeLog:
	PR c/104896
	* backend/rust-compile-expr.cc
	(CompileExpr::compile_integer_literal): Replace "%<%s%>" with
	"%qs" in message wording.
	(CompileExpr::compile_float_literal): Likewise.
	* backend/rust-compile-intrinsic.cc (Intrinsics::compile):
	Likewise.
	* backend/rust-tree.cc (maybe_warn_nodiscard): Likewise.
	* checks/lints/rust-lint-scan-deadcode.h: Likewise.
	* lex/rust-lex.cc (Lexer::parse_partial_unicode_escape): Likewise.
	(Lexer::parse_raw_byte_string): Likewise.
	* lex/rust-token.cc (Token::get_str): Likewise.
	* metadata/rust-export-metadata.cc
	(PublicInterface::write_to_path): Likewise.
	* parse/rust-parse.cc
	(peculiar_fragment_match_compatible_fragment): Likewise.
	(peculiar_fragment_match_compatible): Likewise.
	* resolve/rust-ast-resolve-path.cc (ResolvePath::resolve_path):
	Likewise.
	* resolve/rust-ast-resolve-toplevel.h: Likewise.
	* resolve/rust-ast-resolve-type.cc (ResolveRelativeTypePath::go):
	Likewise.
	* rust-session-manager.cc (validate_crate_name): Likewise.
	(Session::load_extern_crate): Likewise.
	* typecheck/rust-hir-type-check-expr.cc (TypeCheckExpr::visit):
	Likewise.
	(TypeCheckExpr::resolve_fn_trait_call): Likewise.
	* typecheck/rust-hir-type-check-implitem.cc
	(TypeCheckImplItemWithTrait::visit): Likewise.
	* typecheck/rust-hir-type-check-item.cc
	(TypeCheckItem::validate_trait_impl_block): Likewise.
	* typecheck/rust-hir-type-check-struct.cc
	(TypeCheckStructExpr::visit): Likewise.
	* typecheck/rust-tyty-call.cc (TypeCheckCallExpr::visit):
	Likewise.
	* typecheck/rust-tyty.cc (BaseType::bounds_compatible): Likewise.
	* typecheck/rust-unify.cc (UnifyRules::emit_abi_mismatch):
	Likewise.
	* util/rust-attributes.cc (AttributeChecker::visit): Likewise.

libcpp/ChangeLog:
	PR c/104896
	* pch.cc (cpp_valid_state): Replace "%<%s%>" with "%qs" in message
	wording.

Signed-off-by: David Malcolm <dmalcolm@redhat.com>

I enabled include translation to header units in r15-1104-ga29f481bbcaf2b,
but it seems that patch wasn't sufficient, as any diagnostics in the main
source file would show up as coming from the header instead.

Fixed by setting buffer->file for leaving the file transition that my
previous patch made us enter.  And don't push a buffer of newlines, in this
case that messes up line numbers instead of aligning them.

libcpp/ChangeLog:

	* files.cc (_cpp_stack_file): Handle -include of header unit more
	specially.

gcc/testsuite/ChangeLog:

	* g++.dg/modules/dashinclude-1_b.C: Add an #error.
	* g++.dg/modules/dashinclude-1_a.H: Remove dg-module-do run.

The PR shows that we ICE after lexing an invalid unterminated raw string,
because lex_raw_string() pops the main buffer unexpectedly. Resolve by
handling this case the same way as for other directives.

libcpp/ChangeLog:
	PR preprocessor/117118
	* lex.cc (lex_raw_string): Treat an unterminated raw string the same
	way for a deferred pragma as is done for other directives.

gcc/testsuite/ChangeLog:
	PR preprocessor/117118
	* c-c++-common/raw-string-directive-3.c: New test.
	* c-c++-common/raw-string-directive-4.c: New test.

libcpp makes use of the cpp_buffer pfile->a_buff to store things while it is
handling macros. It uses it to store pointers (cpp_hashnode*, for macro
arguments) and cpp_macro objects. This works fine because a cpp_hashnode*
and a cpp_macro have the same alignment requirement on either 32-bit or
64-bit systems (namely, the same alignment as a pointer.)

When 64-bit location_t is enabled on a 32-bit sytem, the alignment
requirement may cease to be the same, because the alignment requirement of a
cpp_macro object changes to that of a uint64_t, which be larger than that of
a pointer. It's not the case for x86 32-bit, but for example, on sparc, a
pointer has 4-byte alignment while a uint64_t has 8. In that case,
intermixing the two within the same cpp_buffer leads to a misaligned
access. The code path that triggers this is the one in _cpp_commit_buff in
which a hash table with its own allocator (i.e. ggc) is not being used, so
it doesn't happen within the compiler itself, but it happens in the other
libcpp clients, such as genmatch.

Fix that up by ensuring _cpp_commit_buff commits a fully aligned chunk of the
buffer, so it's ready for anything it may be used for next.

Also modify CPP_ALIGN so that it guarantees to return an alignment at least
the size of location_t. Currently it returns the max of a pointer and a
double. I am not aware of any platform where a double may have smaller
alignment than a uint64_t, but it does not hurt to add location_t here to be
sure.

libcpp/ChangeLog:

	* lex.cc (_cpp_commit_buff): Make sure that the buffer is properly
	aligned for the next allocation.
	* internal.h (struct dummy): Make sure alignment is large enough for
	a location_t, just in case.

Prepare libcpp to support 64-bit location_t, without yet making
any functional changes, by adding new typedefs that enable code to be
written such that it works with any size location_t. Update the usage of
line maps within libcpp accordingly.

Subsequent patches will prepare the rest of the codebase similarly, and then
afterwards, location_t will be changed to uint64_t.

libcpp/ChangeLog:

	* include/line-map.h (line_map_uint_t): New typedef, the same type
	as location_t.
	(location_diff_t): New typedef.
	(line_map_suggested_range_bits): New constant.
	(struct maps_info_ordinary): Change member types from "unsigned int"
	to "line_map_uint_t".
	(struct maps_info_macro): Likewise.
	(struct location_adhoc_data_map): Likewise.
	(LINEMAPS_ALLOCATED): Change return type from "unsigned int" to
	"line_map_uint_t".
	(LINEMAPS_ORDINARY_ALLOCATED): Likewise.
	(LINEMAPS_MACRO_ALLOCATED): Likewise.
	(LINEMAPS_USED): Likewise.
	(LINEMAPS_ORDINARY_USED): Likewise.
	(LINEMAPS_MACRO_USED): Likewise.
	(linemap_lookup_macro_index): Likewise.
	(LINEMAPS_MAP_AT): Change argument type from "unsigned int" to
	"line_map_uint_t".
	(LINEMAPS_ORDINARY_MAP_AT): Likewise.
	(LINEMAPS_MACRO_MAP_AT): Likewise.
	(line_map_new_raw): Likewise.
	(linemap_module_restore): Likewise.
	(linemap_dump): Likewise.
	(line_table_dump): Likewise.
	(LINEMAPS_LAST_MAP): Add a linemap_assert() for safety.
	(SOURCE_COLUMN): Use a cast to ensure correctness if location_t
	becomes a 64-bit type.
	* line-map.cc (location_adhoc_data_hash): Don't truncate to 32-bit
	prematurely when hashing.
	(line_maps::get_or_create_combined_loc): Adapt types to support
	potentially 64-bit location_t. Use MAX_LOCATION_T rather than a
	hard-coded constant.
	(line_maps::get_range_from_loc): Adapt types and constants to
	support potentially 64-bit location_t.
	(line_maps::pure_location_p): Likewise.
	(line_maps::get_pure_location): Likewise.
	(line_map_new_raw): Likewise.
	(LAST_SOURCE_LINE_LOCATION): Likewise.
	(linemap_add): Likewise.
	(linemap_module_restore): Likewise.
	(linemap_line_start): Likewise.
	(linemap_position_for_column): Likewise.
	(linemap_position_for_line_and_column): Likewise.
	(linemap_position_for_loc_and_offset): Likewise.
	(linemap_ordinary_map_lookup): Likewise.
	(linemap_lookup_macro_index): Likewise.
	(linemap_dump): Likewise.
	(linemap_dump_location): Likewise.
	(linemap_get_file_highest_location): Likewise.
	(line_table_dump): Likewise.
	(linemap_compare_locations): Avoid signed int overflow in the result.
	* macro.cc (num_expanded_macros_counter): Change type of global
	variable from "unsigned int" to "line_map_uint_t".
	(num_macro_tokens_counter): Likewise.

The dependency output for header unit modules is based on the absolute
pathname of the header file, but that's not something that a makefile can
portably refer to.  This patch adds a .c++-header-unit target based on the
header name relative to an element of the include path.

libcpp/ChangeLog:

	* internal.h (_cpp_get_file_dir): Declare.
	* files.cc (_cpp_get_file_dir): New fn.
	* mkdeps.cc (make_write): Use it.

gcc/testsuite/ChangeLog:

	* g++.dg/modules/dep-4.H: New test.

The C++ modules code has a -fmodule-header (or -x c++-{user,system}-header)
option to specify looking up headers to compile to header units on the usual
include paths.  I'd like to have the same functionality for full C++20
modules such as module std, which I proposed to live on the include path at
bits/std.cc.  But this behavior doesn't seem necessarily connected to
modules, so I'm proposing a general C/C++ option to specify the behavior of
looking in the include path for the input files specified on the command
line.

Other ideas for the name of the option are very welcome.

The libcpp change is to allow -fsearch-include-path{,=user} to find files in
the current working directory, like -include.  This can be handy for a quick
compile of both std.cc and a file that imports it, e.g.

g++ -std=c++20 -fmodules -fsearch-include-path bits/std.cc importer.cc

gcc/ChangeLog:

	* doc/cppopts.texi: Document -fsearch-include-path.
	* doc/invoke.texi: Mention it for modules.

gcc/c-family/ChangeLog:

	* c.opt: Add -fsearch-include-path.
	* c-opts.cc (c_common_post_options): Handle it.

gcc/cp/ChangeLog:

	* module.cc (module_preprocess_options): Don't override it.

libcpp/ChangeLog:

	* internal.h (search_path_head): Declare.
	* files.cc (search_path_head): No longer static.
	* init.cc (cpp_read_main_file): Use it.

The following patch adds _Decimal64x type support.  Our dfp libraries (dpd &
libbid) can only handle decimal32, decimal64 and decimal128 formats and I
don't see that changing any time soon, so the following patch just hardcodes
that _Decimal64x has the same mode as _Decimal128 (but is a distinct type).
In the unlikely event some target would introduce something different that
can be of course changed with target hooks but would be an ABI change.
_Decimal128x is optional and we don't have a wider decimal type, so that
type isn't added.

2024-11-15  Jakub Jelinek  <jakub@redhat.com>

gcc/
	* tree-core.h (enum tree_index): Add TI_DFLOAT64X_TYPE.
	* tree.h (dfloat64x_type_node): Define.
	* tree.cc (build_common_tree_nodes): Initialize dfloat64x_type_node.
	* builtin-types.def (BT_DFLOAT64X): New DEF_PRIMITIVE_TYPE.
	(BT_FN_DFLOAT64X): New DEF_FUNCTION_TYPE_0.
	(BT_FN_DFLOAT64X_CONST_STRING, BT_FN_DFLOAT64X_DFLOAT64X): New
	DEF_FUNCTION_TYPE_1.
	* builtins.def (BUILT_IN_FABSD64X, BUILT_IN_INFD64X, BUILT_IN_NAND64X,
	BUILT_IN_NANSD64X): New builtins.
	* builtins.cc (expand_builtin): Handle BUILT_IN_FABSD64X.
	(fold_builtin_0): Handle BUILT_IN_INFD64X.
	(fold_builtin_1): Handle BUILT_IN_FABSD64X.
	* fold-const-call.cc (fold_const_call): Handle CFN_BUILT_IN_NAND64X
	and CFN_BUILT_IN_NANSD64X.
	* ginclude/float.h (DEC64X_MANT_DIG, DEC64X_MIN_EXP, DEC64X_MAX_EXP,
	DEC64X_MAX, DEC64X_EPSILON, DEC64X_MIN, DEC64X_TRUE_MIN,
	DEC64X_SNAN): Redefine.
gcc/c-family/
	* c-common.h (enum rid): Add RID_DFLOAT64X.
	* c-common.cc (c_global_trees): Fix comment typo.  Add
	dfloat64x_type_node.
	(c_common_nodes_and_builtins): Handle RID_DFLOAT64X.
	* c-cppbuiltin.cc (c_cpp_builtins): Call
	builtin_define_decimal_float_constants also for dfloat64x_type_node
	if non-NULL.
	* c-lex.cc (interpret_float): Handle d64x suffixes.
	* c-pretty-print.cc (pp_c_floating_constant): Print d64x suffixes
	on dfloat64x_type_node typed constants.
gcc/c/
	* c-tree.h (enum c_typespec_keyword): Add cts_dfloat64x and adjust
	comment.
	* c-parser.cc (c_keyword_starts_typename, c_token_starts_declspecs,
	c_parser_declspecs, c_parser_gnu_attribute_any_word): Handle
	RID_DFLOAT64X.
	(c_parser_postfix_expression): Handle _Decimal64x arguments in
	__builtin_tgmath.
	(warn_for_abs): Handle BUILT_IN_FABSD64X.
	* c-decl.cc (declspecs_add_type): Handle cts_dfloat64x and
	RID_DFLOAT64X.
	(finish_declspecs): Handle cts_dfloat64x.
	* c-typeck.cc (c_common_type): Handle dfloat64x_type_node.
gcc/testsuite/
	* gcc.dg/dfp/c11-decimal64x-1.c: New test.
	* gcc.dg/dfp/c11-decimal64x-2.c: New test.
	* gcc.dg/dfp/c23-decimal64x-1.c: New test.
	* gcc.dg/dfp/c23-decimal64x-2.c: New test.
	* gcc.dg/dfp/c23-decimal64x-3.c: New test.
	* gcc.dg/dfp/c23-decimal64x-4.c: New test.
libcpp/
	* expr.cc (interpret_float_suffix): Handle d64x and D64x
	suffixes, adjust comment.

C23 roughly says that {d,D}{32,64,128} floating point constant suffixes
are alternate spellings of {df,dd,dl} suffixes in annex H.

So, the following patch allows that alternate spelling.
Or is it intentional it isn't enabled and we need to do everything in
there first before trying to define __STDC_IEC_60559_DFP__?
Like add support for _Decimal32x and _Decimal64x types (including
the d32x and d64x suffixes) etc.

2024-11-13  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* expr.cc (interpret_float_suffix): Handle d32 and D32 suffixes
	for C like df, d64 and D64 like dd and d128 and D128 like
	dl.
gcc/c-family/
	* c-lex.cc (interpret_float): Subtract 3 or 4 from copylen
	rather than 2 if last character of CPP_N_DFLOAT is a digit.
gcc/testsuite/
	* gcc.dg/dfp/c11-constants-3.c: New test.
	* gcc.dg/dfp/c11-constants-4.c: New test.
	* gcc.dg/dfp/c23-constants-3.c: New test.
	* gcc.dg/dfp/c23-constants-4.c: New test.

The following patch implements the C2Y N3298 paper Introduce complex literals
by providing different (or no) diagnostics on imaginary constants (except
for integer ones).
For _DecimalN constants we don't support _Complex _DecimalN and error on any
i/j suffixes mixed with DD/DL/DF, so nothing changed there.

2024-11-13  Jakub Jelinek  <jakub@redhat.com>

	PR c/117029
libcpp/
	* include/cpplib.h (struct cpp_options): Add imaginary_constants
	member.
	* init.cc (struct lang_flags): Add imaginary_constants bitfield.
	(lang_defaults): Add column for imaginary_constants.
	(cpp_set_lang): Copy over imaginary_constants.
	* expr.cc (cpp_classify_number): Diagnose CPP_N_IMAGINARY
	non-CPP_N_FLOATING constants differently for C.
gcc/testsuite/
	* gcc.dg/cpp/pr7263-3.c: Adjust expected diagnostic wording.
	* gcc.dg/c23-imaginary-constants-1.c: New test.
	* gcc.dg/c23-imaginary-constants-2.c: New test.
	* gcc.dg/c23-imaginary-constants-3.c: New test.
	* gcc.dg/c23-imaginary-constants-4.c: New test.
	* gcc.dg/c23-imaginary-constants-5.c: New test.
	* gcc.dg/c23-imaginary-constants-6.c: New test.
	* gcc.dg/c23-imaginary-constants-7.c: New test.
	* gcc.dg/c23-imaginary-constants-8.c: New test.
	* gcc.dg/c23-imaginary-constants-9.c: New test.
	* gcc.dg/c23-imaginary-constants-10.c: New test.
	* gcc.dg/c2y-imaginary-constants-1.c: New test.
	* gcc.dg/c2y-imaginary-constants-2.c: New test.
	* gcc.dg/c2y-imaginary-constants-3.c: New test.
	* gcc.dg/c2y-imaginary-constants-4.c: New test.
	* gcc.dg/c2y-imaginary-constants-5.c: New test.
	* gcc.dg/c2y-imaginary-constants-6.c: New test.
	* gcc.dg/c2y-imaginary-constants-7.c: New test.
	* gcc.dg/c2y-imaginary-constants-8.c: New test.
	* gcc.dg/c2y-imaginary-constants-9.c: New test.
	* gcc.dg/c2y-imaginary-constants-10.c: New test.
	* gcc.dg/c2y-imaginary-constants-11.c: New test.
	* gcc.dg/c2y-imaginary-constants-12.c: New test.

This is an attempt to implement the https://wg21.link/p3034r1 paper,
but I'm afraid the wording in the paper is bad for multiple reasons.
I think I understand the intent, that the module name and partition
if any shouldn't come from macros so that they can be scanned for
without preprocessing, but on the other side doesn't want to disable
macro expansion in pp-module altogether, because e.g. the optional
attribute in module-declaration would be nice to come from macros
as which exact attribute is needed might need to be decided based on
preprocessor checks.
The paper added https://eel.is/c++draft/cpp.module#2
which uses partly the wording from https://eel.is/c++draft/cpp.module#1

The first issue I see is that using that "defined as an object-like macro"
from there means IMHO something very different in those 2 paragraphs.
As per https://eel.is/c++draft/cpp.pre#7.sentence-1 preprocessing tokens
in preprocessing directives aren't subject to macro expansion unless
otherwise stated, and so the export and module tokens aren't expanded
and so the requirement that they aren't defined as an object-like macro
makes perfect sense.  The problem with the new paragraph is that
https://eel.is/c++draft/cpp.module#3.sentence-1 says that the rest of
the tokens are macro expanded and after macro expansion none of the
tokens can be defined as an object-like macro, if they would be, they'd
be expanded to that.  So, I think either the wording needs to change
such that not all preprocessing tokens after module are macro expanded,
only those which are after the pp-module-name and if any pp-module-partition
tokens, or all tokens after module are macro expanded but none of the tokens in
pp-module-name and pp-module-partition if any must come from macro
expansion.  The patch below implements it as if the former would be
specified (but see later), so essentially scans the preprocessing tokens
after module without expansion, if the first one is an identifier, it
disables expansion for it and then if followed by . or : expects another
such identifier (again with disabled expansion), but stops after second
: is seen.

Second issue is that while the global-module-fragment start is fine, matches
the syntax of the new paragraph where the pp-tokens[opt] aren't present,
there is also private-module-fragment in the syntax where module is
followed by : private ; and in that case the colon doesn't match the
pp-module-name grammar and appears now to be invalid.  I think the
https://eel.is/c++draft/cpp.module#2
paragraph needs to change so that it allows also that pp-tokens of
a pp-module may also be : pp-tokens[opt] (and in that case, I think
the colon shouldn't come from a macro and private and/or ; can).

Third issue is that there are too many pp-tokens in
https://eel.is/c++draft/cpp.module , one is all the tokens between
module keyword and the semicolon and one is the optional extra tokens
after pp-module-partition (if any, if missing, after pp-module).
Perhaps introducing some other non-terminal would help talking about it?
So in "where the pp-tokens (if any) shall not begin with a ( preprocessing
token" it isn't obvious which pp-tokens it is talking about (my assumption
is the latter) and also whether ( can't appear there just before macro
expansion or also after expansion.  The patch expects only before expansion,
so
 #define F ();
 export module foo F
would be valid during preprocessing but obviously invalid during
compilation, but
 #define foo(n) n;
 export module foo (3)
would be invalid already during preprocessing.

The last issue applies only if the first issue is resolved to allow
expansion of tokens after : if first token, or after pp-module-partition
if present or after pp-module-name if present.  When non-preprocessing
scanner sees
 export module foo.bar:baz.qux;
it knows nothing can come from preprocessing macros and is ok, but if it
sees
 export module foo.bar:baz qux
then it can't know whether it will be
 export module foo.bar:baz;
or
 export module foo.bar:baz [[]];
or
 export module foo.bar:baz.freddy.garply;
because qux could be validly a macro, which expands to ; or [[]];
or .freddy.garply; etc.  So, either the non-preprocessing scanner would
need to note it as possible export of foo.bar:baz* module partitions
and preprocess if it needs to know the details or just compile, or if that
is not ok, the wording would need to rule out that the expansion of (the
second) pp-tokens if any can't start with . or : (colon would be only
problematic if it isn't present in the tokens before it already).
So, if e.g. defining qux above to . whatever is invalid, then the scanner
can rely it sees the whole module name and partition.

The patch below implements what is above described as the first variant
of the first issue resolution, i.e. disables expansion of as many tokens
as could be in the valid module name and module partition syntax, but
as soon as it e.g. sees two adjacent identifiers, the second one can be
macro expanded.  If it is macro expanded though, the expansion can't
start with . or :, and if it expands to nothing, tokens after it (whether
they come from macro expansion or not) can't start with . or :.
So, effectively:
 #define SEMI ;
 export module SEMI
used to be valid and isn't anymore,
 #define FOO bar
 export module FOO;
isn't valid,
 #define COLON :
 export module COLON private;
isn't valid,
 #define BAR baz
 export module foo.bar:baz.qux.BAR;
isn't valid,
 #define BAZ .qux
 export module foo BAZ;
isn't valid,
 #define FREDDY :garply
 export module foo FREDDY;
isn't valid,
while
 #define QUX [[]]
 export module foo QUX;
or
 #define GARPLY private
 module : GARPLY;
etc. is.

2024-11-01  Jakub Jelinek  <jakub@redhat.com>

	PR c++/114461
libcpp/
	* include/cpplib.h: Implement C++26 P3034R1
	- Module Declarations Shouldn’t be Macros (or more precisely
	its expected intent).
	(NO_DOT_COLON): Define.
	* internal.h (struct cpp_reader): Add diagnose_dot_colon_from_macro_p
	member.
	* lex.cc (cpp_maybe_module_directive): For pp-module, if
	module keyword is followed by CPP_NAME, ensure all CPP_NAME
	tokens possibly matching module name and module partition
	syntax aren't expanded and aren't defined as object-like macros.
	Verify first token after that doesn't start with open paren.
	If the next token after module name/partition is CPP_NAME defined
	as macro, set NO_DOT_COLON flag on it.
	* macro.cc (cpp_get_token_1): Set
	pfile->diagnose_dot_colon_from_macro_p if token to be expanded has
	NO_DOT_COLON bit set in flags.  Before returning, if
	pfile->diagnose_dot_colon_from_macro_p is true and not returning
	CPP_PADDING or CPP_COMMENT and not during macro expansion preparation,
	set pfile->diagnose_dot_colon_from_macro_p to false and diagnose
	if returning CPP_DOT or CPP_COLON.
gcc/testsuite/
	* g++.dg/modules/cpp-7.C: New test.
	* g++.dg/modules/cpp-8.C: New test.
	* g++.dg/modules/cpp-9.C: New test.
	* g++.dg/modules/cpp-10.C: New test.
	* g++.dg/modules/cpp-11.C: New test.
	* g++.dg/modules/cpp-12.C: New test.
	* g++.dg/modules/cpp-13.C: New test.
	* g++.dg/modules/cpp-14.C: New test.
	* g++.dg/modules/cpp-15.C: New test.
	* g++.dg/modules/cpp-16.C: New test.
	* g++.dg/modules/cpp-17.C: New test.
	* g++.dg/modules/cpp-18.C: New test.
	* g++.dg/modules/cpp-19.C: New test.
	* g++.dg/modules/cpp-20.C: New test.
	* g++.dg/modules/pmp-4.C: New test.
	* g++.dg/modules/pmp-5.C: New test.
	* g++.dg/modules/pmp-6.C: New test.
	* g++.dg/modules/token-6.C: New test.
	* g++.dg/modules/token-7.C: New test.
	* g++.dg/modules/token-8.C: New test.
	* g++.dg/modules/token-9.C: New test.
	* g++.dg/modules/token-10.C: New test.
	* g++.dg/modules/token-11.C: New test.
	* g++.dg/modules/token-12.C: New test.
	* g++.dg/modules/token-13.C: New test.
	* g++.dg/modules/token-14.C: New test.
	* g++.dg/modules/token-15.C: New test.
	* g++.dg/modules/token-16.C: New test.
	* g++.dg/modules/dir-only-3.C: Expect an error.
	* g++.dg/modules/dir-only-4.C: Expect an error.
	* g++.dg/modules/dir-only-5.C: New test.
	* g++.dg/modules/atom-preamble-2_a.C: In export module malcolm;
	replace malcolm with kevin.  Don't define malcolm macro.
	* g++.dg/modules/atom-preamble-4.C: Expect an error.
	* g++.dg/modules/atom-preamble-5.C: New test.

I've tried to build stage3 with
-Wleading-whitespace=blanks -Wtrailing-whitespace=blank -Wno-error=leading-whitespace=blanks -Wno-error=trailing-whitespace=blank
added to STRICT_WARN and that expectably resulted in about
2744 unique trailing whitespace warnings and 124837 leading whitespace
warnings when excluding *.md files (which obviously is in big part a
generator issue).  Others from that are generator related, I think those
need to be solved later.

The following patch just fixes up the easy case (trailing whitespace),
which could be easily automated:
for i in `find . -name \*.h -o -name \*.cc -o -name \*.c | xargs grep -l '[ 	]$' | grep -v testsuite/`; do sed -i -e 's/[ 	]*$//' $i; done
I've excluded files which I knew are obviously generated or go FE.

Is there anything else we'd want to avoid the changes?

Due to patch size, I've split it between gcc/ part
and rest (include/, libiberty/, libgcc/, libcpp/, libstdc++-v3/;
this part).

2024-10-24  Jakub Jelinek  <jakub@redhat.com>

include/
	* dyn-string.h: Remove trailing whitespace.
	* libiberty.h: Likewise.
	* xregex.h: Likewise.
	* splay-tree.h: Likewise.
	* partition.h: Likewise.
	* plugin-api.h: Likewise.
	* demangle.h: Likewise.
	* vtv-change-permission.h: Likewise.
	* fibheap.h: Likewise.
	* hsa_ext_image.h: Likewise.
	* hashtab.h: Likewise.
	* libcollector.h: Likewise.
	* sort.h: Likewise.
	* symcat.h: Likewise.
	* hsa_ext_amd.h: Likewise.
libcpp/
	* directives.cc: Remove trailing whitespace.
	* mkdeps.cc: Likewise.
	* line-map.cc: Likewise.
	* internal.h: Likewise.
	* files.cc: Likewise.
	* init.cc: Likewise.
	* makeucnid.cc: Likewise.
	* system.h: Likewise.
	* include/line-map.h: Likewise.
	* include/symtab.h: Likewise.
	* include/cpplib.h: Likewise.
	* expr.cc: Likewise.
	* charset.cc: Likewise.
	* macro.cc: Likewise.
	* errors.cc: Likewise.
	* lex.cc: Likewise.
	* traditional.cc: Likewise.
libgcc/
	* crtstuff.c: Remove trailing whitespace.
	* libgcov.h: Likewise.
	* config/alpha/crtfastmath.c: Likewise.
	* config/alpha/vms-gcc_shell_handler.c: Likewise.
	* config/alpha/vms-unwind.h: Likewise.
	* config/pa/linux-atomic.c: Likewise.
	* config/pa/linux-unwind.h: Likewise.
	* config/pa/quadlib.c: Likewise.
	* config/pa/fptr.c: Likewise.
	* config/s390/32/_fixsfdi.c: Likewise.
	* config/s390/32/_fixunssfdi.c: Likewise.
	* config/s390/32/_fixunsdfdi.c: Likewise.
	* config/c6x/pr-support.c: Likewise.
	* config/lm32/_udivsi3.c: Likewise.
	* config/lm32/libgcc_lm32.h: Likewise.
	* config/lm32/_udivmodsi4.c: Likewise.
	* config/lm32/_mulsi3.c: Likewise.
	* config/lm32/_modsi3.c: Likewise.
	* config/lm32/_umodsi3.c: Likewise.
	* config/lm32/_divsi3.c: Likewise.
	* config/darwin-crt3.c: Likewise.
	* config/msp430/mpy.c: Likewise.
	* config/ia64/tf-signs.c: Likewise.
	* config/ia64/fde-vms.c: Likewise.
	* config/ia64/unwind-ia64.c: Likewise.
	* config/ia64/vms-unwind.h: Likewise.
	* config/ia64/sfp-exceptions.c: Likewise.
	* config/ia64/quadlib.c: Likewise.
	* config/ia64/unwind-ia64.h: Likewise.
	* config/rl78/vregs.h: Likewise.
	* config/arm/bpabi.c: Likewise.
	* config/arm/unwind-arm.c: Likewise.
	* config/arm/pr-support.c: Likewise.
	* config/arm/linux-atomic.c: Likewise.
	* config/arm/bpabi-lib.h: Likewise.
	* config/frv/frvend.c: Likewise.
	* config/frv/cmovw.c: Likewise.
	* config/frv/frvbegin.c: Likewise.
	* config/frv/cmovd.c: Likewise.
	* config/frv/cmovh.c: Likewise.
	* config/aarch64/cpuinfo.c: Likewise.
	* config/i386/crtfastmath.c: Likewise.
	* config/i386/cygming-crtend.c: Likewise.
	* config/i386/32/tf-signs.c: Likewise.
	* config/i386/crtprec.c: Likewise.
	* config/i386/sfp-exceptions.c: Likewise.
	* config/i386/w32-unwind.h: Likewise.
	* config/m32r/initfini.c: Likewise.
	* config/sparc/crtfastmath.c: Likewise.
	* config/gcn/amdgcn_veclib.h: Likewise.
	* config/nios2/linux-atomic.c: Likewise.
	* config/nios2/linux-unwind.h: Likewise.
	* config/nios2/lib2-mul.c: Likewise.
	* config/nios2/lib2-nios2.h: Likewise.
	* config/xtensa/unwind-dw2-xtensa.c: Likewise.
	* config/rs6000/darwin-fallback.c: Likewise.
	* config/rs6000/ibm-ldouble.c: Likewise.
	* config/rs6000/sfp-machine.h: Likewise.
	* config/rs6000/darwin-asm.h: Likewise.
	* config/rs6000/darwin-crt2.c: Likewise.
	* config/rs6000/aix-unwind.h: Likewise.
	* config/rs6000/sfp-exceptions.c: Likewise.
	* config/gthr-vxworks.c: Likewise.
	* config/riscv/atomic.c: Likewise.
	* config/visium/memcpy.c: Likewise.
	* config/darwin-crt-tm.c: Likewise.
	* config/stormy16/lib2funcs.c: Likewise.
	* config/arc/ieee-754/divtab-arc-sf.c: Likewise.
	* config/arc/ieee-754/divtab-arc-df.c: Likewise.
	* config/arc/initfini.c: Likewise.
	* config/sol2/gmon.c: Likewise.
	* config/microblaze/divsi3_table.c: Likewise.
	* config/m68k/fpgnulib.c: Likewise.
	* libgcov-driver.c: Likewise.
	* unwind-dw2.c: Likewise.
	* fp-bit.c: Likewise.
	* dfp-bit.h: Likewise.
	* dfp-bit.c: Likewise.
	* libgcov-driver-system.c: Likewise.
libgcc/config/libbid/
	* _le_td.c: Remove trailing whitespace.
	* bid128_compare.c: Likewise.
	* bid_div_macros.h: Likewise.
	* bid64_to_bid128.c: Likewise.
	* bid64_to_uint32.c: Likewise.
	* bid128_to_uint64.c: Likewise.
	* bid64_div.c: Likewise.
	* bid128_round_integral.c: Likewise.
	* bid_binarydecimal.c: Likewise.
	* bid128_string.c: Likewise.
	* bid_flag_operations.c: Likewise.
	* bid128_to_int64.c: Likewise.
	* _mul_sd.c: Likewise.
	* bid64_mul.c: Likewise.
	* bid128_noncomp.c: Likewise.
	* _gt_dd.c: Likewise.
	* bid64_add.c: Likewise.
	* bid64_string.c: Likewise.
	* bid_from_int.c: Likewise.
	* bid128.c: Likewise.
	* _ge_dd.c: Likewise.
	* _ne_sd.c: Likewise.
	* _dd_to_td.c: Likewise.
	* _unord_sd.c: Likewise.
	* bid64_to_uint64.c: Likewise.
	* _gt_sd.c: Likewise.
	* _sd_to_td.c: Likewise.
	* _addsub_td.c: Likewise.
	* _ne_td.c: Likewise.
	* bid_dpd.c: Likewise.
	* bid128_add.c: Likewise.
	* bid128_next.c: Likewise.
	* _lt_sd.c: Likewise.
	* bid64_next.c: Likewise.
	* bid128_mul.c: Likewise.
	* _lt_dd.c: Likewise.
	* _ge_td.c: Likewise.
	* _unord_dd.c: Likewise.
	* bid64_sqrt.c: Likewise.
	* bid_sqrt_macros.h: Likewise.
	* bid64_fma.c: Likewise.
	* _sd_to_dd.c: Likewise.
	* bid_conf.h: Likewise.
	* bid64_noncomp.c: Likewise.
	* bid_gcc_intrinsics.h: Likewise.
	* _gt_td.c: Likewise.
	* _ge_sd.c: Likewise.
	* bid128_minmax.c: Likewise.
	* bid128_quantize.c: Likewise.
	* bid32_to_bid64.c: Likewise.
	* bid_round.c: Likewise.
	* _td_to_sd.c: Likewise.
	* bid_inline_add.h: Likewise.
	* bid128_fma.c: Likewise.
	* _eq_td.c: Likewise.
	* bid32_to_bid128.c: Likewise.
	* bid64_rem.c: Likewise.
	* bid128_2_str_tables.c: Likewise.
	* _mul_dd.c: Likewise.
	* _dd_to_sd.c: Likewise.
	* bid128_div.c: Likewise.
	* _lt_td.c: Likewise.
	* bid64_compare.c: Likewise.
	* bid64_to_int32.c: Likewise.
	* _unord_td.c: Likewise.
	* bid128_rem.c: Likewise.
	* bid_internal.h: Likewise.
	* bid64_to_int64.c: Likewise.
	* _eq_dd.c: Likewise.
	* _td_to_dd.c: Likewise.
	* bid128_to_int32.c: Likewise.
	* bid128_to_uint32.c: Likewise.
	* _ne_dd.c: Likewise.
	* bid64_quantize.c: Likewise.
	* _le_dd.c: Likewise.
	* bid64_round_integral.c: Likewise.
	* _le_sd.c: Likewise.
	* bid64_minmax.c: Likewise.
libgcc/config/avr/libf7/
	* f7-renames.h: Remove trailing whitespace.
libstdc++-v3/
	* include/debug/debug.h: Remove trailing whitespace.
	* include/parallel/base.h: Likewise.
	* include/parallel/types.h: Likewise.
	* include/parallel/settings.h: Likewise.
	* include/parallel/multiseq_selection.h: Likewise.
	* include/parallel/partition.h: Likewise.
	* include/parallel/random_number.h: Likewise.
	* include/parallel/find_selectors.h: Likewise.
	* include/parallel/partial_sum.h: Likewise.
	* include/parallel/list_partition.h: Likewise.
	* include/parallel/search.h: Likewise.
	* include/parallel/algorithmfwd.h: Likewise.
	* include/parallel/random_shuffle.h: Likewise.
	* include/parallel/multiway_mergesort.h: Likewise.
	* include/parallel/sort.h: Likewise.
	* include/parallel/algobase.h: Likewise.
	* include/parallel/numericfwd.h: Likewise.
	* include/parallel/multiway_merge.h: Likewise.
	* include/parallel/losertree.h: Likewise.
	* include/bits/basic_ios.h: Likewise.
	* include/bits/stringfwd.h: Likewise.
	* include/bits/ostream_insert.h: Likewise.
	* include/bits/stl_heap.h: Likewise.
	* include/bits/unordered_map.h: Likewise.
	* include/bits/hashtable_policy.h: Likewise.
	* include/bits/stl_iterator_base_funcs.h: Likewise.
	* include/bits/valarray_before.h: Likewise.
	* include/bits/regex.h: Likewise.
	* include/bits/postypes.h: Likewise.
	* include/bits/stl_iterator.h: Likewise.
	* include/bits/localefwd.h: Likewise.
	* include/bits/stl_algo.h: Likewise.
	* include/bits/ios_base.h: Likewise.
	* include/bits/stl_function.h: Likewise.
	* include/bits/basic_string.h: Likewise.
	* include/bits/hashtable.h: Likewise.
	* include/bits/valarray_after.h: Likewise.
	* include/bits/char_traits.h: Likewise.
	* include/bits/gslice.h: Likewise.
	* include/bits/locale_facets_nonio.h: Likewise.
	* include/bits/mask_array.h: Likewise.
	* include/bits/specfun.h: Likewise.
	* include/bits/random.h: Likewise.
	* include/bits/slice_array.h: Likewise.
	* include/bits/valarray_array.h: Likewise.
	* include/tr1/float.h: Likewise.
	* include/tr1/functional_hash.h: Likewise.
	* include/tr1/math.h: Likewise.
	* include/tr1/hashtable_policy.h: Likewise.
	* include/tr1/stdio.h: Likewise.
	* include/tr1/complex.h: Likewise.
	* include/tr1/stdbool.h: Likewise.
	* include/tr1/stdarg.h: Likewise.
	* include/tr1/inttypes.h: Likewise.
	* include/tr1/fenv.h: Likewise.
	* include/tr1/stdlib.h: Likewise.
	* include/tr1/wchar.h: Likewise.
	* include/tr1/tgmath.h: Likewise.
	* include/tr1/limits.h: Likewise.
	* include/tr1/wctype.h: Likewise.
	* include/tr1/stdint.h: Likewise.
	* include/tr1/ctype.h: Likewise.
	* include/tr1/random.h: Likewise.
	* include/tr1/shared_ptr.h: Likewise.
	* include/ext/mt_allocator.h: Likewise.
	* include/ext/sso_string_base.h: Likewise.
	* include/ext/debug_allocator.h: Likewise.
	* include/ext/vstring_fwd.h: Likewise.
	* include/ext/pointer.h: Likewise.
	* include/ext/pod_char_traits.h: Likewise.
	* include/ext/malloc_allocator.h: Likewise.
	* include/ext/vstring.h: Likewise.
	* include/ext/bitmap_allocator.h: Likewise.
	* include/ext/pool_allocator.h: Likewise.
	* include/ext/type_traits.h: Likewise.
	* include/ext/ropeimpl.h: Likewise.
	* include/ext/codecvt_specializations.h: Likewise.
	* include/ext/throw_allocator.h: Likewise.
	* include/ext/extptr_allocator.h: Likewise.
	* include/ext/atomicity.h: Likewise.
	* include/ext/concurrence.h: Likewise.
	* include/c_compatibility/wchar.h: Likewise.
	* include/c_compatibility/stdint.h: Likewise.
	* include/backward/hash_fun.h: Likewise.
	* include/backward/binders.h: Likewise.
	* include/backward/hashtable.h: Likewise.
	* include/backward/auto_ptr.h: Likewise.
	* libsupc++/eh_arm.cc: Likewise.
	* libsupc++/unwind-cxx.h: Likewise.
	* libsupc++/si_class_type_info.cc: Likewise.
	* libsupc++/vec.cc: Likewise.
	* libsupc++/class_type_info.cc: Likewise.
	* libsupc++/vmi_class_type_info.cc: Likewise.
	* libsupc++/guard_error.cc: Likewise.
	* libsupc++/bad_typeid.cc: Likewise.
	* libsupc++/eh_personality.cc: Likewise.
	* libsupc++/atexit_arm.cc: Likewise.
	* libsupc++/pmem_type_info.cc: Likewise.
	* libsupc++/vterminate.cc: Likewise.
	* libsupc++/eh_terminate.cc: Likewise.
	* libsupc++/bad_cast.cc: Likewise.
	* libsupc++/exception_ptr.h: Likewise.
	* libsupc++/eh_throw.cc: Likewise.
	* libsupc++/bad_alloc.cc: Likewise.
	* libsupc++/nested_exception.cc: Likewise.
	* libsupc++/pointer_type_info.cc: Likewise.
	* libsupc++/pbase_type_info.cc: Likewise.
	* libsupc++/bad_array_new.cc: Likewise.
	* libsupc++/pure.cc: Likewise.
	* libsupc++/eh_exception.cc: Likewise.
	* libsupc++/bad_array_length.cc: Likewise.
	* libsupc++/cxxabi.h: Likewise.
	* libsupc++/guard.cc: Likewise.
	* libsupc++/eh_catch.cc: Likewise.
	* libsupc++/cxxabi_forced.h: Likewise.
	* libsupc++/tinfo.h: Likewise.

The following patch on top of the r15-4346 patch adds
-Wleading-whitespace= warning option.
This warning doesn't care how much one actually indents which line
in the source (that is something that can't be easily done in the
preprocessor without doing syntactic analysis), but just simple checks
on what kind of whitespace is used in the indentation.
I think it is still useful to get warnings about such issues early,
while git diagnoses some of it in patches (e.g. the tab after space
case), getting the warnings earlier might help avoiding such issues
sooner.

There are projects which ban use of tabs and require just spaces,
others which require indentation just with horizontal tabs, and finally
projects which want indentation with tabs for multiples of tabstop size
followed by spaces (fewer than tabstop size), like GCC.
For all 3 kinds the warning diagnoses indentation with '\v' or '\f'
characters (unless line contains just whitespace), and for the last one
also cases where a space in the indentation is followed by horizontal
tab or where there are N or more consecutive spaces in the indentation
(for -ftabstop=N).

BTW, for additional testing I've enabled the warnings (without -Werror
for them) in stage3.  There are many warnings (both trailing and leading
whitespace), some of them something that can be easily fixed in the headers
or source files, but others with whitespace issues in generated sources,
so if we enable the warnings, either we'd need to adjust the generators
or disable the warnings in (some of the) generated files.

2024-10-23  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* include/cpplib.h (struct cpp_options): Add
	cpp_warn_leading_whitespace and cpp_tabstop members.
	(enum cpp_warning_reason): Add CPP_W_LEADING_WHITESPACE.
	* internal.h (struct _cpp_line_note): Document new
	line note kinds.
	* init.cc (cpp_create_reader): Set cpp_tabstop to 8.
	* lex.cc (find_leading_whitespace_issues): New function.
	(_cpp_clean_line): Use it.
	(_cpp_process_line_notes): Handle 'L', 'S' and 'T' line notes.
	(lex_raw_string): Clear type on 'L', 'S' and 'T' line notes
	inside of raw string literals.
gcc/
	* doc/invoke.texi (Wleading-whitespace=): Document.
gcc/c-family/
	* c.opt (Wleading-whitespace=): New option.
	* c-opts.cc (c_common_post_options): Set cpp_opts->cpp_tabstop
	to global_dc->m_tabstop.
gcc/testsuite/
	* c-c++-common/cpp/Wleading-whitespace-1.c: New test.
	* c-c++-common/cpp/Wleading-whitespace-2.c: New test.
	* c-c++-common/cpp/Wleading-whitespace-3.c: New test.
	* c-c++-common/cpp/Wleading-whitespace-4.c: New test.

libcpp is not currently set up to be able to generate valid
locations for tokens lexed from a _Pragma string. Instead, after obtaining
the tokens, it sets their locations all to the location of the _Pragma
operator itself. This makes things like _Pragma("GCC diagnostic") work well
enough, but if any diagnostics are issued during lexing, prior to resetting
the token locations, those diagnostics get issued at the invalid
locations. Fix that up by adding a new field pfile->diagnostic_override_loc
that instructs libcpp to issue diagnostics at the alternate location.

libcpp/ChangeLog:

	PR preprocessor/114423
	* internal.h (struct cpp_reader): Add DIAGNOSTIC_OVERRIDE_LOC
	field.
	* directives.cc (destringize_and_run): Set the new field to the
	location of the _Pragma operator.
	* errors.cc (cpp_diagnostic_at): Support DIAGNOSTIC_OVERRIDE_LOC to
	temporarily issue diagnostics at a different location.
	(cpp_diagnostic_with_line): Likewise.

gcc/testsuite/ChangeLog:

	PR preprocessor/114423
	* c-c++-common/cpp/pragma-diagnostic-loc.c: New test.
	* c-c++-common/cpp/diagnostic-pragma-1.c: Adjust expected output.
	* g++.dg/pch/operator-1.C: Likewise.

The following patch partially implements the N3353 paper.
In particular, it adds support for the delimited escape sequences
(\u{123}, \x{123}, \o{123}) which were added already for C++23,
all I had to do is split the delimited escape sequence guarding from
named universal character escape sequence guards
(\N{LATIN CAPITAL LETTER C WITH CARON}), which C++23 has but C2Y doesn't
and emit different diagnostics for C from C++ for the delimited escape
sequences.
And it adds support for the new style of octal literals, 0o137 or 0O1777.
I have so far added that just for C and not C++, because I have no idea
whether C++ will want to handle it similarly.

What the patch doesn't do is any kind of diagnostics for obsoletion of
\137 or 0137, as discussed in the PR, I think it is way too early for that.
Perhaps some non-default warning later on.

2024-10-17  Jakub Jelinek  <jakub@redhat.com>

	PR c/117028
libcpp/
	* include/cpplib.h (struct cpp_options): Add named_uc_escape_seqs,
	octal_constants and cpp_warn_c23_c2y_compat members.
	(enum cpp_warning_reason): Add CPP_W_C23_C2Y_COMPAT enumerator.
	* init.cc (struct lang_flags): Add named_uc_escape_seqs and
	octal_constants bit-fields.
	(lang_defaults): Add initializers for them into the table.
	(cpp_set_lang): Initialize named_uc_escape_seqs and octal_constants.
	(cpp_create_reader): Initialize cpp_warn_c23_c2y_compat to -1.
	* charset.cc (_cpp_valid_ucn): Test
	CPP_OPTION (pfile, named_uc_escape_seqs) rather than
	CPP_OPTION (pfile, delimited_escape_seqs) in \N{} related tests.
	Change wording of C cpp_pedwarning for \u{} and emit
	-Wc23-c2y-compat warning for it too if needed.  Formatting fixes.
	(convert_hex): Change wording of C cpp_pedwarning for \u{} and emit
	-Wc23-c2y-compat warning for it too if needed.
	(convert_oct): Likewise.
	* expr.cc (cpp_classify_number): Handle C2Y 0o or 0O prefixed
	octal constants.
	(cpp_interpret_integer): Likewise.
gcc/c-family/
	* c.opt (Wc23-c2y-compat): Add CPP and CppReason parameters.
	* c-opts.cc (set_std_c2y): Use CLK_STDC2Y or CLK_GNUC2Y rather
	than CLK_STDC23 and CLK_GNUC23.  Formatting fix.
	* c-lex.cc (interpret_integer): Handle C2Y 0o or 0O prefixed
	and wb/WB/uwb/UWB suffixed octal constants.
gcc/testsuite/
	* gcc.dg/bitint-112.c: New test.
	* gcc.dg/c23-digit-separators-1.c: Add _Static_assert for
	valid binary constant with digit separator.
	* gcc.dg/c23-octal-constants-1.c: New test.
	* gcc.dg/c23-octal-constants-2.c: New test.
	* gcc.dg/c2y-digit-separators-1.c: New test.
	* gcc.dg/c2y-digit-separators-2.c: New test.
	* gcc.dg/c2y-octal-constants-1.c: New test.
	* gcc.dg/c2y-octal-constants-2.c: New test.
	* gcc.dg/c2y-octal-constants-3.c: New test.
	* gcc.dg/cpp/c23-delimited-escape-seq-1.c: New test.
	* gcc.dg/cpp/c23-delimited-escape-seq-2.c: New test.
	* gcc.dg/cpp/c2y-delimited-escape-seq-1.c: New test.
	* gcc.dg/cpp/c2y-delimited-escape-seq-2.c: New test.
	* gcc.dg/cpp/c2y-delimited-escape-seq-3.c: New test.
	* gcc.dg/cpp/c2y-delimited-escape-seq-4.c: New test.
	* gcc.dg/octal-constants-1.c: New test.
	* gcc.dg/octal-constants-2.c: New test.
	* gcc.dg/octal-constants-3.c: New test.
	* gcc.dg/octal-constants-4.c: New test.
	* gcc.dg/system-octal-constants-1.c: New test.
	* gcc.dg/system-octal-constants-1.h: New file.

While working on PR117028 C2Y changes, I've noticed weird ternary
operator formatting (operand1 ? operand2: operand3).
The usual formatting is operand1 ? operand2 : operand3
where we have around 18000+ cases of that (counting only what fits
on one line) and
indent -nbad -bap -nbc -bbo -bl -bli2 -bls -ncdb -nce -cp1 -cs -di2 -ndj \
       -nfc1 -nfca -hnl -i2 -ip5 -lp -pcs -psl -nsc -nsob
documented in
https://www.gnu.org/prep/standards/html_node/Formatting.html#Formatting
does the same.
Some code was even trying to save space as much as possible and used
operand1?operand2:operand3 or
operand1 ? operand2:operand3

Today I've grepped for such cases (the grep was '?.*[^ ]:' and I had to
skim through various false positives with that where the : matched e.g.
stuff inside of strings, or *.md pattern macros or :: scope) and the
following patch is a fix for what I found.

2024-10-16  Jakub Jelinek  <jakub@redhat.com>

gcc/
	* attribs.cc (lookup_scoped_attribute_spec): ?: operator formatting
	fixes.
	* basic-block.h (FOR_BB_INSNS_SAFE): Likewise.
	* cfgcleanup.cc (outgoing_edges_match): Likewise.
	* cgraph.cc (cgraph_node::dump): Likewise.
	* config/arc/arc.cc (gen_acc1, gen_acc2): Likewise.
	* config/arc/arc.h (CLASS_MAX_NREGS, CONSTANT_ADDRESS_P): Likewise.
	* config/arm/arm.cc (arm_print_operand): Likewise.
	* config/cris/cris.md (*b<rnzcond:code><mode>): Likewise.
	* config/darwin.cc (darwin_asm_declare_object_name,
	darwin_emit_common): Likewise.
	* config/darwin-driver.cc (darwin_driver_init): Likewise.
	* config/epiphany/epiphany.md (call, sibcall, call_value,
	sibcall_value): Likewise.
	* config/i386/i386.cc (gen_push2): Likewise.
	* config/i386/i386.h (ix86_cur_cost): Likewise.
	* config/i386/openbsdelf.h (FUNCTION_PROFILER): Likewise.
	* config/loongarch/loongarch-c.cc (loongarch_cpu_cpp_builtins):
	Likewise.
	* config/loongarch/loongarch-cpu.cc (fill_native_cpu_config):
	Likewise.
	* config/riscv/riscv.cc (riscv_union_memmodels): Likewise.
	* config/riscv/zc.md (*mva01s<X:mode>, *mvsa01<X:mode>): Likewise.
	* config/rs6000/mmintrin.h (_mm_cmpeq_pi8, _mm_cmpgt_pi8,
	_mm_cmpeq_pi16, _mm_cmpgt_pi16, _mm_cmpeq_pi32, _mm_cmpgt_pi32):
	Likewise.
	* config/v850/predicates.md (pattern_is_ok_for_prologue): Likewise.
	* config/xtensa/constraints.md (d, C, W): Likewise.
	* coverage.cc (coverage_begin_function, build_init_ctor,
	build_gcov_exit_decl): Likewise.
	* df-problems.cc (df_create_unused_note): Likewise.
	* diagnostic.cc (diagnostic_set_caret_max_width): Likewise.
	* diagnostic-path.cc (path_summary::path_summary): Likewise.
	* expr.cc (expand_expr_divmod): Likewise.
	* gcov.cc (format_gcov): Likewise.
	* gcov-dump.cc (dump_gcov_file): Likewise.
	* genmatch.cc (main): Likewise.
	* incpath.cc (remove_duplicates, register_include_chains): Likewise.
	* ipa-devirt.cc (dump_odr_type): Likewise.
	* ipa-icf.cc (sem_item_optimizer::merge_classes): Likewise.
	* ipa-inline.cc (inline_small_functions): Likewise.
	* ipa-polymorphic-call.cc (ipa_polymorphic_call_context::dump):
	Likewise.
	* ipa-sra.cc (create_parameter_descriptors): Likewise.
	* ipa-utils.cc (find_always_executed_bbs): Likewise.
	* predict.cc (predict_loops): Likewise.
	* selftest.cc (read_file): Likewise.
	* sreal.h (SREAL_SIGN, SREAL_ABS): Likewise.
	* tree-dump.cc (dequeue_and_dump): Likewise.
	* tree-ssa-ccp.cc (bit_value_binop): Likewise.
gcc/c-family/
	* c-opts.cc (c_common_init_options, c_common_handle_option,
	c_common_finish, set_std_c89, set_std_c99, set_std_c11,
	set_std_c17, set_std_c23, set_std_cxx98, set_std_cxx11,
	set_std_cxx14, set_std_cxx17, set_std_cxx20, set_std_cxx23,
	set_std_cxx26): ?: operator formatting fixes.
gcc/cp/
	* search.cc (lookup_member): ?: operator formatting fixes.
	* typeck.cc (cp_build_modify_expr): Likewise.
libcpp/
	* expr.cc (interpret_float_suffix): ?: operator formatting fixes.

This patch actually optimizes #embed, so far in C.

For a simple testcase (for 494447200 bytes long cc1plus):
cat embed-11.c
unsigned char a[] = {
  #embed "cc1plus"
};
time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c

real    0m13.647s
user    0m7.157s
sys     0m2.597s
time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c

real    0m28.649s
user    0m26.653s
sys     0m1.958s

and when configured against binutils with .base64 support
time ./xgcc -B ./ -S -std=c23 -O2 embed-11.c

real    0m4.283s
user    0m2.288s
sys     0m0.859s
time ./xgcc -B ./ -c -std=c23 -O2 embed-11.c

real    0m6.888s
user    0m5.876s
sys     0m1.002s

(all times with --enable-checking=yes,rtl,extra compiler).

Even just
./cc1plus -E -o embed-11.i embed-11.c
(which doesn't have this optimization yet and so preprocesses it as
1.3GB preprocessed file) needed almost 25GB of compile time RAM (but
preprocessed fine).
And compiling that embed-11.i with -std=c23 -O0 by unpatched gcc
I gave up after 400 seconds when it already ate 45GB of RAM and didn't
produce a single byte into embed-11.s yet.

The patch introduces a new CPP_EMBED token which contains raw memory image
virtually representing a sequence of int literals.
To simplify the parsing complexities, the preprocessor guarantees CPP_EMBED
is only emitted if there are 4+ (it actually does that for 64+ right now)
literals in the sequence and emits CPP_NUMBER CPP_COMMA CPP_EMBED CPP_COMMA
CPP_NUMBER tokens (with more CPP_EMBED separated by CPP_COMMA if it is
longer than 2GB, as STRING_CSTs in GCC and also the new RAW_DATA_CST etc.
are limited to INT_MAX elements).  The main reason is that the preprocessor
doesn't really know in which context #embed directive appears, there could
be e.g.
{ 25 *
  #embed "whatever"
* 2 - 15 }
or similar and dealing with this special case deep in the expression parsing
is undesirable.
With the CPP_NUMBERs around it, I believe in the C FE the only places which
need handling of the CPP_EMBED token are initializer parsing (that is the
only one which adds actual optimizations for it), comma expressions (I
believe nothing really cares whether it is 25,13,95 or
25,13,0,1,2,3,4,5,6,7,8,9,10,13,95 etc., so besides the 2 outer CPP_NUMBER
the parsing just adds one INTEGER_CST to the comma expression, I doubt users
want to be spammed with millions of -Wunused warnings per #embed),
whatever uses c_parser_expr_list (function calls, attribute arguments,
OpenMP sizes clause argument, OpenACC tile clause argument and whatever uses
c_parser_get_builtin_args (mainly for __builtin_shufflevector).  Please correct
me if I'm wrong.

The patch introduces a RAW_DATA_CST tree code, which can then be used inside
of array CONSTRUCTOR elt values.  In some sense RAW_DATA_CST is similar to
STRING_CST, but right now STRING_CST is used only if the whole array
initializer is that constant, while RAW_DATA_CST at index idx (should be
always INTEGER_CST index, another advantage of the CPP_NUMBER around is that
[30 ... 250] =
  #embed "whatever"
really does what it would do with a integer sequence there) stands for
[idx] = RAW_DATA_POINTER (val)[0],
[idx+1] = RAW_DATA_POINTER (val)[1],
...
[idx+RAW_DATA_LENGTH (val)-1] = RAW_DATA_POINTER (val)[RAW_DATA_LENGTH (val)-1].
Another important thing is that unlike STRING_CST which has the data
embedded in it RAW_DATA_CST doesn't own the data, it has RAW_DATA_OWNER
which owns the data (that can be a STRING_CST, e.g. used for PCH or LTO
after reading LTO in) or another RAW_DATA_CST (with NULL RAW_DATA_OWNER,
standing for data owned by libcpp buffers).  The advantage is that it can be
cheaply peeled off, or split into multiple smaller pieces, e.g. if one uses
designated initializer to store something into the middle of a 10GB #embed
array, in no case we need to actually copy data around for that.
Right now RAW_DATA_CST is only used in initializers of integral arrays where
the integer type has (host) CHAR_BIT precision, so usually char/signed
char/unsigned char (for C++ later maybe std::byte); in theory we could say
allocate 4 times as big buffer for conversions to int array and depending
on endianity and storage order reversal etc., but I'm not sure if that is
something that will be actually needed in the wild.
And an optimization inside of c-common.cc attempts to undo that CPP_NUMBER
CPP_EMBED CPP_NUMBER division in case one uses #embed the usual way and
doesn't use the boundary literals in weird ways and the values there match
the surrounding bytes in the owner buffer.

For LTO, in order to avoid copying perhaps gigabytes long data around,
the hacks in the streamer out/in cause the data owned by libcpp to be
streamed right into the stream and streamed back as a STRING_CST which
owns the data.

2024-10-16  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* include/cpplib.h (TTYPE_TABLE): Add CPP_EMBED token type.
	* files.cc (finish_embed): For limit >= 64 and C preprocessing
	instead of emitting CPP_NUMBER CPP_COMMA separated sequence for the
	whole embed emit it just for the first and last byte and in between
	emit a CPP_EMBED token or tokens if too large.
gcc/
	* treestruct.def (TS_RAW_DATA_CST): New.
	* tree.def (RAW_DATA_CST): New tree code.
	* tree-core.h (struct tree_raw_data): New type.
	(union tree_node): Add raw_data_cst member.
	* tree.h (RAW_DATA_LENGTH, RAW_DATA_POINTER, RAW_DATA_OWNER): Define.
	(gt_ggc_mx, gt_pch_nx): Declare overloads for tree_raw_data *.
	* tree.cc (tree_node_structure_for_code): Handle RAW_DATA_CST.
	(initialize_tree_contains_struct): Handle TS_RAW_DATA_CST.
	(tree_code_size): Handle RAW_DATA_CST.
	(initializer_zerop): Likewise.
	(gt_ggc_mx, gt_pch_nx): Define overloads for tree_raw_data *.
	* gimplify.cc (gimplify_init_ctor_eval): Handle RAW_DATA_CST.
	* fold-const.cc (operand_compare::operand_equal_p): Handle
	RAW_DATA_CST.  Formatting fix.
	(operand_compare::hash_operand): Handle RAW_DATA_CST.
	(native_encode_initializer): Likewise.
	(get_array_ctor_element_at_index): Likewise.
	(fold): Likewise.
	* gimple-fold.cc (fold_array_ctor_reference): Likewise.  Formatting
	fix.
	* varasm.cc (const_hash_1): Handle RAW_DATA_CST.
	(initializer_constant_valid_p_1): Likewise.
	(array_size_for_constructor): Likewise.
	(output_constructor_regular_field): Likewise.
	* expr.cc (categorize_ctor_elements_1): Likewise.
	(expand_expr_real_1) <case ARRAY_REF>: Punt for RAW_DATA_CST.
	* tree-streamer.cc (streamer_check_handled_ts_structures): Mark
	TS_RAW_DATA_CST as handled.
	* tree-streamer-in.cc (streamer_alloc_tree): Handle RAW_DATA_CST.
	(lto_input_ts_raw_data_cst_tree_pointers): New function.
	(streamer_read_tree_body): Call it for RAW_DATA_CST.
	* tree-streamer-out.cc (write_ts_raw_data_cst_tree_pointers): New
	function.
	(streamer_write_tree_body): Call it for RAW_DATA_CST.
	(streamer_write_tree_header): Handle RAW_DATA_CST.
	* lto-streamer-out.cc (DFS::DFS_write_tree_body): Handle RAW_DATA_CST.
	* tree-pretty-print.cc (dump_generic_node): Likewise.
gcc/c-family/
	* c-ppoutput.cc (token_streamer::stream): Add special code to spell
	CPP_EMBED token.
	* c-lex.cc (c_lex_with_flags): Handle CPP_EMBED.  Formatting fix.
	* c-common.cc (c_parse_error): Handle CPP_EMBED.
	(braced_list_to_string): Optimize RAW_DATA_CST surrounded by
	INTEGER_CSTs which match some bytes before or after RAW_DATA_CST in
	its owner.
gcc/c/
	* c-parser.cc (c_parser_braced_init): Handle CPP_EMBED.
	(c_parser_get_builtin_args): Likewise.
	(c_parser_expression): Likewise.
	(c_parser_expr_list): Likewise.
	* c-typeck.cc (digest_init): Handle RAW_DATA_CST.  Formatting fix.
	(init_node_successor): New function.
	(add_pending_init): Handle RAW_DATA_CST.
	(set_nonincremental_init): Formatting fix.
	(output_init_element): Handle RAW_DATA_CST.  Formatting fixes.
	(maybe_split_raw_data): New function.
	(process_init_element): Use maybe_split_raw_data.  Handle
	RAW_DATA_CST.
gcc/testsuite/
	* c-c++-common/cpp/embed-20.c: New test.
	* c-c++-common/cpp/embed-21.c: New test.
	* c-c++-common/cpp/embed-28.c: New test.
	* gcc.dg/cpp/embed-8.c: New test.
	* gcc.dg/cpp/embed-9.c: New test.
	* gcc.dg/cpp/embed-10.c: New test.
	* gcc.dg/cpp/embed-11.c: New test.
	* gcc.dg/cpp/embed-12.c: New test.
	* gcc.dg/cpp/embed-13.c: New test.
	* gcc.dg/cpp/embed-14.c: New test.
	* gcc.dg/cpp/embed-15.c: New test.
	* gcc.dg/cpp/embed-16.c: New test.
	* gcc.dg/pch/embed-1.c: New test.
	* gcc.dg/pch/embed-1.hs: New test.
	* gcc.dg/lto/embed-1_0.c: New test.
	* gcc.dg/lto/embed-1_1.c: New test.

Trailing blanks is something even git diff diagnoses; while it is a coding
style issue, if it is so common that git diff diagnoses it, I think it could
be useful to various projects to check that at compile time.

Dunno if it should be included in -Wextra, currently it isn't, and due to
tons of trailing whitespace in our sources, haven't enabled it for when
building gcc itself either.

Note, git diff also diagnoses indentation with tab following space, wonder
if we couldn't have trivial warning options where one would simply ask for
checking of indentation with no tabs, just spaces vs. indentation with
tabs followed by spaces (but never tab width or more spaces in the
indentation).  I think that would be easy to do also on the libcpp side.
Checking how much something should be exactly indented requires syntax
analysis (at least some limited one) and can consider columns of first token
on line, but what the exact indentation blanks were is something only libcpp
knows.

On Thu, Sep 19, 2024 at 08:17:24AM +0200, Richard Biener wrote:
> Generally I like diagnosing this early.  For the above I'd say -Wtrailing-whitespace=
> with a set of things to diagnose (and a sane default - just spaces and tabs - for
> -Wtrailiing-whitespace) would be nice.  As for naming possibly follow the
> is{space,blank,cntrl} character classifications?  If those are a good
> fit, that is.

The patch currently allows blank (' ' '\t') and space (' ' '\t' '\f' '\v'),
cntrl not yet added, not anything non-ASCII, but in theory could
be added later (though, non-ASCII would be just for inside of comments,
say non-breaking space etc. in the source is otherwise an error).

2024-10-15  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* include/cpplib.h (struct cpp_options): Add
	cpp_warn_trailing_whitespace member.
	(enum cpp_warning_reason): Add CPP_W_TRAILING_WHITESPACE.
	* internal.h (struct _cpp_line_note): Document 'W' line note.
	* lex.cc (_cpp_clean_line): Add 'W' line note for trailing whitespace
	except for trailing whitespace after backslash.  Formatting fix.
	(_cpp_process_line_notes): Emit -Wtrailing-whitespace diagnostics.
	Formatting fixes.
	(lex_raw_string): Clear type on 'W' notes.
gcc/
	* doc/invoke.texi (Wtrailing-whitespace): Document.
gcc/c-family/
	* c.opt (Wtrailing-whitespace=): New option.
	(Wtrailing-whitespace): New alias.
	* c.opt.urls: Regenerate.
gcc/testsuite/
	* c-c++-common/cpp/Wtrailing-whitespace-1.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-2.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-3.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-4.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-5.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-6.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-7.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-8.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-9.c: New test.
	* c-c++-common/cpp/Wtrailing-whitespace-10.c: New test.

Within the compiler, module keywords "import", "module", and "export" that
are recognized as part of module directives gain an extra trailing space to
distinguish them from other non-keyword uses of those words in the code.
But when dumping preprocessed output, printing those spaces creates a
gratuitous inconsistency with non-modules preprocessing, as revealed by
several of the g++.dg/modules/cpp* tests if modules are enabled by default
in C++20 mode.

libcpp/ChangeLog:

	* lex.cc (cpp_output_token): Omit terminal space from name.

gcc/testsuite/ChangeLog:

	* g++.dg/modules/cpp-2_c.C: Expect only one space after import.
	* g++.dg/modules/cpp-5_c.C
	* g++.dg/modules/dep-2.C
	* g++.dg/modules/dir-only-2_b.C
	* g++.dg/modules/pr99050_b.C
	* g++.dg/modules/inc-xlate-1_b.H
	* g++.dg/modules/legacy-3_b.H
	* g++.dg/modules/legacy-3_c.H: Likewise.

_Pragma("GCC system_header") currently takes effect only partially. It does
succeed in updating the line_map, so that checks like in_system_header_at()
return correctly, but it does not update pfile->buffer->sysp.  One result is
that a subsequent #include does not set up the system header state properly
for the newly included file, as pointed out in the PR. Fix by propagating
the new system header state back to the buffer after processing the pragma.

libcpp/ChangeLog:

	PR preprocessor/114436
	* directives.cc (destringize_and_run): If the _Pragma changed the
	buffer system header state (e.g. because it was "GCC
	system_header"), propagate that change back to the actual buffer
	too.

gcc/testsuite/ChangeLog:

	PR preprocessor/114436
	* c-c++-common/cpp/pragma-system-header-1.h: New test.
	* c-c++-common/cpp/pragma-system-header-2.h: New test.
	* c-c++-common/cpp/pragma-system-header.c: New test.

The implementation of #pragma push_macro and #pragma pop_macro has to date
made use of an ad-hoc function, _cpp_lex_identifier(), which lexes an
identifier out of a string. When support was added for extended characters
in identifiers ($, UCNs, or UTF-8), that support was added only for the
"normal" way of lexing identifiers out of a cpp_buffer (_cpp_lex_direct) and
not for the ad-hoc way. Consequently, extended identifiers are not usable
with these pragmas.

The logic for lexing identifiers has become more complicated than it was
when _cpp_lex_identifier() was written -- it now handles things like \N{}
escapes in C++, for instance -- and it no longer seems practical to maintain
a redundant code path for lexing identifiers. Address the issue by changing
the implementation of #pragma {push,pop}_macro to lex identifiers in the
expected way, i.e. by pushing a cpp_buffer and lexing the identifier from
there.

The existing implementation has some quirks because of the ad-hoc parsing
logic. For example:

 #pragma push_macro("X ")
 ...
 #pragma pop_macro("X")

will not restore macro X (note the extra space in the first string). However:

 #pragma push_macro("X ")
 ...
 #pragma pop_macro("X ")

actually does sucessfully restore "X". This is because the key for looking
up the saved macro on the push stack is the original string passed, so the
string passed to pop_macro needs to match it exactly. It is not that easy to
reproduce this logic in the world of extended characters, given that for
example it should be valid to pass a UCN to push_macro, and the
corresponding UTF-8 to pop_macro. Given that this aspect of the existing
behavior seems unintentional and has no tests (and does not match other
implementations), I opted to make the new logic more straightforward. The
string passed needs to lex to one token, which must be a valid identifier,
or else no action is taken and no error is generated. Any diagnostics
encountered during lexing (e.g., due to a UTF-8 character not permitted to
appear in an identifier) are also suppressed.

It could be nice (for GCC 15) to also add a warning if a pop_macro does not
match a previous push_macro.

libcpp/ChangeLog:

	PR preprocessor/109704
	* include/cpplib.h (class cpp_auto_suppress_diagnostics): New class.
	* errors.cc
	(cpp_auto_suppress_diagnostics::cpp_auto_suppress_diagnostics): New
	function.
	(cpp_auto_suppress_diagnostics::~cpp_auto_suppress_diagnostics): New
	function.
	* charset.cc (noop_diagnostic_cb): Remove.
	(cpp_interpret_string_ranges): Refactor diagnostic suppression logic
	into new class cpp_auto_suppress_diagnostics.
	(count_source_chars): Likewise.
	* directives.cc (cpp_pop_definition): Add cpp_hashnode argument.
	(lex_identifier_from_string): New static helper function.
	(push_pop_macro_common): Refactor common logic from
	do_pragma_push_macro and do_pragma_pop_macro; use
	lex_identifier_from_string instead of _cpp_lex_identifier.
	(do_pragma_push_macro): Reimplement using push_pop_macro_common.
	(do_pragma_pop_macro): Likewise.
	* internal.h (_cpp_lex_identifier): Remove.
	* lex.cc (lex_identifier_intern): Remove.
	(_cpp_lex_identifier): Remove.

gcc/testsuite/ChangeLog:

	PR preprocessor/109704
	* c-c++-common/cpp/pragma-push-pop-utf8.c: New test.
	* g++.dg/pch/pushpop-2.C: New test.
	* g++.dg/pch/pushpop-2.Hs: New test.
	* gcc.dg/pch/pushpop-2.c: New test.
	* gcc.dg/pch/pushpop-2.hs: New test.

When working on #embed support, or -Wheader-guard or other recent libcpp
changes, I've been annoyed by the libcpp diagnostics being visually
different from normal gcc diagnostics, especially in the area of quoting
stuff in the diagnostic messages.
Normall GCC diagnostics is gcc_diag/gcc_tdiag, one can use
%</%>, %qs etc. in there, while libcpp diagnostics was marked as printf
and in libcpp we've been very creative with quoting stuff, either
no quotes at all, or "something" quoting, or 'something' quoting, or
`something' quoting (but in none of the cases it used colors consistently
with the rest of the compiler).

Now, libcpp diagnostics is always emitted using a callback,
pfile->cb.diagnostic.  On the gcc/ side, this callback is initialized with
genmatch.cc:  cb->diagnostic = diagnostic_cb;
c-family/c-opts.cc:  cb->diagnostic = c_cpp_diagnostic;
fortran/cpp.cc:  cb->diagnostic = cb_cpp_diagnostic;
where the latter two just use diagnostic_report_diagnostic, so actually
support all the gcc_diag stuff, only the genmatch.cc case didn't.

So, the following patch changes genmatch.cc to use pp_format* instead
of vfprintf so that it supports the gcc_diag formatting (pretty-print.o
unfortunately has various dependencies, so had to link genmatch with
libcommon.a libbacktrace.a and tweak Makefile.in so that there are no
circular dependencies) and marks the libcpp diagnostic routines as
gcc_diag rather than printf.  That change resulted in hundreds of
-Wformat-diag new warnings (most of them useful and resulting IMHO in
better diagnostics), so the rest of the patch is changing the format
strings to make -Wformat-diag happy and adjusting the testsuite for
the differences in how is the diagnostic reformatted.

Dunno if some out of GCC tree projects use libcpp, that case would
make it harder because one couldn't use vfprintf in the diagnostic
callback anymore, but there is always David's libdiagnostic which could
be used for that purpose IMHO.

2024-10-12  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* include/cpplib.h (ATTRIBUTE_CPP_PPDIAG): Define.
	(struct cpp_callbacks): Use ATTRIBUTE_CPP_PPDIAG instead of
	ATTRIBUTE_FPTR_PRINTF on diagnostic callback.
	(cpp_error, cpp_warning, cpp_pedwarning, cpp_warning_syshdr): Use
	ATTRIBUTE_CPP_PPDIAG (3, 4) instead of ATTRIBUTE_PRINTF_3.
	(cpp_warning_at, cpp_pedwarning_at): Use ATTRIBUTE_CPP_PPDIAG (4, 5)
	instead of ATTRIBUTE_PRINTF_4.
	(cpp_error_with_line, cpp_warning_with_line, cpp_pedwarning_with_line,
	cpp_warning_with_line_syshdr): Use ATTRIBUTE_CPP_PPDIAG (5, 6)
	instead of ATTRIBUTE_PRINTF_5.
	(cpp_error_at): Use ATTRIBUTE_CPP_PPDIAG (4, 5) instead of
	ATTRIBUTE_PRINTF_4.
	* Makefile.in (po/$(PACKAGE).pot): Use --language=GCC-source rather
	than --language=c.
	* errors.cc (cpp_diagnostic_at, cpp_diagnostic,
	cpp_diagnostic_with_line): Use ATTRIBUTE_CPP_PPDIAG instead of
	-ATTRIBUTE_FPTR_PRINTF.
	* charset.cc (cpp_host_to_exec_charset, _cpp_valid_ucn, convert_hex,
	convert_oct, convert_escape): Fix up -Wformat-diag warnings.
	(cpp_interpret_string_ranges, count_source_chars): Use
	ATTRIBUTE_CPP_PPDIAG instead of ATTRIBUTE_FPTR_PRINTF.
	(narrow_str_to_charconst): Fix up -Wformat-diag warnings.
	* directives.cc (check_eol_1, directive_diagnostics, lex_macro_node,
	do_undef, glue_header_name, parse_include, do_include_common,
	do_include_next, _cpp_parse_embed_params, do_embed, read_flag,
	do_line, do_linemarker, register_pragma_1, do_pragma_once,
	do_pragma_push_macro, do_pragma_pop_macro, do_pragma_poison,
	do_pragma_system_header, do_pragma_warning_or_error, _cpp_do__Pragma,
	do_else, do_elif, do_endif, parse_answer, do_assert,
	cpp_define_unused): Likewise.
	* expr.cc (cpp_classify_number, parse_defined, eval_token,
	_cpp_parse_expr, reduce, check_promotion): Likewise.
	* files.cc (_cpp_find_file, finish_base64_embed,
	_cpp_pop_file_buffer): Likewise.
	* init.cc (sanity_checks): Likewise.
	* lex.cc (_cpp_process_line_notes, maybe_warn_bidi_on_char,
	_cpp_warn_invalid_utf8, _cpp_skip_block_comment,
	warn_about_normalization, forms_identifier_p, maybe_va_opt_error,
	identifier_diagnostics_on_lex, cpp_maybe_module_directive): Likewise.
	* macro.cc (class vaopt_state, builtin_has_include_1,
	builtin_has_include, builtin_has_embed, _cpp_warn_if_unused_macro,
	_cpp_builtin_macro_text, builtin_macro, stringify_arg,
	_cpp_arguments_ok, collect_args, enter_macro_context,
	_cpp_save_parameter, parse_params, create_iso_definition,
	_cpp_create_definition, check_trad_stringification): Likewise.
	* pch.cc (cpp_valid_state): Likewise.
	* traditional.cc (_cpp_scan_out_logical_line, recursive_macro):
	Likewise.
gcc/
	* Makefile.in (generated_files): Remove {gimple,generic}-match*.
	(generated_match_files): New variable.  Add a dependency of
	$(filter-out $(OBJS-libcommon),$(ALL_HOST_OBJS)) files on those.
	(build/genmatch$(build_exeext)): Depend on and link against
	libcommon.a and $(LIBBACKTRACE).
	* genmatch.cc: Include pretty-print.h and input.h.
	(ggc_internal_cleared_alloc, ggc_free): Remove.
	(fatal): New function.
	(line_table): Remove.
	(linemap_client_expand_location_to_spelling_point): Remove.
	(diagnostic_cb): Use gcc_diag rather than printf format.  Use
	pp_format_verbatim on a temporary pretty_printer instead of
	vfprintf.
	(fatal_at, warning_at): Use gcc_diag rather than printf format.
	(output_line_directive): Rename location_hash to loc_hash.
	(parser::eat_ident, parser::parse_operation, parser::parse_expr,
	parser::parse_pattern, parser::finish_match_operand): Fix up
	-Wformat-diag warnings.
gcc/c-family/
	* c-lex.cc (c_common_has_attribute,
	c_common_lex_availability_macro): Fix up -Wformat-diag warnings.
gcc/testsuite/
	* c-c++-common/cpp/counter-2.c: Adjust expected diagnostics for
	libcpp diagnostic formatting changes.
	* c-c++-common/cpp/embed-3.c: Likewise.
	* c-c++-common/cpp/embed-4.c: Likewise.
	* c-c++-common/cpp/embed-16.c: Likewise.
	* c-c++-common/cpp/embed-18.c: Likewise.
	* c-c++-common/cpp/eof-2.c: Likewise.
	* c-c++-common/cpp/eof-3.c: Likewise.
	* c-c++-common/cpp/fmax-include-depth.c: Likewise.
	* c-c++-common/cpp/has-builtin.c: Likewise.
	* c-c++-common/cpp/line-2.c: Likewise.
	* c-c++-common/cpp/line-3.c: Likewise.
	* c-c++-common/cpp/macro-arg-count-1.c: Likewise.
	* c-c++-common/cpp/macro-arg-count-2.c: Likewise.
	* c-c++-common/cpp/macro-ranges.c: Likewise.
	* c-c++-common/cpp/named-universal-char-escape-4.c: Likewise.
	* c-c++-common/cpp/named-universal-char-escape-5.c: Likewise.
	* c-c++-common/cpp/pr88974.c: Likewise.
	* c-c++-common/cpp/va-opt-error.c: Likewise.
	* c-c++-common/cpp/va-opt-pedantic.c: Likewise.
	* c-c++-common/cpp/Wheader-guard-2.c: Likewise.
	* c-c++-common/cpp/Wheader-guard-3.c: Likewise.
	* c-c++-common/cpp/Winvalid-utf8-1.c: Likewise.
	* c-c++-common/cpp/Winvalid-utf8-2.c: Likewise.
	* c-c++-common/cpp/Winvalid-utf8-3.c: Likewise.
	* c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-1.c:
	Likewise.
	* c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-3.c:
	Likewise.
	* c-c++-common/pr68833-3.c: Likewise.
	* c-c++-common/raw-string-directive-1.c: Likewise.
	* gcc.dg/analyzer/named-constants-Wunused-macros.c: Likewise.
	* gcc.dg/binary-constants-4.c: Likewise.
	* gcc.dg/builtin-redefine.c: Likewise.
	* gcc.dg/cpp/19951025-1.c: Likewise.
	* gcc.dg/cpp/c11-warning-1.c: Likewise.
	* gcc.dg/cpp/c11-warning-2.c: Likewise.
	* gcc.dg/cpp/c11-warning-3.c: Likewise.
	* gcc.dg/cpp/c23-elifdef-2.c: Likewise.
	* gcc.dg/cpp/c23-warning-2.c: Likewise.
	* gcc.dg/cpp/embed-2.c: Likewise.
	* gcc.dg/cpp/embed-3.c: Likewise.
	* gcc.dg/cpp/embed-4.c: Likewise.
	* gcc.dg/cpp/expr.c: Likewise.
	* gcc.dg/cpp/gnu11-elifdef-2.c: Likewise.
	* gcc.dg/cpp/gnu11-elifdef-3.c: Likewise.
	* gcc.dg/cpp/gnu11-elifdef-4.c: Likewise.
	* gcc.dg/cpp/gnu11-warning-1.c: Likewise.
	* gcc.dg/cpp/gnu11-warning-2.c: Likewise.
	* gcc.dg/cpp/gnu11-warning-3.c: Likewise.
	* gcc.dg/cpp/gnu23-warning-2.c: Likewise.
	* gcc.dg/cpp/include6.c: Likewise.
	* gcc.dg/cpp/pr35322.c: Likewise.
	* gcc.dg/cpp/tr-warn6.c: Likewise.
	* gcc.dg/cpp/undef2.c: Likewise.
	* gcc.dg/cpp/warn-comments.c: Likewise.
	* gcc.dg/cpp/warn-comments-2.c: Likewise.
	* gcc.dg/cpp/warn-comments-3.c: Likewise.
	* gcc.dg/cpp/warn-cxx-compat.c: Likewise.
	* gcc.dg/cpp/warn-cxx-compat-2.c: Likewise.
	* gcc.dg/cpp/warn-deprecated.c: Likewise.
	* gcc.dg/cpp/warn-deprecated-2.c: Likewise.
	* gcc.dg/cpp/warn-long-long.c: Likewise.
	* gcc.dg/cpp/warn-long-long-2.c: Likewise.
	* gcc.dg/cpp/warn-normalized-1.c: Likewise.
	* gcc.dg/cpp/warn-normalized-2.c: Likewise.
	* gcc.dg/cpp/warn-normalized-3.c: Likewise.
	* gcc.dg/cpp/warn-normalized-4-bytes.c: Likewise.
	* gcc.dg/cpp/warn-normalized-4-unicode.c: Likewise.
	* gcc.dg/cpp/warn-redefined.c: Likewise.
	* gcc.dg/cpp/warn-redefined-2.c: Likewise.
	* gcc.dg/cpp/warn-traditional.c: Likewise.
	* gcc.dg/cpp/warn-traditional-2.c: Likewise.
	* gcc.dg/cpp/warn-trigraphs-1.c: Likewise.
	* gcc.dg/cpp/warn-trigraphs-2.c: Likewise.
	* gcc.dg/cpp/warn-trigraphs-3.c: Likewise.
	* gcc.dg/cpp/warn-trigraphs-4.c: Likewise.
	* gcc.dg/cpp/warn-undef.c: Likewise.
	* gcc.dg/cpp/warn-undef-2.c: Likewise.
	* gcc.dg/cpp/warn-unused-macros.c: Likewise.
	* gcc.dg/cpp/warn-unused-macros-2.c: Likewise.
	* gcc.dg/pch/counter-2.c: Likewise.
	* g++.dg/cpp0x/udlit-error1.C: Likewise.
	* g++.dg/cpp23/named-universal-char-escape1.C: Likewise.
	* g++.dg/cpp23/named-universal-char-escape2.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-1.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-2.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-3.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-4.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-5.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-6.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-7.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-8.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-9.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-10.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-11.C: Likewise.
	* g++.dg/cpp23/Winvalid-utf8-12.C: Likewise.
	* g++.dg/cpp/elifdef-3.C: Likewise.
	* g++.dg/cpp/elifdef-5.C: Likewise.
	* g++.dg/cpp/elifdef-6.C: Likewise.
	* g++.dg/cpp/elifdef-7.C: Likewise.
	* g++.dg/cpp/embed-1.C: Likewise.
	* g++.dg/cpp/embed-2.C: Likewise.
	* g++.dg/cpp/pedantic-errors.C: Likewise.
	* g++.dg/cpp/warning-1.C: Likewise.
	* g++.dg/cpp/warning-2.C: Likewise.
	* g++.dg/ext/bitint1.C: Likewise.
	* g++.dg/ext/bitint2.C: Likewise.