This patch adds similar optimizations to the C++ FE as have been
implemented earlier in the C FE.
The libcpp hunk enables use of CPP_EMBED token even for C++, not just
C; the preprocessor guarantees there is always a CPP_NUMBER CPP_COMMA
before CPP_EMBED and CPP_COMMA CPP_NUMBER after it which simplifies
parsing (unless #embed is more than 2GB, in that case it could be
CPP_NUMBER CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED CPP_COMMA CPP_EMBED
CPP_COMMA CPP_NUMBER etc. with each CPP_EMBED covering at most INT_MAX
bytes).
Similarly to the C patch, this patch parses it into RAW_DATA_CST tree
in the braced initializers (and from there peels into INTEGER_CSTs unless
it is an initializer of an std::byte array or integral array with CHAR_BIT
element precision), parses CPP_EMBED in cp_parser_expression into just
the last INTEGER_CST in it because I think users don't need millions of
-Wunused-value warnings because they did useless
  int a = (
  #embed "megabyte.dat"
  );
and so most of the inner INTEGER_CSTs would be there just for the warning,
and in the rest of contexts like template argument list, function argument
list, attribute argument list, ...) parse it into a sequence of INTEGER_CSTs
(I wrote a range/iterator classes to simplify that).

My dumb
cat embed-11.c
constexpr unsigned char a[] = {
  #embed "cc1plus"
};
const unsigned char *b = a;
testcase where cc1plus is 492329008 bytes long when configured
--enable-checking=yes,rtl,extra against recent binutils with .base64 gas
support results in:
time ./xg++ -B ./ -S -O2 embed-11.c

real    0m4.350s
user    0m2.427s
sys     0m0.830s
time ./xg++ -B ./ -c -O2 embed-11.c

real    0m6.932s
user    0m6.034s
sys     0m0.888s
(compared to running out of memory or very long compilation).
On a shorter inclusion,
cat embed-12.c
constexpr unsigned char a[] = {
  #embed "xg++"
};
const unsigned char *b = a;
where xg++ is 15225904 bytes long, this takes using GCC with the #embed
patchset except for this patch:
time ~/src/gcc/obj36/gcc/xg++ -B ~/src/gcc/obj36/gcc/ -S -O2 embed-12.c

real    0m33.190s
user    0m32.327s
sys     0m0.790s
and with this patch:
time ./xg++ -B ./ -S -O2 embed-12.c

real    0m0.118s
user    0m0.090s
sys     0m0.028s

The patch doesn't change anything on what the first patch in the series
introduces even for C++, namely that #embed is expanded (actually or as if)
into a sequence of literals like
127,69,76,70,2,1,1,3,0,0,0,0,0,0,0,0,2,0,62,0,1,0,0,0,80,211,64,0,0,0,0,0,64,0,0,0,0,0,0,0,8,253
and so each element has int type.
That is how I believe it is in C23, and the different versions of the
C++ P1967 paper specified there some casts, P1967R12 in particular
"Otherwise, the integral constant expression is the value of std::fgetc’s return is cast
to unsigned char."
but please see
https://github.com/llvm/llvm-project/pull/97274#issuecomment-2230929277
comment and whether we really want the preprocessor to preprocess it for
C++ as (or as-if)
static_cast<unsigned char>(127),static_cast<unsigned char>(69),static_cast<unsigned char>(76),static_cast<unsigned char>(70),static_cast<unsigned char>(2),...
i.e. 9 tokens per byte rather than 2, or
(unsigned char)127,(unsigned char)69,...
or
((unsigned char)127),((unsigned char)69),...
etc.
Without a literal suffix for unsigned char constant literals it is horrible,
plus the incompatibility between C and C++.  Sure, we could use the magic
form more often for C++ to save the size and do the 9 or how many tokens
form only for the boundary constants and use #embed "." __gnu__::__base64__("...")
for what is in between if there are at least 2 tokens inside of it.
E.g. (unsigned char)127 vs. static_cast<unsigned char>(127) behaves
differently if there is constexpr long long p[] = { ... };
...
  #embed __FILE__
[p]

2024-12-06  Jakub Jelinek  <jakub@redhat.com>

libcpp/
	* files.cc (finish_embed): Use CPP_EMBED even for C++.
gcc/
	* tree.h (RAW_DATA_UCHAR_ELT, RAW_DATA_SCHAR_ELT): Define.
gcc/cp/ChangeLog:
	* cp-tree.h (class raw_data_iterator): New type.
	(class raw_data_range): New type.
	* parser.cc (cp_parser_postfix_open_square_expression): Handle
	parsing of CPP_EMBED.
	(cp_parser_parenthesized_expression_list): Likewise.  Use
	cp_lexer_next_token_is.
	(cp_parser_expression): Handle parsing of CPP_EMBED.
	(cp_parser_template_argument_list): Likewise.
	(cp_parser_initializer_list): Likewise.
	(cp_parser_oacc_clause_tile): Likewise.
	(cp_parser_omp_tile_sizes): Likewise.
	* pt.cc (tsubst_expr): Handle RAW_DATA_CST.
	* constexpr.cc (reduced_constant_expression_p): Likewise.
	(raw_data_cst_elt): New function.
	(find_array_ctor_elt): Handle RAW_DATA_CST.
	(cxx_eval_array_reference): Likewise.
	* typeck2.cc (digest_init_r): Emit -Wnarrowing and/or -Wconversion
	diagnostics.
	(process_init_constructor_array): Handle RAW_DATA_CST.
	* decl.cc (maybe_deduce_size_from_array_init): Likewise.
	(is_direct_enum_init): Fail for RAW_DATA_CST.
	(cp_maybe_split_raw_data): New function.
	(consume_init): New function.
	(reshape_init_array_1): Add VECTOR_P argument.  Handle RAW_DATA_CST.
	(reshape_init_array): Adjust reshape_init_array_1 caller.
	(reshape_init_vector): Likewise.
	(reshape_init_class): Handle RAW_DATA_CST.
	(reshape_init_r): Likewise.
gcc/testsuite/
	* c-c++-common/cpp/embed-22.c: New test.
	* c-c++-common/cpp/embed-23.c: New test.
	* g++.dg/cpp/embed-4.C: New test.
	* g++.dg/cpp/embed-5.C: New test.
	* g++.dg/cpp/embed-6.C: New test.
	* g++.dg/cpp/embed-7.C: New test.
	* g++.dg/cpp/embed-8.C: New test.
	* g++.dg/cpp/embed-9.C: New test.
	* g++.dg/cpp/embed-10.C: New test.
	* g++.dg/cpp/embed-11.C: New test.
	* g++.dg/cpp/embed-12.C: New test.
	* g++.dg/cpp/embed-13.C: New test.
	* g++.dg/cpp/embed-14.C: New test.