Skip to content
Snippets Groups Projects
  • Joseph Myers's avatar
    8f51cf38
    preprocessor: Fix pp-number lexing of digit separators [PR83873, PR97604] · 8f51cf38
    Joseph Myers authored
    When the preprocessor lexes preprocessing numbers in lex_number, it
    accepts digit separators in more cases than actually permitted in
    pp-numbers by the standard syntax.
    
    One thing this accepts is adjacent digit separators; there is some
    code to reject those later, but as noted in bug 83873 it fails to
    cover the case of adjacent digit separators within a floating-point
    exponent.  Accepting adjacent digit separators only results in a
    missing diagnostic, not in valid code being rejected or being accepted
    with incorrect semantics, because the correct lexing in such a case
    would have '' start the following preprocessing tokens, and no valid
    preprocessing token starts '' while ' isn't valid on its own as a
    preprocessing token either.  So this patch fixes that case by moving
    the error for adjacent digit separators to lex_number (allowing a more
    specific diagnostic than if '' were excluded from the pp-number
    completely).
    
    Other cases inappropriately accepted involve digit separators before
    '.', 'e+', 'e-', 'p+' or 'p-' (or corresponding uppercase variants).
    In those cases, as shown by the test digit-sep-pp-number.C added, this
    can result in valid code being wrongly rejected as a result of too
    many characters being included in the pp-number.  So this case is
    fixed by terminating the pp-number at the correct character according
    to the standard.  That test also covers the case where a digit
    separator was followed by an identifier-nondigit that is not a
    nondigit (e.g. a UCN); that case was already handled correctly.
    
    Bootstrapped with no regressions for x86_64-pc-linux-gnu.
    
    libcpp/
    	PR c++/83873
    	PR preprocessor/97604
    	* lex.c (lex_number): Reject adjacent digit separators here.  Do
    	not allow digit separators before '.' or an exponent with sign.
    	* expr.c (cpp_classify_number): Do not check for adjacent digit
    	separators here.
    
    gcc/testsuite/
    	PR c++/83873
    	PR preprocessor/97604
    	* g++.dg/cpp1y/digit-sep-neg-2.C,
    	g++.dg/cpp1y/digit-sep-pp-number.C: New tests.
    	* g++.dg/cpp1y/digit-sep-line-neg.C, g++.dg/cpp1y/digit-sep-neg.C:
    	Adjust expected messages.
    8f51cf38
    History
    preprocessor: Fix pp-number lexing of digit separators [PR83873, PR97604]
    Joseph Myers authored
    When the preprocessor lexes preprocessing numbers in lex_number, it
    accepts digit separators in more cases than actually permitted in
    pp-numbers by the standard syntax.
    
    One thing this accepts is adjacent digit separators; there is some
    code to reject those later, but as noted in bug 83873 it fails to
    cover the case of adjacent digit separators within a floating-point
    exponent.  Accepting adjacent digit separators only results in a
    missing diagnostic, not in valid code being rejected or being accepted
    with incorrect semantics, because the correct lexing in such a case
    would have '' start the following preprocessing tokens, and no valid
    preprocessing token starts '' while ' isn't valid on its own as a
    preprocessing token either.  So this patch fixes that case by moving
    the error for adjacent digit separators to lex_number (allowing a more
    specific diagnostic than if '' were excluded from the pp-number
    completely).
    
    Other cases inappropriately accepted involve digit separators before
    '.', 'e+', 'e-', 'p+' or 'p-' (or corresponding uppercase variants).
    In those cases, as shown by the test digit-sep-pp-number.C added, this
    can result in valid code being wrongly rejected as a result of too
    many characters being included in the pp-number.  So this case is
    fixed by terminating the pp-number at the correct character according
    to the standard.  That test also covers the case where a digit
    separator was followed by an identifier-nondigit that is not a
    nondigit (e.g. a UCN); that case was already handled correctly.
    
    Bootstrapped with no regressions for x86_64-pc-linux-gnu.
    
    libcpp/
    	PR c++/83873
    	PR preprocessor/97604
    	* lex.c (lex_number): Reject adjacent digit separators here.  Do
    	not allow digit separators before '.' or an exponent with sign.
    	* expr.c (cpp_classify_number): Do not check for adjacent digit
    	separators here.
    
    gcc/testsuite/
    	PR c++/83873
    	PR preprocessor/97604
    	* g++.dg/cpp1y/digit-sep-neg-2.C,
    	g++.dg/cpp1y/digit-sep-pp-number.C: New tests.
    	* g++.dg/cpp1y/digit-sep-line-neg.C, g++.dg/cpp1y/digit-sep-neg.C:
    	Adjust expected messages.