Skip to content
Snippets Groups Projects
  • Dodji Seketeli's avatar
    828a7f76
    PR preprocessor/53229 - Fix diagnostics location when pasting tokens · 828a7f76
    Dodji Seketeli authored
    As stated in the audit trail of this problem report, consider this
    test case:
    
        $ cat test.c
    	 1	struct x {
    	 2	  int i;
    	 3	};
    	 4	struct x x;
    	 5
    	 6	#define TEST(X) x.##X
    	 7
    	 8	void foo (void)
    	 9	{
    	10	  TEST(i) = 0;
    	11	}
        $
    
        $ cc1 -quiet test.c
        test.c: In function 'foo':
        test.c:10:1: error: pasting "." and "i" does not give a valid preprocessing token
           TEST(i) = 0;
         ^
        $
    
    So, when pasting tokens, the error diagnostic uses the global and
    imprecise input_location variable, leading to an imprecise output.
    
    To properly fix this, I think libcpp should keep the token of the
    pasting operator '##', instead of representing it with flag on the LHS
    operand's token.  That way, it could use its location.  Doing that
    would be quite intrusive though.  So this patch just uses the location
    of the LHS of the pasting operator, for now.  It's IMHO better than
    the current situation.
    
    The patch makes paste_tokens take a location parameter that is used in
    the diagnostics.  This change can still be useful later when we can
    use the location of the pasting operator, because paste_tokens will
    just be passed the new, more precise location.
    
    Incidentally, it appeared that when getting tokens from within
    preprocessor directives (like what is done in gcc.dg/cpp/paste12.c),
    with -ftrack-macro-expansion disabled, the location of the expansion
    point of macros was being lost because
    cpp_reader::set_invocation_location wasn't being properly set.  It's
    because when cpp_get_token_1 calls enter_macro_context, there is a
    little period of time between the beginning of that later function and
    when the macro is really pushed (and thus when the macro is really
    expanded) where we wrongly consider that we are not expanding the
    macro because macro_of_context is still NULL.  In that period of time,
    in the occurrences of indirect recursive calls to cpp_get_token_1,
    this later function wrongly sets cpp_reader::invocation_location
    because cpp_reader::set_invocation_location is not being properly set.
    
    To avoid that confusion the patch does away with
    cpp_reader::set_invocation_location and introduces a new flag
    cpp_reader::about_to_expand_macro_p that is set in the small time
    interval exposed earlier.  A new in_macro_expansion_p is introduced as
    well, so that cpp_get_token_1 can now accurately detect when we are in
    the process of expanding a macro, and thus correctly collect the
    location of the expansion point.
    
    People seem to like screenshots.
    
    Thus, after the patch, we now have:
    
        $ cc1 -quiet test.c
        test.c: In function 'foo':
        test.c:6:18: error: pasting "." and "i" does not give a valid preprocessing token
         #define TEST(X) x.##X
    		      ^
        test.c:10:3: note: in expansion of macro 'TEST'
           TEST(i) = 0;
           ^
        $
    
    Bootstrapped and tested on x86_64-unknown-linux-gnu against trunk.
    
    libcpp/
    
    	PR preprocessor/53229
    	* internal.h (cpp_reader::set_invocation_location): Remove.
    	(cpp_reader::about_to_expand_macro_p): New member flag.
    	* directives.c (do_pragma):  Remove Kludge as
    	pfile->set_invocation_location is no more.
    	* macro.c (cpp_get_token_1): Do away with the use of
    	cpp_reader::set_invocation_location.  Just collect the macro
    	expansion point when we are about to expand the top-most macro.
    	Do not override cpp_reader::about_to_expand_macro_p.
    	This fixes gcc.dg/cpp/paste12.c by making get_token_no_padding
    	properly handle locations of expansion points.
    	(cpp_get_token_with_location): Adjust, as
    	cpp_reader::set_invocation_location is no more.
    	(paste_tokens): Take a virtual location parameter for
    	the LHS of the pasting operator.  Use it in diagnostics.  Update
    	comments.
    	(paste_all_tokens): Tighten the assert.  Propagate the location of
    	the expansion point when no virtual locations are available.
    	Pass the virtual location to paste_tokens.
    	(in_macro_expansion_p): New static function.
    	(enter_macro_context): Set the cpp_reader::about_to_expand_macro_p
    	flag until we really start expanding the macro.
    
    gcc/testsuite/
    
    	PR preprocessor/53229
    	* gcc.dg/cpp/paste6.c: Force to run without
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste8.c: Likewise.
    	* gcc.dg/cpp/paste8-2.c: New test, like paste8.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste12.c: Force to run without
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste12-2.c: New test, like paste12.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste13.c: Likewise.
    	* gcc.dg/cpp/paste14.c: Likewise.
    	* gcc.dg/cpp/paste14-2.c: New test, like paste14.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste18.c: New test.
    
    From-SVN: r187945
    828a7f76
    History
    PR preprocessor/53229 - Fix diagnostics location when pasting tokens
    Dodji Seketeli authored
    As stated in the audit trail of this problem report, consider this
    test case:
    
        $ cat test.c
    	 1	struct x {
    	 2	  int i;
    	 3	};
    	 4	struct x x;
    	 5
    	 6	#define TEST(X) x.##X
    	 7
    	 8	void foo (void)
    	 9	{
    	10	  TEST(i) = 0;
    	11	}
        $
    
        $ cc1 -quiet test.c
        test.c: In function 'foo':
        test.c:10:1: error: pasting "." and "i" does not give a valid preprocessing token
           TEST(i) = 0;
         ^
        $
    
    So, when pasting tokens, the error diagnostic uses the global and
    imprecise input_location variable, leading to an imprecise output.
    
    To properly fix this, I think libcpp should keep the token of the
    pasting operator '##', instead of representing it with flag on the LHS
    operand's token.  That way, it could use its location.  Doing that
    would be quite intrusive though.  So this patch just uses the location
    of the LHS of the pasting operator, for now.  It's IMHO better than
    the current situation.
    
    The patch makes paste_tokens take a location parameter that is used in
    the diagnostics.  This change can still be useful later when we can
    use the location of the pasting operator, because paste_tokens will
    just be passed the new, more precise location.
    
    Incidentally, it appeared that when getting tokens from within
    preprocessor directives (like what is done in gcc.dg/cpp/paste12.c),
    with -ftrack-macro-expansion disabled, the location of the expansion
    point of macros was being lost because
    cpp_reader::set_invocation_location wasn't being properly set.  It's
    because when cpp_get_token_1 calls enter_macro_context, there is a
    little period of time between the beginning of that later function and
    when the macro is really pushed (and thus when the macro is really
    expanded) where we wrongly consider that we are not expanding the
    macro because macro_of_context is still NULL.  In that period of time,
    in the occurrences of indirect recursive calls to cpp_get_token_1,
    this later function wrongly sets cpp_reader::invocation_location
    because cpp_reader::set_invocation_location is not being properly set.
    
    To avoid that confusion the patch does away with
    cpp_reader::set_invocation_location and introduces a new flag
    cpp_reader::about_to_expand_macro_p that is set in the small time
    interval exposed earlier.  A new in_macro_expansion_p is introduced as
    well, so that cpp_get_token_1 can now accurately detect when we are in
    the process of expanding a macro, and thus correctly collect the
    location of the expansion point.
    
    People seem to like screenshots.
    
    Thus, after the patch, we now have:
    
        $ cc1 -quiet test.c
        test.c: In function 'foo':
        test.c:6:18: error: pasting "." and "i" does not give a valid preprocessing token
         #define TEST(X) x.##X
    		      ^
        test.c:10:3: note: in expansion of macro 'TEST'
           TEST(i) = 0;
           ^
        $
    
    Bootstrapped and tested on x86_64-unknown-linux-gnu against trunk.
    
    libcpp/
    
    	PR preprocessor/53229
    	* internal.h (cpp_reader::set_invocation_location): Remove.
    	(cpp_reader::about_to_expand_macro_p): New member flag.
    	* directives.c (do_pragma):  Remove Kludge as
    	pfile->set_invocation_location is no more.
    	* macro.c (cpp_get_token_1): Do away with the use of
    	cpp_reader::set_invocation_location.  Just collect the macro
    	expansion point when we are about to expand the top-most macro.
    	Do not override cpp_reader::about_to_expand_macro_p.
    	This fixes gcc.dg/cpp/paste12.c by making get_token_no_padding
    	properly handle locations of expansion points.
    	(cpp_get_token_with_location): Adjust, as
    	cpp_reader::set_invocation_location is no more.
    	(paste_tokens): Take a virtual location parameter for
    	the LHS of the pasting operator.  Use it in diagnostics.  Update
    	comments.
    	(paste_all_tokens): Tighten the assert.  Propagate the location of
    	the expansion point when no virtual locations are available.
    	Pass the virtual location to paste_tokens.
    	(in_macro_expansion_p): New static function.
    	(enter_macro_context): Set the cpp_reader::about_to_expand_macro_p
    	flag until we really start expanding the macro.
    
    gcc/testsuite/
    
    	PR preprocessor/53229
    	* gcc.dg/cpp/paste6.c: Force to run without
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste8.c: Likewise.
    	* gcc.dg/cpp/paste8-2.c: New test, like paste8.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste12.c: Force to run without
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste12-2.c: New test, like paste12.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste13.c: Likewise.
    	* gcc.dg/cpp/paste14.c: Likewise.
    	* gcc.dg/cpp/paste14-2.c: New test, like paste14.c but run with
    	-ftrack-macro-expansion.
    	* gcc.dg/cpp/paste18.c: New test.
    
    From-SVN: r187945