Skip to content
Snippets Groups Projects
Commit 1d8de1e9 authored by Konstantinos Eleftheriou's avatar Konstantinos Eleftheriou Committed by Philipp Tomsich
Browse files

Add target-independent store forwarding avoidance pass


This pass detects cases of expensive store forwarding and tries to
avoid them by reordering the stores and using suitable bit insertion
sequences.  For example it can transform this:

     strb    w2, [x1, 1]
     ldr     x0, [x1]      # Expensive store forwarding to larger load.

To:

     ldr     x0, [x1]
     strb    w2, [x1]
     bfi     x0, x2, 0, 8

Assembly like this can appear with bitfields or type punning / unions.
On stress-ng when running the cpu-union microbenchmark the following
speedups have been observed.

  Neoverse-N1:      +29.4%
  Intel Coffeelake: +13.1%
  AMD 5950X:        +17.5%

The transformation is rejected on cases that cause store_bit_field to
generate subreg expressions on different register classes.  Files
avoid-store-forwarding-4.c and avoid-store-forwarding-5.c contain such
cases and have been marked as XFAIL.

Due to biasing of its operands in store_bit_field, there is a special
handling for machines with BITS_BIG_ENDIAN != BYTES_BIG_ENDIAN. The
need for this was exosed by an issue exposed on the H8 architecture,
which uses big-endian ordering, but BITS_BIG_ENDIAN is false. In that
case, the START parameter of store_bit_field needs to be calculated
from the end of the destination register.

gcc/ChangeLog:

	* Makefile.in (OBJS): Add avoid-store-forwarding.o.
	* common.opt (favoid-store-forwarding): New option.
	* common.opt.urls: Regenerate.
	* doc/invoke.texi: New param store-forwarding-max-distance.
	* doc/passes.texi: Document new pass.
	* doc/tm.texi: Regenerate.
	* doc/tm.texi.in: Document new pass.
	* params.opt (store-forwarding-max-distance): New param.
	* passes.def: Add pass_rtl_avoid_store_forwarding before
	pass_early_remat.
	* target.def (avoid_store_forwarding_p): New DEFHOOK.
	* target.h (struct store_fwd_info): Declare.
	* targhooks.cc (default_avoid_store_forwarding_p): New function.
	* targhooks.h (default_avoid_store_forwarding_p): Declare.
	* tree-pass.h (make_pass_rtl_avoid_store_forwarding): Declare.
	* avoid-store-forwarding.cc: New file.
	* avoid-store-forwarding.h: New file.
	* timevar.def (TV_AVOID_STORE_FORWARDING): New timevar.

gcc/testsuite/ChangeLog:

	* gcc.target/aarch64/avoid-store-forwarding-1.c: New test.
	* gcc.target/aarch64/avoid-store-forwarding-2.c: New test.
	* gcc.target/aarch64/avoid-store-forwarding-3.c: New test.
	* gcc.target/aarch64/avoid-store-forwarding-4.c: New test.
	* gcc.target/aarch64/avoid-store-forwarding-5.c: New test.
	* gcc.target/x86_64/abi/callabi/avoid-store-forwarding-1.c: New test.
	* gcc.target/x86_64/abi/callabi/avoid-store-forwarding-2.c: New test.

Co-authored-by: default avatarPhilipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: default avatarPhilipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: default avatarKonstantinos Eleftheriou <konstantinos.eleftheriou@vrull.eu>
parent ba4cf2e2
No related branches found
No related tags found
No related merge requests found
Showing
with 953 additions and 0 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment