-
- Downloads
RISC-V: RISC-V: Support gather_load/scatter RVV auto-vectorization
This patch fully support gather_load/scatter_store: 1. Support single-rgroup on both RV32/RV64. 2. Support indexed element width can be same as or smaller than Pmode. 3. Support VLA SLP with gather/scatter. 4. Fully tested all gather/scatter with LMUL = M1/M2/M4/M8 both VLA and VLS. 5. Fix bug of handling (subreg:SI (const_poly_int:DI)) 6. Fix bug on vec_perm which is used by gather/scatter SLP. All kinds of GATHER/SCATTER are normalized into LEN_MASK_*. We fully supported these 4 kinds of gather/scatter: 1. LEN_MASK_GATHER_LOAD/LEN_MASK_SCATTER_STORE with dummy length and dummy mask (Full vector). 2. LEN_MASK_GATHER_LOAD/LEN_MASK_SCATTER_STORE with dummy length and real mask. 3. LEN_MASK_GATHER_LOAD/LEN_MASK_SCATTER_STORE with real length and dummy mask. 4. LEN_MASK_GATHER_LOAD/LEN_MASK_SCATTER_STORE with real length and real mask. Base on the disscussions with Richards, we don't lower vlse/vsse in RISC-V backend for strided load/store. Instead, we leave it to the middle-end to handle that. Regression is pass ok for trunk ? gcc/ChangeLog: * config/riscv/autovec.md (len_mask_gather_load<VNX1_QHSD:mode><VNX1_QHSDI:mode>): New pattern. (len_mask_gather_load<VNX2_QHSD:mode><VNX2_QHSDI:mode>): Ditto. (len_mask_gather_load<VNX4_QHSD:mode><VNX4_QHSDI:mode>): Ditto. (len_mask_gather_load<VNX8_QHSD:mode><VNX8_QHSDI:mode>): Ditto. (len_mask_gather_load<VNX16_QHSD:mode><VNX16_QHSDI:mode>): Ditto. (len_mask_gather_load<VNX32_QHS:mode><VNX32_QHSI:mode>): Ditto. (len_mask_gather_load<VNX64_QH:mode><VNX64_QHI:mode>): Ditto. (len_mask_gather_load<mode><mode>): Ditto. (len_mask_scatter_store<VNX1_QHSD:mode><VNX1_QHSDI:mode>): Ditto. (len_mask_scatter_store<VNX2_QHSD:mode><VNX2_QHSDI:mode>): Ditto. (len_mask_scatter_store<VNX4_QHSD:mode><VNX4_QHSDI:mode>): Ditto. (len_mask_scatter_store<VNX8_QHSD:mode><VNX8_QHSDI:mode>): Ditto. (len_mask_scatter_store<VNX16_QHSD:mode><VNX16_QHSDI:mode>): Ditto. (len_mask_scatter_store<VNX32_QHS:mode><VNX32_QHSI:mode>): Ditto. (len_mask_scatter_store<VNX64_QH:mode><VNX64_QHI:mode>): Ditto. (len_mask_scatter_store<mode><mode>): Ditto. * config/riscv/predicates.md (const_1_operand): New predicate. (vector_gs_scale_operand_16): Ditto. (vector_gs_scale_operand_32): Ditto. (vector_gs_scale_operand_64): Ditto. (vector_gs_extension_operand): Ditto. (vector_gs_scale_operand_16_rv32): Ditto. (vector_gs_scale_operand_32_rv32): Ditto. * config/riscv/riscv-protos.h (enum insn_type): Add gather/scatter. (expand_gather_scatter): New function. * config/riscv/riscv-v.cc (gen_const_vector_dup): Add gather/scatter. (emit_vlmax_masked_store_insn): New function. (emit_nonvlmax_masked_store_insn): Ditto. (modulo_sel_indices): Ditto. (expand_vec_perm): Fix SLP for gather/scatter. (prepare_gather_scatter): New function. (expand_gather_scatter): Ditto. * config/riscv/riscv.cc (riscv_legitimize_move): Fix bug of (subreg:SI (DI CONST_POLY_INT)). * config/riscv/vector-iterators.md: Add gather/scatter. * config/riscv/vector.md (vec_duplicate<mode>): Use "@" instead. (@vec_duplicate<mode>): Ditto. (@pred_indexed_<order>store<VNX16_QHS:mode><VNX16_QHSDI:mode>): Fix name. (@pred_indexed_<order>store<VNX16_QHSD:mode><VNX16_QHSDI:mode>): Ditto. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/rvv.exp: Add gather/scatter tests. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-11.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-12.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-11.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-12.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-11.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-11.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_run-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_run-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-10.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-3.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-4.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-5.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-6.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-7.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-8.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/scatter_store_run-9.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_load-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_load-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_store-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_store-2.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-1.c: New test. * gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-2.c: New test.
Showing
- gcc/config/riscv/autovec.md 256 additions, 0 deletionsgcc/config/riscv/autovec.md
- gcc/config/riscv/predicates.md 32 additions, 1 deletiongcc/config/riscv/predicates.md
- gcc/config/riscv/riscv-protos.h 3 additions, 0 deletionsgcc/config/riscv/riscv-protos.h
- gcc/config/riscv/riscv-v.cc 253 additions, 33 deletionsgcc/config/riscv/riscv-v.cc
- gcc/config/riscv/riscv.cc 9 additions, 2 deletionsgcc/config/riscv/riscv.cc
- gcc/config/riscv/vector-iterators.md 99 additions, 19 deletionsgcc/config/riscv/vector-iterators.md
- gcc/config/riscv/vector.md 23 additions, 7 deletionsgcc/config/riscv/vector.md
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-1.c 38 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-1.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-10.c 35 additions, 0 deletions....target/riscv/rvv/autovec/gather-scatter/gather_load-10.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-11.c 32 additions, 0 deletions....target/riscv/rvv/autovec/gather-scatter/gather_load-11.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-12.c 112 additions, 0 deletions....target/riscv/rvv/autovec/gather-scatter/gather_load-12.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-2.c 38 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-2.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-3.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-3.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-4.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-4.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-5.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-5.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-6.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-6.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-7.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-7.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-8.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-8.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load-9.c 35 additions, 0 deletions...c.target/riscv/rvv/autovec/gather-scatter/gather_load-9.c
- gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/gather_load_run-1.c 41 additions, 0 deletions...rget/riscv/rvv/autovec/gather-scatter/gather_load_run-1.c
Loading
Please register or sign in to comment