vect: Add maskload else value support.
This patch adds an else operand to vectorized masked load calls. The current implementation adds else-value arguments to the respective target-querying functions that is used to supply the vectorizer with the proper else value. We query the target for its supported else operand and uses that for the maskload call. If necessary, i.e. if the mode has padding bits and if the else operand is nonzero, a VEC_COND enforcing a zero else value is emitted. gcc/ChangeLog: * optabs-query.cc (supports_vec_convert_optab_p): Return icode. (get_supported_else_val): Return supported else value for optab's operand at index. (supports_vec_gather_load_p): Add else argument. (supports_vec_scatter_store_p): Ditto. * optabs-query.h (supports_vec_gather_load_p): Ditto. (get_supported_else_val): Ditto. * optabs-tree.cc (target_supports_mask_load_store_p): Ditto. (can_vec_mask_load_store_p): Ditto. (target_supports_len_load_store_p): Ditto. (get_len_load_store_mode): Ditto. * optabs-tree.h (target_supports_mask_load_store_p): Ditto. (can_vec_mask_load_store_p): Ditto. * tree-vect-data-refs.cc (vect_lanes_optab_supported_p): Ditto. (vect_gather_scatter_fn_p): Ditto. (vect_check_gather_scatter): Ditto. (vect_load_lanes_supported): Ditto. * tree-vect-patterns.cc (vect_recog_gather_scatter_pattern): Ditto. * tree-vect-slp.cc (vect_get_operand_map): Adjust indices for else operand. (vect_slp_analyze_node_operations): Skip undefined else operand. * tree-vect-stmts.cc (exist_non_indexing_operands_for_use_p): Add else operand handling. (vect_get_vec_defs_for_operand): Handle undefined else operand. (check_load_store_for_partial_vectors): Add else argument. (vect_truncate_gather_scatter_offset): Ditto. (vect_use_strided_gather_scatters_p): Ditto. (get_group_load_store_type): Ditto. (get_load_store_type): Ditto. (vect_get_mask_load_else): Ditto. (vect_get_else_val_from_tree): Ditto. (vect_build_one_gather_load_call): Add zero else operand. (vectorizable_load): Use else operand. * tree-vectorizer.h (vect_gather_scatter_fn_p): Add else argument. (vect_load_lanes_supported): Ditto. (vect_get_mask_load_else): Ditto. (vect_get_else_val_from_tree): Ditto.
Showing
- gcc/optabs-query.cc 51 additions, 19 deletionsgcc/optabs-query.cc
- gcc/optabs-query.h 2 additions, 1 deletiongcc/optabs-query.h
- gcc/optabs-tree.cc 49 additions, 17 deletionsgcc/optabs-tree.cc
- gcc/optabs-tree.h 6 additions, 2 deletionsgcc/optabs-tree.h
- gcc/tree-vect-data-refs.cc 52 additions, 22 deletionsgcc/tree-vect-data-refs.cc
- gcc/tree-vect-patterns.cc 10 additions, 2 deletionsgcc/tree-vect-patterns.cc
- gcc/tree-vect-slp.cc 17 additions, 5 deletionsgcc/tree-vect-slp.cc
- gcc/tree-vect-stmts.cc 272 additions, 54 deletionsgcc/tree-vect-stmts.cc
- gcc/tree-vectorizer.h 7 additions, 3 deletionsgcc/tree-vectorizer.h
Loading
Please register or sign in to comment