Go to the source code of this file.
Macros | |
| #define | DLTHE_FLAG_UPDATE_FREQ |
| #define | DLTHE_RECORD_COPY_NUMBER |
| #define | DLTHE_FLAG_COMPLETTE_PEEL |
| #define | DLTHE_FLAG_FLAT_PROFILE |
Enumerations | |
| enum | { CP_SIMPLE_PREHEADERS = 1 , CP_FALLTHRU_PREHEADERS = 2 } |
Variables | |
| edge | mfb_kj_edge |
Macro Definition Documentation
◆ DLTHE_FLAG_COMPLETTE_PEEL
| #define DLTHE_FLAG_COMPLETTE_PEEL |
Referenced by duplicate_loop_body_to_header_edge(), and try_unroll_loop_completely().
◆ DLTHE_FLAG_FLAT_PROFILE
| #define DLTHE_FLAG_FLAT_PROFILE |
Referenced by duplicate_loop_body_to_header_edge(), tree_transform_and_unroll_loop(), and unroll_loop_constant_iterations().
◆ DLTHE_FLAG_UPDATE_FREQ
| #define DLTHE_FLAG_UPDATE_FREQ |
Referenced by duplicate_loop_body_to_header_edge(), tree_transform_and_unroll_loop(), try_peel_loop(), try_unroll_loop_completely(), unroll_loop_constant_iterations(), unroll_loop_runtime_iterations(), and unroll_loop_stupid().
◆ DLTHE_RECORD_COPY_NUMBER
| #define DLTHE_RECORD_COPY_NUMBER |
Referenced by duplicate_loop_body_to_header_edge(), unroll_loop_constant_iterations(), unroll_loop_runtime_iterations(), and unroll_loop_stupid().
Enumeration Type Documentation
◆ anonymous enum
Loop manipulation header. Copyright (C) 2014-2024 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
| Enumerator | |
|---|---|
| CP_SIMPLE_PREHEADERS | |
| CP_FALLTHRU_PREHEADERS | |
Function Documentation
◆ add_loop()
Given LOOP structure with filled header and latch, find the body of the corresponding loop and add it to loops tree. Insert the LOOP as a son of outer.
References add_bb_to_loop(), cfun, flow_loop_tree_node_add(), flow_loop_tree_node_remove(), FOR_EACH_EDGE, free(), get_loop_body_with_size(), ggc_alloc(), i, basic_block_def::loop_father, loop_outer(), n_basic_blocks_for_fn, loop::num_nodes, place_new_loop(), remove_bb_from_loops(), and rescan_loop_exit().
Referenced by create_empty_loop_on_edge(), expand_oacc_for(), expand_omp_atomic_pipeline(), expand_omp_for_generic(), expand_omp_for_ordered_loops(), expand_omp_for_static_chunk(), expand_omp_for_static_nochunk(), expand_omp_simd(), expand_omp_taskloop_for_inner(), form_subloop(), loop_version(), simd_clone_adjust(), and tree_optimize_tail_calls_1().
◆ adjust_loop_info_after_peeling()
Update loop estimates after peeling LOOP by NPEEL. If PRECISE is false only likely exists were duplicated and thus do not update any estimates that are supposed to be always reliable.
References loop::any_estimate, loop::any_likely_upper_bound, loop::any_upper_bound, gcc_unreachable, ggc_alloc(), wi::leu_p(), loop::nb_iterations_estimate, loop::nb_iterations_likely_upper_bound, and loop::nb_iterations_upper_bound.
Referenced by try_peel_loop().
◆ can_duplicate_loop_p()
Check whether LOOP's body can be duplicated.
References can_copy_bbs_p(), free(), get_loop_body(), ggc_alloc(), and loop::num_nodes.
Referenced by can_unroll_loop_p(), decide_unrolling(), and parallelize_loops().
◆ copy_loop_info()
Duplicate loop bounds and other information we store about the loop into its duplicate.
References loop::any_estimate, loop::any_likely_upper_bound, loop::any_upper_bound, loop::can_be_parallel, loop::constraints, loop::dont_vectorize, loop::estimate_state, loop::finite_p, loop::force_vectorize, gcc_checking_assert, loop::in_oacc_kernels_region, loop::nb_iterations_estimate, loop::nb_iterations_likely_upper_bound, loop::nb_iterations_upper_bound, loop::owned_clique, loop::safelen, loop::simdlen, loop::unroll, and loop::warned_aggressive_loop_optimizations.
Referenced by copy_loops(), duplicate_loop(), and loop_version().
◆ create_empty_if_region_on_edge()
Creates an if region as shown above. CONDITION is used to create the test for the if. | | ------------- ------------- | | pred_bb | | pred_bb | | ------------- ------------- | | | | | | ENTRY_EDGE | | ENTRY_EDGE V | | ====> ------------- | | | cond_bb | | | | CONDITION | | | ------------- | V / \ | ------------- e_false / \ e_true | | succ_bb | V V | ------------- ----------- ----------- | | false_bb | | true_bb | | ----------- ----------- | \ / | \ / | V V | ------------- | | join_bb | | ------------- | | exit_edge (result) | V | ----------- | | succ_bb | | ----------- |
References CDI_DOMINATORS, force_gimple_operand_gsi(), ggc_alloc(), gimple_build_cond_from_tree(), gsi_insert_after(), gsi_last_bb(), GSI_NEW_STMT, make_edge(), NULL, NULL_TREE, set_immediate_dominator(), single_pred_p(), single_succ_edge(), and split_edge().
◆ create_empty_loop_on_edge()
|
extern |
create_empty_loop_on_edge | | - pred_bb - ------ pred_bb ------ | | | | iv0 = initial_value | | -----|----- ---------|----------- | | ______ | entry_edge | | entry_edge / | | | | ====> | -V---V- loop_header ------------- | V | | iv_before = phi (iv0, iv_after) | | - succ_bb - | ---|----------------------------- | | | | | | ----------- | ---V--- loop_body --------------- | | | iv_after = iv_before + stride | | | | if (iv_before < upper_bound) | | | ---|--------------\-------------- | | | \ exit_e | | V \ | | - loop_latch - V- succ_bb - | | | | | | | | /------------- ----------- | \ ___ / Creates an empty loop as shown above, the IV_BEFORE is the SSA_NAME that is used before the increment of IV. IV_BEFORE should be used for adding code to the body that uses the IV. OUTER is the outer loop in which the new loop should be inserted. Both INITIAL_VALUE and UPPER_BOUND expressions are gimplified and inserted on the loop entry edge. This implies that this function should be used only when the UPPER_BOUND expression is a loop invariant.
References add_loop(), alloc_loop(), CDI_DOMINATORS, create_iv(), profile_probability::even(), force_gimple_operand(), force_gimple_operand_gsi(), gcc_assert, ggc_alloc(), gimple_build_cond(), gimple_cond_lhs(), gimple_cond_set_lhs(), gsi_commit_edge_inserts(), gsi_insert_after(), gsi_insert_seq_on_edge(), gsi_last_bb(), GSI_NEW_STMT, loop::header, loop::latch, loop_preheader_edge(), make_edge(), NULL, NULL_TREE, redirect_edge_succ_nodup(), scale_loop_frequencies(), set_immediate_dominator(), single_exit(), single_pred_edge(), single_succ(), single_succ_edge(), split_block_after_labels(), split_edge(), and update_dominators_in_loop().
◆ create_preheader()
| basic_block create_preheader | ( | class loop * | loop, |
| int | flags ) |
Creates a pre-header for a LOOP. Returns newly created block. Unless CP_SIMPLE_PREHEADERS is set in FLAGS, we only force LOOP to have single entry; otherwise we also force preheader block to have only one successor. When CP_FALLTHRU_PREHEADERS is set in FLAGS, we force the preheader block to be a fallthru predecessor to the loop header and to have only predecessors from outside of the loop. The function also updates dominators.
References BB_END, cfun, CP_FALLTHRU_PREHEADERS, CP_SIMPLE_PREHEADERS, dump_file, EDGE_COMPLEX, EDGE_PRED, ENTRY_BLOCK_PTR_FOR_FN, basic_block_def::flags, FOR_EACH_EDGE, gcc_assert, ggc_alloc(), has_preds_from_loop(), loop::header, JUMP_P, loop::latch, loop_latch_edge(), make_forwarder_block(), mfb_keep_just(), mfb_kj_edge, move_block_after(), NULL, loop::num, basic_block_def::preds, single_succ_edge(), single_succ_p(), and split_edge().
Referenced by create_preheaders(), generate_loops_for_partition(), and fwd_jt_path_registry::thread_through_loop_header().
◆ create_preheaders()
|
extern |
Create preheaders for each loop; for meaning of FLAGS see create_preheader.
References cfun, create_preheader(), current_loops, LOOPS_HAVE_PREHEADERS, and loops_state_set().
Referenced by apply_loop_flags(), and tree_estimate_probability().
◆ duplicate_loop()
Copies copy of LOOP as subloop of TARGET loop, placing newly created loop into loops structure. If AFTER is non-null the new loop is added at AFTER->next, otherwise in front of TARGETs sibling list.
References alloc_loop(), cfun, copy_loop_info(), flow_loop_tree_node_add(), ggc_alloc(), place_new_loop(), and set_loop_copy().
Referenced by copy_loops_to(), duplicate_subloops(), gimple_duplicate_sese_tail(), and slpeel_tree_duplicate_loop_to_edge_cfg().
◆ duplicate_loop_body_to_header_edge()
|
extern |
◆ duplicate_subloops()
Copies structure of subloops of LOOP into TARGET loop, placing newly created loops into loop tree at the end of TARGETs sibling list in the original order.
References duplicate_loop(), duplicate_subloops(), gcc_assert, ggc_alloc(), loop::inner, and loop::next.
Referenced by copy_loops_to(), duplicate_subloops(), gimple_duplicate_sese_tail(), and slpeel_tree_duplicate_loop_to_edge_cfg().
◆ force_single_succ_latches()
Forces all loop latches to have only single successor.
References cfun, find_edge(), gcc_checking_assert, loop::header, loop::latch, LOOPS_HAVE_SIMPLE_LATCHES, loops_state_set(), NULL, single_succ_p(), and split_edge().
Referenced by apply_loop_flags().
◆ loop_version()
| class loop * loop_version | ( | class loop * | loop, |
| void * | cond_expr, | ||
| basic_block * | condition_bb, | ||
| profile_probability | then_prob, | ||
| profile_probability | else_prob, | ||
| profile_probability | then_scale, | ||
| profile_probability | else_scale, | ||
| bool | place_after ) |
Main entry point for Loop Versioning transformation.
This transformation given a condition and a loop, creates
-if (condition) { loop_copy1 } else { loop_copy2 },
where loop_copy1 is the loop transformed in one way, and loop_copy2
is the loop transformed in another way (or unchanged). COND_EXPR
may be a run time test for things that were not resolved by static
analysis (overlapping ranges (anti-aliasing), alignment, etc.).
If non-NULL, CONDITION_BB is set to the basic block containing the
condition.
THEN_PROB is the probability of the then edge of the if. THEN_SCALE
is the ratio by that the frequencies in the original loop should
be scaled. ELSE_SCALE is the ratio by that the frequencies in the
new loop should be scaled.
If PLACE_AFTER is true, we place the new loop after LOOP in the
instruction stream, otherwise it is placed before LOOP.
References add_bb_to_loop(), add_loop(), alloc_loop(), cfg_hook_duplicate_loop_body_to_header_edge(), copy_loop_info(), free(), get_bb_copy(), get_loop_body_in_dom_order(), ggc_alloc(), loop::header, i, loop::latch, loop_outer(), loop_preheader_edge(), loop_redirect_edge(), lv_adjust_loop_entry_edge(), lv_flush_pending_stmts(), move_block_after(), NULL, remove_bb_from_loops(), scale_loop_frequencies(), set_loop_copy(), single_pred_edge(), single_succ_edge(), split_edge(), and update_dominators_in_loop().
Referenced by do_split_loop_on_cond(), gen_parallel_loop(), split_loop(), tree_transform_and_unroll_loop(), tree_unswitch_loop(), vect_loop_versioning(), version_loop_by_alias_check(), and version_loop_for_if_conversion().
◆ mfb_keep_just()
References mfb_kj_edge.
Referenced by create_preheader(), and fwd_jt_path_registry::thread_through_loop_header().
◆ place_new_loop()
Creates place for a new LOOP in loops structure of FN.
References loops_for_fn(), loop::num, number_of_loops(), and vec_safe_push().
Referenced by add_loop(), copy_loops(), duplicate_loop(), and input_cfg().
◆ remove_path()
Removes path beginning at edge E, i.e. remove basic blocks dominated by E and update loop structures and dominators. Return true if we were able to remove the path, false otherwise (and nothing is affected then).
References bitmap_bit_p, bitmap_clear(), bitmap_set_bit, can_remove_branch_p(), cancel_loop_tree(), CDI_DOMINATORS, cfun, dominated_by_p(), EXIT_BLOCK_PTR_FOR_FN, find_path(), first_dom_son(), fix_bb_placements(), fix_loop_placements(), FOR_EACH_EDGE, free(), get_immediate_dominator(), ggc_alloc(), i, basic_block_def::index, iterate_fix_dominators(), last_basic_block_for_fn, loop::latch, basic_block_def::loop_father, loop_outer(), LOOPS_HAVE_MARKED_IRREDUCIBLE_REGIONS, loops_state_satisfies_p(), mark_irreducible_loops(), n_basic_blocks_for_fn, next_dom_son(), remove_bbs(), remove_branch(), single_pred_edge(), single_pred_p(), split_edge(), and unloop().
Referenced by tree_transform_and_unroll_loop(), unloop_loops(), unroll_loop_constant_iterations(), and unroll_loop_runtime_iterations().
◆ scale_dominated_blocks_in_loop()
| void scale_dominated_blocks_in_loop | ( | class loop * | loop, |
| basic_block | bb, | ||
| profile_count | num, | ||
| profile_count | den ) |
Scales the frequencies of all basic blocks in LOOP that are strictly dominated by BB by NUM/DEN.
References profile_count::apply_scale(), CDI_DOMINATORS, basic_block_def::count, first_dom_son(), flow_bb_inside_loop_p(), ggc_alloc(), next_dom_son(), loop::num, worklist, and profile_count::zero().
Referenced by tree_transform_and_unroll_loop(), and update_loop_exit_probability_scale_dom_bbs().
◆ scale_loop_frequencies()
|
extern |
Scale profile of loop by P.
References free(), get_loop_body(), loop::num_nodes, and scale_bbs_frequencies().
Referenced by create_empty_loop_on_edge(), loop_version(), scale_loop_profile(), vect_loop_versioning(), and vect_transform_loop().
◆ scale_loop_profile()
|
extern |
Scale profile in LOOP by P. If ITERATION_BOUND is not -1, scale even further if loop is predicted to iterate too many times. Before caling this function, preheader block profile should be already scaled to final count. This is necessary because loop iterations are determined by comparing header edge count to latch ege count and thus they need to be scaled synchronously.
References profile_probability::always(), basic_block_def::count, profile_probability::dump(), dump_file, dump_flags, expected_loop_iterations_by_profile(), ggc_alloc(), loop::header, loop_count_in(), loop_exit_for_scaling(), loop::num, scale_loop_frequencies(), TDF_DETAILS, sreal::to_double(), profile_count::uninitialized(), and update_loop_exit_probability_scale_dom_bbs().
Referenced by scale_profile_for_vect_loop(), tree_transform_and_unroll_loop(), try_unroll_loop_completely(), and vect_do_peeling().
◆ unloop()
|
extern |
Remove the latch edge of a LOOP and update loops to indicate that the LOOP was removed. After this function, original loop latch will have no successor, which caller is expected to fix somehow. If this may cause the information about irreducible regions to become invalid, IRRED_INVALIDATED is set to true. LOOP_CLOSED_SSA_INVALIDATED, if non-NULL, is a bitmap where we store basic blocks that had non-trivial update on their loop_father.
References add_bb_to_loop(), delete_loop(), fix_bb_placements(), flow_loop_tree_node_add(), flow_loop_tree_node_remove(), free(), get_loop_body(), ggc_alloc(), i, loop::inner, loop::latch, loop_outer(), loop_preheader_edge(), loop::num_nodes, remove_bb_from_loops(), remove_edge(), and single_succ_edge().
Referenced by remove_path(), and unloop_loops().
◆ unloop_loops()
|
extern |
Cancel all fully unrolled loops by putting __builtin_unreachable on the latch edge. We do it after all unrolling since unlooping moves basic blocks across loop boundaries trashing loop closed SSA form as well as SCEV info needed to be intact during unrolling. IRRED_INVALIDATED is used to bookkeep if information about irreducible regions may become invalid as a result of the transformation. LOOP_CLOSED_SSA_INVALIDATED is used to bookkepp the case when we need to go into loop closed SSA form.
References add_bb_to_loop(), BASIC_BLOCK_FOR_FN, CDI_DOMINATORS, cfun, create_basic_block(), current_loops, edges_to_remove, FOR_EACH_VEC_ELT, gcc_assert, ggc_alloc(), gimple_build_builtin_unreachable(), gsi_insert_after(), GSI_NEW_STMT, gsi_start_bb(), i, loop::latch, loop_latch_edge(), loops_to_unloop, loops_to_unloop_nunroll, make_edge(), profile_probability::never(), NULL, remove_exits_and_undefined_stmts(), remove_path(), set_immediate_dominator(), unloop(), and profile_count::zero().
Referenced by canonicalize_induction_variables(), and tree_unroll_loops_completely().
◆ update_exit_probability_after_unrolling()
◆ update_loop_exit_probability_scale_dom_bbs()
| edge update_loop_exit_probability_scale_dom_bbs | ( | class loop * | loop, |
| edge | exit_edge, | ||
| profile_count | desired_count ) |
Assume that loop's entry count and profile up to a given EXIT_EDGE is consistent. Update exit probability so loop exists with PROFILE_COUNT and rescale profile of basic blocks inside loop dominated by EXIT_EDGE->src. This is useful after number of iteraitons of loop has changed. If EXIT_EDGE is NULL, the function will try to identify suitable exit. If DESIRED_COUNT is NULL, loop entry count will be used. In consistent profile usually loop exists as many times as it is entred. Return updated exit if successfull and NULL otherwise.
References cfun, basic_block_def::count, dump_file, dump_flags, FOR_EACH_EDGE, ggc_alloc(), just_once_each_iteration_p(), loop::latch, loop_count_in(), loop_exit_edge_p(), loop_exit_for_scaling(), NULL, loop::num, scale_dominated_blocks_in_loop(), set_edge_probability_and_rescale_others(), single_pred_p(), and TDF_DETAILS.
Referenced by scale_loop_profile(), split_loop(), tree_transform_and_unroll_loop(), and unroll_loop_constant_iterations().
Variable Documentation
◆ mfb_kj_edge
|
extern |
A callback for make_forwarder block, to redirect all edges except for MFB_KJ_EDGE to the entry part. E is the edge for that we should decide whether to redirect it.
Referenced by create_preheader(), mfb_keep_just(), and fwd_jt_path_registry::thread_through_loop_header().
