GCC Middle and Back End API Reference
|
Go to the source code of this file.
Macros | |
#define | PROB_VERY_UNLIKELY (REG_BR_PROB_BASE / 2000 - 1) |
#define | PROB_EVEN (REG_BR_PROB_BASE / 2) |
#define | PROB_VERY_LIKELY (REG_BR_PROB_BASE - PROB_VERY_UNLIKELY) |
#define | PROB_ALWAYS (REG_BR_PROB_BASE) |
#define | PROB_UNLIKELY (REG_BR_PROB_BASE / 5 - 1) |
#define | PROB_LIKELY (REG_BR_PROB_BASE - PROB_UNLIKELY) |
#define | PROB_UNINITIALIZED (-1) |
#define | DEF_PREDICTOR(ENUM, NAME, HITRATE, FLAGS) |
Enumerations | |
enum | br_predictor { DEF_PREDICTOR } |
enum | prediction { NOT_TAKEN , TAKEN } |
Variables | |
profile_probability | split_branch_probability |
#define PROB_ALWAYS (REG_BR_PROB_BASE) |
Referenced by expr_expected_value_1().
#define PROB_EVEN (REG_BR_PROB_BASE / 2) |
Referenced by combine_predictions_for_bb(), and combine_predictions_for_insn().
#define PROB_LIKELY (REG_BR_PROB_BASE - PROB_UNLIKELY) |
#define PROB_UNINITIALIZED (-1) |
Referenced by predict_insn_def().
#define PROB_UNLIKELY (REG_BR_PROB_BASE / 5 - 1) |
#define PROB_VERY_LIKELY (REG_BR_PROB_BASE - PROB_VERY_UNLIKELY) |
Referenced by combine_predictions_for_bb().
#define PROB_VERY_UNLIKELY (REG_BR_PROB_BASE / 2000 - 1) |
Definitions for branch prediction routines in the GNU compiler. Copyright (C) 2001-2024 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
Random guesstimation given names. PROB_VERY_UNLIKELY should be small enough so basic block predicted by it gets below HOT_BB_FREQUENCY_FRACTION.
Referenced by combine_predictions_for_bb().
enum br_predictor |
enum prediction |
|
extern |
Add REG_BR_PROB note to JUMP with PROB.
References add_int_reg_note(), find_reg_note(), gcc_checking_assert, JUMP_P, and profile_probability::to_reg_br_prob_note().
Referenced by asan_clear_shadow(), combine_predictions_for_insn(), compare_and_jump_seq(), doloop_modify(), emit_cmp_and_jump_insn_1(), emit_likely_jump_insn(), emit_unlikely_jump_insn(), expand_addsub_overflow(), expand_mul_overflow(), expand_neg_overflow(), and try_split().
|
extern |
Return the optimization type that should be used for basic block BB.
References optimize_bb_for_speed_p(), OPTIMIZE_FOR_SIZE, and OPTIMIZE_FOR_SPEED.
Referenced by convert_mult_to_fma(), tree_switch_conversion::switch_conversion::is_exp_index_transform_viable(), reassociate_bb(), and replacement_internal_fn().
Same predicate as edge_probability_reliable_p, working on notes.
References profile_probability::from_reg_br_prob_note(), gcc_assert, profile_probability::probably_reliable_p(), REG_NOTE_KIND, and XINT.
|
extern |
Build PREDICT_EXPR.
References build1(), build_int_cst(), integer_type_node, SET_PREDICT_EXPR_OUTCOME, and void_type_node.
|
extern |
Decide whether function is hot, cold or unlikely executed.
References cfun, count, current_function_decl, DECL_ATTRIBUTES, DECL_NAME, DECL_STATIC_CONSTRUCTOR, DECL_STATIC_DESTRUCTOR, ECF_NORETURN, ENTRY_BLOCK_PTR_FOR_FN, flags_from_decl_or_type(), FOR_EACH_BB_FN, cgraph_node::frequency, cgraph_node::get(), lookup_attribute(), MAIN_NAME_P, maybe_hot_bb_p(), NODE_FREQUENCY_EXECUTED_ONCE, NODE_FREQUENCY_HOT, NODE_FREQUENCY_NORMAL, NODE_FREQUENCY_UNLIKELY_EXECUTED, NULL, cgraph_node::only_called_at_exit, cgraph_node::only_called_at_startup, probably_never_executed_bb_p(), warn_function_cold(), and profile_count::zero().
Referenced by autofdo::auto_profile(), branch_prob(), estimate_bb_frequencies(), execute_fixup_cfg(), and ipa_merge_profiles().
|
extern |
Set RTL expansion to default mode (i.e. when profile info is not known).
References crtl.
Referenced by combine_instructions(), computation_cost(), gimplify_body(), init_expmed(), init_set_costs(), peephole2_optimize(), prepare_function_start(), split_all_insns(), and thread_prologue_and_epilogue_insns().
|
extern |
Same predicate as above, working on edges.
|
extern |
Return true if function is likely to be expensive, so there is no point to optimize performance of prologue, epilogue or do inlining at the expense of code size growth. THRESHOLD is the limit of number of instructions function can execute at average to be still considered not expensive.
References active_insn_p(), cfun, basic_block_def::count, count, dump_file, ENTRY_BLOCK_PTR_FOR_FN, FOR_BB_INSNS, FOR_EACH_BB_FN, basic_block_def::index, profile_count::initialized_p(), and profile_count::zero().
Force edge E to be cold. If IMPOSSIBLE is true, for edge to have count and probability 0 otherwise keep low probability to represent possible error in a guess. This is used i.e. in case we predict loop to likely iterate given number of times but we are not 100% sure. This function locally updates profile without attempt to keep global consistency which cannot be reached in full generality without full profile rebuild from probabilities alone. Doing so is not necessarily a good idea because frequencies and counts may be more realistic then probabilities. In some cases (such as for elimination of early exits during full loop unrolling) the caller can ensure that profile will get consistent afterwards.
References profile_probability::always(), cfun, current_ir_type(), dump_file, dump_flags, ENTRY_BLOCK_PTR_FOR_FN, FOR_EACH_EDGE, force_edge_cold(), gsi_end_p(), gsi_next(), gsi_start_bb(), gsi_stmt(), IR_GIMPLE, maybe_hot_bb_p(), MIN, profile_probability::never(), set_edge_probability_and_rescale_others(), single_pred_edge(), single_pred_p(), stmt_can_terminate_bb_p(), TDF_DETAILS, profile_count::to_frequency(), update_br_prob_note(), profile_probability::very_unlikely(), and profile_count::zero().
Referenced by duplicate_loop_body_to_header_edge(), force_edge_cold(), isolate_path(), and try_unroll_loop_completely().
|
extern |
Return the optimization type that should be used for function FUN.
References OPTIMIZE_FOR_SIZE, OPTIMIZE_FOR_SPEED, and optimize_function_for_speed_p().
|
extern |
Determine the threshold for hot BB counts.
References dump_file, profile_count::max_count, min_count, PRId64, profile_info, set_hot_bb_threshold(), and gcov_summary::sum_max.
Referenced by dump_function_to_file(), ipa_propagate_frequency(), maybe_hot_count_p(), and output_profile_summary().
|
extern |
Predict edge E with the given PROBABILITY.
References bb_predictions, cfun, EDGE_COUNT, ENTRY_BLOCK_PTR_FOR_FN, edge_prediction::ep_next, edge_prediction::ep_probability, and i.
|
extern |
Return true if the one of outgoing edges is already predicted by PREDICTOR.
References bb_predictions, edge_prediction::ep_next, and i.
|
extern |
Set edge->probability for each successor edge of BB.
References BB_END, bb_estimate_probability_locally(), and combine_predictions_for_insn().
Referenced by compute_outgoing_frequencies().
|
extern |
In the case of COMDAT routines, multiple object files will contain the same function and the linker will select one for the binary. In that case all the other copies from the profile instrument binary will be missing profile counts. Look for cases where this happened, due to non-zero call counts going to 0-count functions, and drop the profile to guessed so that we can use the estimated probabilities and avoid optimizing only for size. The other case where the profile may be missing is when the routine is not going to be emitted to the object file, e.g. for "extern template" class methods. Those will be marked DECL_EXTERNAL. Emit a warning in all other cases of non-zero calls to 0-count functions.
References cgraph_edge::callee, cgraph_node::callees, cgraph_edge::caller, cgraph_node::callers, function::cfg, cgraph_edge::count, cgraph_node::count, symtab_node::decl, DECL_COMDAT, DECL_EXTERNAL, DECL_STRUCT_FUNCTION, drop_profile(), FOR_EACH_DEFINED_FUNCTION, profile_count::initialized_p(), profile_count::ipa(), cgraph_edge::next_caller, profile_count::nonzero_p(), profile_info, PROFILE_READ, profile_status_for_fn, gcov_summary::runs, cgraph_node::tp_first_run, worklist, and profile_count::zero().
Referenced by tree_profiling().
|
extern |
Return the optimization type that should be used for the current instruction.
References OPTIMIZE_FOR_SIZE, OPTIMIZE_FOR_SPEED, and optimize_insn_for_speed_p().
Referenced by expand_sfix_optab().
|
extern |
Invert all branch predictions or probability notes in the INSN. This needs to be done each time we invert the condition used by the jump.
References profile_probability::from_reg_br_prob_note(), GEN_INT, INTVAL, profile_probability::invert(), REG_BR_PROB_BASE, REG_NOTE_KIND, REG_NOTES, profile_probability::to_reg_br_prob_note(), XEXP, and XINT.
Referenced by redirect_jump_2().
|
extern |
Return true if basic block BB of function FUN can be CPU intensive and should thus be optimized for maximum performance.
References basic_block_def::count, gcc_checking_assert, and maybe_hot_count_p().
Referenced by compute_function_frequency(), dump_bb_info(), force_edge_cold(), optimize_bb_for_size_p(), and rtl_profile_for_bb().
|
extern |
Return TRUE if COUNT is considered to be hot in function FUN.
References count, function::decl, ENTRY_BLOCK_PTR_FOR_FN, cgraph_node::frequency, cgraph_node::get(), get_hot_bb_threshold(), MAX, NODE_FREQUENCY_EXECUTED_ONCE, NODE_FREQUENCY_HOT, NODE_FREQUENCY_UNLIKELY_EXECUTED, PROFILE_ABSENT, profile_info, PROFILE_READ, profile_status_for_fn, gcov_summary::runs, and profile_count::zero().
Referenced by afdo_callsite_hot_enough_for_early_inline(), drop_profile(), maybe_hot_bb_p(), maybe_hot_edge_p(), and cgraph_edge::maybe_hot_p().
Return true if edge E can be CPU intensive and should thus be optimized for maximum performance.
References cfun, and maybe_hot_count_p().
Referenced by optimize_edge_for_size_p(), and rtl_profile_for_edge().
|
extern |
Return TRUE if basic block BB should be optimized for size.
References cfun, count, maybe_hot_bb_p(), optimize_function_for_size_p(), OPTIMIZE_SIZE_BALANCED, OPTIMIZE_SIZE_MAX, and profile_count::zero().
Referenced by tree_switch_conversion::switch_conversion::array_value_type(), coalesce_cost_bb(), compute_alignments(), create_coalesce_list_for_region(), eliminate_partially_redundant_load(), eliminate_partially_redundant_loads(), expand_complex_multiplication(), gimple_divmod_fixed_value_transform(), gimple_mod_pow2_value_transform(), gimple_mod_subtract_transform(), gimple_stringops_transform(), ignore_bb_p(), optimize_bb_for_speed_p(), optimize_loop_for_size_p(), try_crossjump_bb(), try_head_merge_bb(), and vect_recog_divmod_pattern().
|
extern |
Return TRUE if basic block BB should be optimized for speed.
References optimize_bb_for_size_p().
Referenced by attempt_change(), bb_ok_for_noce_convert_multiple_sets(), bb_optimization_type(), bb_valid_for_noce_process_p(), find_candidates_dom_walker::before_dom_children(), vect_optimize_slp_pass::build_vertices(), calculate_elim_costs_all_insns(), cheap_bb_rtx_cost_p(), combine_instructions(), compute_alignments(), copy_bb_p(), create_new_invariant(), cse_extended_basic_block(), find_if_case_1(), find_if_case_2(), get_computation_cost(), get_preferred_alternatives(), get_preferred_alternatives(), get_stored_val(), gimple_expand_builtin_cabs(), gimple_expand_builtin_pow(), gimple_fold_builtin_strcat(), move2add_use_add2_insn(), move2add_use_add3_insn(), noce_find_if_block(), noce_process_if_block(), note_reg_elim_costly(), optimize_cands_for_speed_p(), optimize_loop_for_speed_p(), optimize_range_tests_to_bit_test(), outgoing_edges_match(), back_threader_profitability::possibly_profitable_path_p(), reload_cse_move2add(), reload_cse_simplify_operands(), reload_cse_simplify_set(), replace_reciprocal(), replace_reciprocal_squares(), replace_uncond_cands_and_profitable_phis(), should_replace_address(), simplify_builtin_call(), try_replace_in_use(), try_replace_reg(), and value_replacement().
|
extern |
Return TRUE if edge E should be optimized for size.
References cfun, maybe_hot_edge_p(), optimize_function_for_size_p(), OPTIMIZE_SIZE_BALANCED, OPTIMIZE_SIZE_MAX, and unlikely_executed_edge_p().
Referenced by coalesce_cost_edge(), and optimize_edge_for_speed_p().
Return TRUE if edge E should be optimized for speed.
References optimize_edge_for_size_p().
Referenced by connect_traces(), do_pre_partial_partial_insertion(), do_pre_regular_insertion(), find_traces_1_round(), and back_threader_profitability::profitable_path_p().
|
extern |
Return true if function FUN should always be optimized for size.
References function::decl, cgraph_node::get(), cgraph_node::optimize_for_size_p(), OPTIMIZE_SIZE_MAX, and OPTIMIZE_SIZE_NO.
Referenced by bb_to_key(), better_edge_p(), compute_alignments(), connect_better_edge_p(), connect_traces(), decide_copy_try_finally(), find_traces_1_round(), gimple_fold_builtin_fputs(), gimple_fold_builtin_stpcpy(), gimple_fold_builtin_strchr(), gimple_fold_builtin_strcpy(), inherit_reload_reg(), fwd_jt_path_registry::mark_threaded_blocks(), optimize_bb_for_size_p(), optimize_edge_for_size_p(), optimize_function_for_speed_p(), optimize_insn_for_size_p(), relax_delay_slots(), sra_get_max_scalarization_size(), and want_to_gcse_p().
Return true if function FUN should always be optimized for speed.
References optimize_function_for_size_p().
Referenced by assemble_start_function(), cheap_bb_rtx_cost_p(), decompose_multiword_subregs(), fold_truth_andor_1(), function_optimization_type(), gimple_expand_builtin_pow(), gimple_expand_builtin_powi(), gimplify_init_constructor(), optimize_range_tests(), reorder_basic_blocks_simple(), simplify_context::simplify_binary_operation_1(), use_shrink_wrapping_separate(), and want_to_gcse_p().
|
extern |
Return TRUE if the current function is optimized for size.
References cfun, optimize_function_for_size_p(), and OPTIMIZE_SIZE_BALANCED.
Referenced by expand_clrsb_using_clz(), tree_switch_conversion::jump_table_cluster::find_jump_tables(), inline_expand_builtin_bytecmp(), and optimize_insn_for_speed_p().
|
extern |
Return TRUE if the current function is optimized for speed.
References optimize_insn_for_size_p().
Referenced by avoid_expensive_constant(), can_do_by_pieces(), can_store_by_pieces(), choose_mult_variant(), clear_storage_hints(), compress_float_constant(), convert_mode_scalar(), do_jump(), tree_switch_conversion::bit_test_cluster::emit(), emit_case_dispatch_table(), emit_conditional_move(), emit_move_complex(), emit_store_flag(), emit_store_flag_int(), expand_abs_nojump(), expand_binop(), expand_ccmp_expr_1(), expand_DIVMOD(), expand_divmod(), expand_expr_divmod(), expand_expr_real_2(), expand_gimple_cond(), expand_mul_overflow(), expand_mult(), expand_one_cmpl_abs_nojump(), expand_POPCOUNT(), expand_sdiv_pow2(), expand_shift_1(), expand_smod_pow2(), expand_unop(), expand_widening_mult(), expmed_mult_highpart(), expmed_mult_highpart_optab(), insn_optimization_type(), maybe_optimize_mod_cmp(), maybe_optimize_pow2p_mod_cmp(), output_asm_name(), precompute_register_parameters(), prefer_and_bit_test(), prepare_cmp_insn(), store_by_pieces(), and synth_mult().
|
extern |
Return TRUE if LOOP should be optimized for size.
References loop::header, and optimize_bb_for_size_p().
Referenced by decide_unrolling(), optimize_loop_nest_for_size_p(), tree_ssa_split_loops(), and tree_ssa_unswitch_loops().
Return TRUE if LOOP should be optimized for speed.
References loop::header, and optimize_bb_for_speed_p().
Referenced by add_iv_candidate_for_use(), adjust_setup_cost(), doloop_optimize(), loop_distribution::execute(), move_single_loop_invariants(), optimize_loop_nest_for_speed_p(), split_paths(), tree_predictive_commoning(), tree_ssa_iv_optimize_loop(), and try_peel_loop().
|
extern |
Return TRUE if nest rooted at LOOP should be optimized for size.
References loop::inner, loop_outer(), MIN, loop::next, optimize_loop_for_size_p(), and OPTIMIZE_SIZE_NO.
Referenced by loop_prefetch_arrays(), parallelize_loops(), and tree_loop_unroll_and_jam().
Return TRUE if nest rooted at LOOP should be optimized for speed.
References loop::inner, loop_outer(), loop::next, and optimize_loop_for_speed_p().
Referenced by tree_unroll_loops_completely_1(), try_vectorize_loop(), vect_enhance_data_refs_alignment(), and vect_mark_for_runtime_alias_test().
|
extern |
Predict edge E by given predictor if possible.
References predict_edge(), REG_BR_PROB_BASE, and TAKEN.
Referenced by maybe_predict_edge(), predict_iv_comparison(), tree_estimate_probability_bb(), and tree_predict_by_opcode().
|
extern |
Predict insn by given predictor.
References gcc_assert, predict_insn(), PROB_UNINITIALIZED, REG_BR_PROB_BASE, and TAKEN.
Referenced by bb_estimate_probability_locally(), prepare_call_address(), and stack_protect_epilogue().
Return true if edge E is likely to be well predictable by branch predictor.
References REG_BR_PROB_BASE.
Referenced by default_max_noce_ifcvt_seq_cost(), find_if_case_1(), and find_if_case_2().
|
extern |
References predictor_info::name.
Referenced by dump_generic_node(), and pp_gimple_stmt_1().
|
extern |
Return true if basic block BB of function FUN is probably never executed.
References basic_block_def::count, and probably_never_executed().
Referenced by bb_to_key(), check_bb_profile(), compute_function_frequency(), dump_bb_info(), find_rarely_executed_basic_blocks_and_crossing_edges(), and push_to_next_round_p().
Return true if edge E of function FUN is probably never executed.
References probably_never_executed(), and unlikely_executed_edge_p().
Referenced by check_bb_profile(), find_rarely_executed_basic_blocks_and_crossing_edges(), back_threader_profitability::possibly_profitable_path_p(), back_threader_profitability::profitable_path_p(), and single_likely_exit().
|
extern |
All basic blocks that are reachable only from unlikely basic blocks are unlikely.
References basic_block_def::aux, cfun, basic_block_def::count, count, dump_file, dump_flags, ENTRY_BLOCK_PTR_FOR_FN, FOR_ALL_BB_FN, FOR_EACH_EDGE, basic_block_def::index, NULL, basic_block_def::succs, TDF_DETAILS, worklist, and profile_count::zero().
Referenced by determine_unlikely_bbs(), and find_rarely_executed_basic_blocks_and_crossing_edges().
|
extern |
Rebuild function frequencies. Passes are in general expected to maintain profile by hand, however in some cases this is not possible: for example when inlining several functions with loops freuqencies might run out of scale and thus needs to be recomputed.
References cfun, connect_infinite_loops_to_exit(), basic_block_def::count, count, profile_count::differs_from_p(), dump_file, ENTRY_BLOCK_PTR_FOR_FN, estimate_bb_frequencies(), FOR_BB_BETWEEN, FOR_EACH_EDGE, basic_block_def::index, profile_count::initialized_p(), loop_optimizer_finalize(), loop_optimizer_init(), LOOPS_HAVE_MARKED_IRREDUCIBLE_REGIONS, NULL, basic_block_def::preds, PROFILE_ABSENT, profile_status_for_fn, remove_fake_exit_edges(), profile_count::uninitialized(), and profile_count::zero().
Referenced by tree_function_versioning().
|
extern |
Remove all predictions on given basic block that are attached to edge E.
References bb_predictions, filter_predictions(), and not_equal_edge_p().
Referenced by remove_edge_raw().
|
extern |
Perform a dry run of the branch prediction pass and report comparsion of the predicted and real profile into the dump file.
References cfun, dump_file, dump_flags, flow_loops_dump(), loop_optimizer_finalize(), loop_optimizer_init(), LOOPS_NORMAL, NULL, number_of_loops(), scev_finalize(), scev_initialize(), TDF_DETAILS, and tree_estimate_probability().
Referenced by branch_prob().
|
extern |
Predict edge E with given probability if possible.
References any_condjump_p(), BB_END, predict_insn(), and REG_BR_PROB_BASE.
|
extern |
|
extern |
Set RTL expansion for BB profile.
References cfun, crtl, and maybe_hot_bb_p().
Referenced by combine_instructions(), construct_exit_block(), expand_gimple_basic_block(), peephole2_optimize(), simplify_builtin_call(), split_all_insns(), and thread_prologue_and_epilogue_insns().
|
extern |
Set RTL expansion for edge profile.
References crtl, and maybe_hot_edge_p().
|
extern |
Set the threshold for hot BB counts.
References min_count.
Referenced by get_hot_bb_threshold(), input_profile_summary(), and ipa_profile().
|
extern |
Predict branch probabilities and estimate profile of the tree CFG. This function can be called from the loop optimizers to recompute the profile information. If DRY_RUN is set, do not modify CFG and only produce dump files.
References assert_is_empty(), bb_predictions, calculate_dominance_info(), CDI_POST_DOMINATORS, cfun, combine_predictions_for_bb(), connect_infinite_loops_to_exit(), CP_SIMPLE_PREHEADERS, create_preheaders(), determine_unlikely_bbs(), estimate_bb_frequencies(), FOR_EACH_BB_FN, free_dominance_info(), NULL, number_of_loops(), predict_loops(), PROFILE_READ, profile_status_for_fn, record_loop_exits(), remove_fake_exit_edges(), ssa_expected_value, tree_bb_level_predictions(), and tree_estimate_probability_bb().
Referenced by report_predictor_hitrates().
|
extern |
Set edge->probability for each successor edge of BB.
References assert_is_empty(), bb_predictions, combine_predictions_for_bb(), NULL, ssa_expected_value, and tree_estimate_probability_bb().
Referenced by gimple_find_sub_bbs().
|
extern |
Convert counts measured by profile driven feedback to frequencies. Return nonzero iff there was any nonzero execution count.
References cfun, basic_block_def::count, ENTRY_BLOCK_PTR_FOR_FN, FOR_BB_BETWEEN, profile_count::ipa(), profile_count::max(), profile_count::nonzero_p(), NULL, and profile_count::uninitialized().
Referenced by autofdo::afdo_annotate_cfg(), compute_branch_probabilities(), execute_fixup_cfg(), expand_omp_taskreg(), expand_thunk(), input_function(), ipa_merge_profiles(), optimize_inline_calls(), and tree_function_versioning().
|
extern |
|
extern |
In emit-rtl.cc.
Probability of the conditional branch currently proceeded by try_split.
Referenced by init_emit_regs(), and try_split().