Go to the source code of this file.
Data Structures | |
| class | edge_growth_cache_entry |
Variables | |
| fast_call_summary< edge_growth_cache_entry *, va_heap > * | edge_growth_cache |
| int | ncalls_inlined |
| int | nfunctions_inlined |
| function_summary< tree * > * | ipa_saved_clone_sources |
Function Documentation
◆ clone_inlined_nodes()
| void clone_inlined_nodes | ( | struct cgraph_edge * | e, |
| bool | duplicate, | ||
| bool | update_original, | ||
| int * | overall_size ) |
E is expected to be an edge being inlined. Clone destination node of the edge and redirect it to the new clone. DUPLICATE is used for bookkeeping on whether we are actually creating new clones or re-using node originally representing out-of-line function call. By default the offline copy is removed, when it appears dead after inlining. UPDATE_ORIGINAL prevents this transformation. If OVERALL_SIZE is non-NULL, the size is updated to reflect the transformation.
References symtab_node::alias, cgraph_edge::callee, cgraph_node::callees, cgraph_edge::caller, cgraph_node::callers, can_remove_node_now_p(), clone_inlined_nodes(), cgraph_node::count, cgraph_edge::count, cgraph_node::create_clone(), symtab_node::decl, symtab_node::definition, dump_callgraph_transformation(), symtab_node::externally_visible, gcc_assert, ggc_alloc(), inline_account_function_p(), cgraph_edge::inline_failed, cgraph_node::inlined_to, ipa_size_summaries, cgraph_node::ipa_transforms_to_apply, master_clone_with_noninline_clones_p(), symtab_node::next, cgraph_edge::next_callee, cgraph_edge::next_caller, nfunctions_inlined, NULL, overall_size, cgraph_edge::redirect_callee(), symtab_node::remove_from_same_comdat_group(), update_noncloned_counts(), cgraph_node::used_as_abstract_origin, and vNULL.
Referenced by clone_inlined_nodes(), inline_call(), and recursive_inlining().
◆ do_estimate_edge_hints()
| ipa_hints do_estimate_edge_hints | ( | struct cgraph_edge * | edge | ) |
Estimate the growth of the caller when inlining EDGE. Only to be called via estimate_edge_size.
References do_estimate_edge_time(), edge_growth_cache, ipa_call_context::estimate_size_and_time(), evaluate_properties_for_edge(), gcc_checking_assert, ggc_alloc(), NULL, simple_edge_hints(), cgraph_node::ultimate_alias_target(), and vNULL.
Referenced by estimate_edge_hints().
◆ do_estimate_edge_size()
| int do_estimate_edge_size | ( | struct cgraph_edge * | edge | ) |
Return estimated callee growth after inlining EDGE. Only to be called via estimate_edge_size.
References do_estimate_edge_time(), edge_growth_cache, ipa_call_context::estimate_size_and_time(), evaluate_properties_for_edge(), gcc_checking_assert, ggc_alloc(), NULL, cgraph_node::ultimate_alias_target(), and vNULL.
Referenced by estimate_edge_size().
◆ do_estimate_edge_time()
| sreal do_estimate_edge_time | ( | struct cgraph_edge * | edge, |
| sreal * | ret_nonspec_time ) |
Estimate the time cost for the caller when inlining EDGE. Only to be called via estimate_edge_time, that handles the caching mechanism. When caching, also update the cache entry. Compute both time and size, since we always need both metrics eventually.
References cgraph_node::count, node_context_cache_entry::ctx, symtab_node::decl, DECL_ATTRIBUTES, ipa_cached_call_context::duplicate_from(), edge_growth_cache, node_context_summary::entry, ipa_call_context::equal_to(), ipa_call_context::estimate_size_and_time(), evaluate_properties_for_edge(), ipa_call_context::exists_p(), gcc_assert, gcc_checking_assert, ggc_alloc(), node_context_cache_entry::hints, edge_growth_cache_entry::hints, INLINE_HINT_known_hot, ipa_call_summaries, ipa_fn_summaries, profile_count::ipa_p(), lookup_attribute(), node_context_cache, node_context_cache_clear, node_context_cache_hit, node_context_cache_miss, node_context_cache_entry::nonspec_time, edge_growth_cache_entry::nonspec_time, NULL, opt_for_fn, ipa_cached_call_context::release(), simple_edge_hints(), node_context_cache_entry::size, edge_growth_cache_entry::size, node_context_cache_entry::time, edge_growth_cache_entry::time, and cgraph_node::ultimate_alias_target().
Referenced by do_estimate_edge_hints(), do_estimate_edge_size(), and estimate_edge_time().
◆ early_inliner()
In ipa-inline.cc
References function::always_inline_functions_inlined, cgraph_node::callees, current_function_decl, symtab_node::decl, DECL_ATTRIBUTES, dump_enabled_p(), dump_file, dump_printf(), early_inline_small_functions(), eni_size_weights, eni_time_weights, estimate_num_insns(), flatten_function(), cgraph_node::get(), ggc_alloc(), inline_always_inline_functions(), ipa_call_summaries, ipa_node_params_sum, ipa_update_overall_fn_summary(), lookup_attribute(), MSG_OPTIMIZED_LOCATIONS, NULL, opt_for_fn, optimize_inline_calls(), symtab_node::remove_all_references(), seen_error(), timevar_pop(), timevar_push(), todo, and symtab_node::verify().
Referenced by autofdo::early_inline().
◆ estimate_edge_growth()
|
inline |
Return estimated callee growth after inlining EDGE.
References ipa_call_summary::call_stmt_size, estimate_edge_size(), gcc_checking_assert, and ipa_call_summaries.
Referenced by can_inline_edge_by_limits_p(), do_estimate_growth_1(), edge_badness(), estimate_size_after_inlining(), inline_call(), inline_small_functions(), want_early_inline_function_p(), and want_inline_small_function_p().
◆ estimate_edge_hints()
|
inline |
Return estimated callee runtime increase after inlining EDGE.
References do_estimate_edge_hints(), edge_growth_cache, edge_growth_cache_entry::hints, and NULL.
Referenced by edge_badness(), inline_small_functions(), and want_inline_small_function_p().
◆ estimate_edge_size()
|
inline |
Return estimated size of the inline sequence of EDGE.
References do_estimate_edge_size(), edge_growth_cache, NULL, and edge_growth_cache_entry::size.
Referenced by estimate_edge_growth(), and inline_small_functions().
◆ estimate_edge_time()
|
inline |
Return estimated callee runtime increase after inlining EDGE.
References do_estimate_edge_time(), edge_growth_cache, NULL, and edge_growth_cache_entry::time.
Referenced by big_speedup_p(), can_inline_edge_by_limits_p(), edge_badness(), and inline_small_functions().
◆ estimate_growth()
| int estimate_growth | ( | struct cgraph_node * | node | ) |
Estimate the growth caused by inlining NODE into all callers.
References cgraph_node::call_for_symbol_and_aliases(), do_estimate_growth_1(), growth_data::growth, INT_MAX, ipa_size_summaries, growth_data::node, offline_size(), growth_data::self_recursive, ipa_size_summary::size, and growth_data::uninlinable.
Referenced by edge_badness(), and inline_small_functions().
◆ estimate_min_edge_growth()
|
inline |
Return lower bound on estimated callee growth after inlining EDGE.
References ipa_call_summary::call_stmt_size, ipa_call_summaries, ipa_fn_summaries, and cgraph_node::ultimate_alias_target().
Referenced by want_early_inline_function_p().
◆ estimate_size_after_inlining()
| int estimate_size_after_inlining | ( | struct cgraph_node * | node, |
| struct cgraph_edge * | edge ) |
In ipa-inline-analysis.cc
Estimate the size of NODE after inlining EDGE which should be an edge to either NODE or a call inlined into NODE.
References estimate_edge_growth(), gcc_assert, ggc_alloc(), ipa_call_summaries, ipa_size_summaries, and ipa_size_summary::size.
Referenced by caller_growth_limits(), check_callers(), and recursive_inlining().
◆ free_growth_caches()
Free growth caches.
References dump_file, edge_growth_cache, ggc_alloc(), node_context_cache, node_context_cache_clear, node_context_cache_hit, node_context_cache_miss, and NULL.
Referenced by inline_small_functions().
◆ growth_positive_p()
| bool growth_positive_p | ( | struct cgraph_node * | node, |
| struct cgraph_edge * | known_edge, | ||
| int | edge_growth ) |
Decide if growth of NODE is positive. This is cheaper than calculating actual growth. If edge growth of KNOWN_EDGE is known it is passed by EDGE_GROWTH.
References cgraph_node::call_for_symbol_and_aliases(), cgraph_node::callers, cgraph_inline_failed_type(), check_callers(), CIF_FINAL_ERROR, do_estimate_growth_1(), edge_growth_cache, FOR_EACH_ALIAS, ggc_alloc(), growth_data::growth, cgraph_edge::inline_failed, ipa_call_summaries, ipa_fn_summaries, ipa_size_summaries, cgraph_edge::next_caller, growth_data::node, NULL, offline_size(), ipa_ref::referring, growth_data::self_recursive, edge_growth_cache_entry::size, and growth_data::uninlinable.
Referenced by want_inline_function_to_all_callers_p(), and want_inline_small_function_p().
◆ initialize_growth_caches()
| void initialize_growth_caches | ( | ) |
Allocate edge growth caches.
References edge_growth_cache, ggc_alloc(), node_context_cache, and symtab.
Referenced by inline_small_functions().
◆ inline_account_function_p()
| bool inline_account_function_p | ( | struct cgraph_node * | node | ) |
Return true if NODE should be accounted for overall size estimate. Skip all nodes optimized for size so we can measure the growth of hot part of program no matter of the padding.
References symtab_node::decl, DECL_EXTERNAL, cgraph_node::frequency, ggc_alloc(), NODE_FREQUENCY_UNLIKELY_EXECUTED, and opt_for_fn.
Referenced by clone_inlined_nodes(), inline_call(), and inline_small_functions().
◆ inline_call()
| bool inline_call | ( | struct cgraph_edge * | e, |
| bool | update_original, | ||
| vec< cgraph_edge * > * | new_edges, | ||
| int * | overall_size, | ||
| bool | update_overall_summary, | ||
| bool * | callee_removed ) |
In ipa-inline-transform.cc
Mark edge E as inlined and update callgraph accordingly. UPDATE_ORIGINAL specify whether profile of original function should be updated. If any new indirect edges are discovered in the process, add them to NEW_EDGES, unless it is NULL. If UPDATE_OVERALL_SUMMARY is false, do not bother to recompute overall size of caller after inlining. Caller is required to eventually do it via ipa_update_overall_fn_summary. If callee_removed is non-NULL, set it to true if we removed callee node. Return true iff any new callgraph edges were discovered as a result of inlining.
References symtab_node::alias, build_optimization_node(), symbol_table::call_cgraph_insertion_hooks(), symbol_table::call_cgraph_removal_hooks(), cgraph_edge::callee, cgraph_node::callees, cgraph_edge::caller, cgraph_node::calls_comdat_local, can_remove_node_now_p(), cfun, cgraph_node::check_calls_comdat_local_p(), check_speculations(), clone_inlined_nodes(), symtab_node::comdat_local_p(), symtab_node::decl, DECL_FUNCTION_PERSONALITY, DECL_FUNCTION_SPECIFIC_OPTIMIZATION, DECL_POSSIBLY_INLINED, DECL_STRUCT_FUNCTION, dump_file, symtab_node::dump_name(), estimate_edge_growth(), expand_thunk(), gcc_assert, thunk_info::get(), thunk_info::get_create(), cgraph_node::get_untransformed_body(), ggc_alloc(), global_options, global_options_set, cgraph_edge::in_polymorphic_cdtor, inline_account_function_p(), cgraph_edge::inline_failed, cgraph_node::inlined_to, ipa_fn_summaries, ipa_merge_fn_summary_after_inlining(), ipa_merge_modref_summary_after_inlining(), ipa_propagate_indirect_call_infos(), ipa_size_summaries, ipa_update_overall_fn_summary(), mark_all_inlined_calls_cdtor(), ncalls_inlined, cgraph_edge::next_callee, cgraph_edge::next_caller, NULL, opt_for_fn, opts_for_fn(), overall_size, cgraph_edge::prev_caller, cgraph_edge::redirect_callee(), set_cfun(), symtab, cgraph_node::thunk, thunk_expansion, and cgraph_node::ultimate_alias_target().
Referenced by autofdo::afdo_indirect_call(), early_inline_small_functions(), flatten_function(), inline_always_inline_functions(), inline_small_functions(), inline_to_all_callers_1(), and recursive_inlining().
◆ inline_transform()
| unsigned int inline_transform | ( | struct cgraph_node * | node | ) |
Apply inline plan to function.
References profile_count::adjust_for_ipa_scaling(), profile_count::apply_scale(), cgraph_node::callees, cfun, symtab_node::clear_stmts_in_references(), cgraph_node::clones, basic_block_def::count, cgraph_node::count, current_function_decl, symtab_node::decl, profile_count::dump(), dump_file, ENTRY_BLOCK_PTR_FOR_FN, execute_fixup_cfg(), FOR_ALL_BB_FN, ggc_alloc(), profile_count::initialized_p(), cgraph_edge::inline_failed, cgraph_node::materialize_clone(), profile_count::max(), maybe_materialize_called_clones(), cgraph_edge::next_callee, cgraph_node::next_sibling_clone, opt_for_fn, optimize_inline_calls(), preserve_function_body_p(), cgraph_edge::redirect_call_stmt_to_callee(), symtab_node::remove_all_references(), save_inline_function_body(), timevar_pop(), timevar_push(), todo, TODO_update_ssa_any, TODO_update_ssa_only_virtuals, and profile_count::uninitialized().
◆ reset_node_cache()
| void reset_node_cache | ( | struct cgraph_node * | node | ) |
Reset cache for NODE. This must be done each time NODE body is modified.
References node_context_cache.
Referenced by inline_small_functions(), recursive_inlining(), and reset_edge_caches().
Variable Documentation
◆ edge_growth_cache
|
extern |
Analysis used by inlining decision heuristics. Copyright (C) 2003-2024 Free Software Foundation, Inc. Contributed by Jan Hubicka This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
Cached node/edge growths.
Referenced by check_callers(), do_estimate_edge_hints(), do_estimate_edge_size(), do_estimate_edge_time(), estimate_edge_hints(), estimate_edge_size(), estimate_edge_time(), free_growth_caches(), growth_positive_p(), initialize_growth_caches(), inline_small_functions(), ipa_remove_from_growth_caches(), recursive_inlining(), and reset_edge_caches().
◆ ipa_saved_clone_sources
|
extern |
For each node that was made the holder of function body by save_inline_function_body, this summary contains pointer to the previous holder of the body.
Referenced by expand_all_functions(), cgraph_edge::redirect_call_stmt_to_callee(), and save_inline_function_body().
◆ ncalls_inlined
|
extern |
Callgraph transformations to handle inlining Copyright (C) 2003-2024 Free Software Foundation, Inc. Contributed by Jan Hubicka This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
The inline decisions are stored in callgraph in "inline plan" and applied later. To mark given call inline, use inline_call function. The function marks the edge inlinable and, if necessary, produces virtual clone in the callgraph representing the new copy of callee's function body. The inline plan is applied on given function body by inline_transform.
Referenced by inline_call(), and ipa_inline().
◆ nfunctions_inlined
|
extern |
Referenced by clone_inlined_nodes(), and ipa_inline().
