auto-profile.cc File Reference
#include "config.h"#include "system.h"#include "coretypes.h"#include "backend.h"#include "tree.h"#include "gimple.h"#include "predict.h"#include "alloc-pool.h"#include "tree-pass.h"#include "ssa.h"#include "cgraph.h"#include "gcov-io.h"#include "diagnostic-core.h"#include "profile.h"#include "langhooks.h"#include "context.h"#include "pass_manager.h"#include "cfgloop.h"#include "tree-cfg.h"#include "tree-cfgcleanup.h"#include "tree-into-ssa.h"#include "gimple-iterator.h"#include "value-prof.h"#include "symbol-summary.h"#include "sreal.h"#include "ipa-cp.h"#include "ipa-prop.h"#include "ipa-fnsummary.h"#include "ipa-inline.h"#include "tree-inline.h"#include "auto-profile.h"#include "tree-pretty-print.h"#include "gimple-pretty-print.h"#include "output.h"
Include dependency graph for auto-profile.cc:
Data Structures | |
| class | autofdo::edge_info |
| struct | autofdo::decl_lineno |
| class | autofdo::count_info |
| struct | autofdo::string_compare |
| class | autofdo::string_table |
| class | autofdo::function_instance |
| class | autofdo::autofdo_source_profile |
| struct | autofdo::scale |
Namespaces | |
| namespace | autofdo |
Macros | |
| #define | INCLUDE_MAP |
| #define | INCLUDE_SET |
| #define | DEFAULT_AUTO_PROFILE_FILE "fbdata.afdo" |
| #define | AUTO_PROFILE_VERSION 2 |
| #define | AFDO_EINFO(e) |
Typedefs | |
| typedef auto_vec< decl_lineno, 20 > | autofdo::inline_stack |
| typedef auto_vec< char * > | autofdo::string_vector |
| typedef std::map< unsigned, gcov_type > | autofdo::icall_target_map |
| typedef std::set< gimple * > | autofdo::stmt_set |
| typedef hash_set< int_hash< int, -1, -2 > > | autofdo::name_index_set |
| typedef hash_map< int_hash< int, -1, -2 >, int > | autofdo::name_index_map |
| typedef std::set< basic_block > | autofdo::bb_set |
Variables | |
| gcov_type | afdo_hot_bb_threshod = -1 |
| static string_table * | autofdo::afdo_string_table |
| static autofdo_source_profile * | autofdo::afdo_source_profile |
| static gcov_summary * | autofdo::afdo_profile_info |
| static gcov_type | autofdo::afdo_count_scale = 1 |
Macro Definition Documentation
◆ AFDO_EINFO
| #define AFDO_EINFO | ( | e | ) |
Value:
Intermediate edge info used when propagating AutoFDO profile information. We can't edge->count() directly since it's computed from edge's probability while probability is yet not decided during propagation.
Referenced by autofdo::afdo_adjust_guessed_profile(), autofdo::afdo_calculate_branch_prob(), autofdo::afdo_propagate_circuit(), and autofdo::afdo_propagate_edge().
◆ AUTO_PROFILE_VERSION
| #define AUTO_PROFILE_VERSION 2 |
Referenced by autofdo::read_profile().
◆ DEFAULT_AUTO_PROFILE_FILE
| #define DEFAULT_AUTO_PROFILE_FILE "fbdata.afdo" |
The following routines implements AutoFDO optimization.
This optimization uses sampling profiles to annotate basic block counts
and uses heuristics to estimate branch probabilities.
There are three phases in AutoFDO:
Phase 1: At startup.
Read profile from the profile data file.
The following info is read from the profile datafile:
* string_table: a map between function name and its index.
* autofdo_source_profile: a map from function_instance name to
function_instance. This is represented as a forest of
function_instances.
* WorkingSet: a histogram of how many instructions are covered for a
given percentage of total cycles. This is describing the binary
level information (not source level). This info is used to help
decide if we want aggressive optimizations that could increase
code footprint (e.g. loop unroll etc.)
A function instance is an instance of function that could either be a
standalone symbol, or a clone of a function that is inlined into another
function.
Phase 2: In afdo_offline pass.
Remove function instances from other translation units
and offline all cross-translation unit inlining done during train
run compilation. This is necessary to not lose profiles with
LTO train run.
Phase 3: During early optimization.
AFDO inline + value profile transformation.
This happens during early optimization.
During early inlning AFDO inliner is executed which
uses autofdo_source_profile to find if a callsite is:
* inlined in the profiled binary.
* callee body is hot in the profiling run.
If both condition satisfies, early inline will inline the callsite
regardless of the code growth.
Performing this early has benefit of doing early optimizations
before read IPA passe and getting more "context sensitivity" of
the profile read. Profile of inlined functions may differ
significantly form one inline instance to another and from the
offline version.
This is controlled by -fauto-profile-inlining and is independent
of -fearly-inlining.
Phase 4: In AFDO pass.
Offline all functions that has been inlined in the
train run but were not inlined in early inlining nor AFDO
inline.
Phase 5: In AFDO pass.
Annotate control flow graph.
* Annotate basic block count
* Estimate branch probability
* Use earlier static profile to fill in the gaps
if AFDO profile is ambigous
After the above 5 phases, all profile is readily annotated on the GCC IR.
AutoFDO tries to reuse all FDO infrastructure as much as possible to make
use of the profile. E.g. it uses existing mechanism to calculate the basic
block/edge frequency, as well as the cgraph node/edge count.
Referenced by read_autofdo_file().
◆ INCLUDE_MAP
| #define INCLUDE_MAP |
Read and annotate call graph profile from the auto profile data file. Copyright (C) 2014-2025 Free Software Foundation, Inc. Contributed by Dehao Chen (dehao@google.com) This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
◆ INCLUDE_SET
| #define INCLUDE_SET |
Function Documentation
◆ afdo_callsite_hot_enough_for_early_inline()
| bool afdo_callsite_hot_enough_for_early_inline | ( | struct cgraph_edge * | edge | ) |
Returns TRUE if EDGE is hot enough to be inlined early.
References profile_count::afdo(), autofdo::afdo_source_profile, count, profile_count::dump(), dump_file, profile_count::from_gcov_type(), and maybe_hot_afdo_count_p().
Referenced by inline_functions_by_afdo().
◆ afdo_vpt_for_early_inline()
| bool afdo_vpt_for_early_inline | ( | cgraph_node * | node | ) |
Do indirect call promotion during early inlining to make the
IR match the profiled binary before actual annotation.
This is needed because an indirect call might have been promoted
and inlined in the profiled binary. If we do not promote and
inline these indirect calls before annotation, the profile for
these promoted functions will be lost.
e.g. foo() --indirect_call--> bar()
In profiled binary, the callsite is promoted and inlined, making
the profile look like:
foo: {
loc_foo_1: count_1
bar@loc_foo_2: {
loc_bar_1: count_2
loc_bar_2: count_3
}
}
Before AutoFDO pass, loc_foo_2 is not promoted thus not inlined.
If we perform annotation on it, the profile inside bar@loc_foo2
will be wasted.
To avoid this, we promote loc_foo_2 and inline the promoted bar
function before annotation, so the profile inside bar@loc_foo2
will be useful.
References autofdo::afdo_source_profile, autofdo::afdo_vpt(), changed, autofdo::count_info::count, symtab_node::decl, gimple_bb(), gsi_end_p(), gsi_next(), gsi_start_bb(), gsi_stmt(), MAX, NULL, stmt_loc_used_by_debug_info(), and autofdo::count_info::targets.
Referenced by inline_functions_by_afdo().
◆ end_auto_profile()
| void end_auto_profile | ( | void | ) |
Free the resources.
References autofdo::afdo_source_profile, autofdo::afdo_string_table, NULL, and profile_info.
Referenced by compile_file().
◆ make_pass_ipa_auto_profile()
| simple_ipa_opt_pass * make_pass_ipa_auto_profile | ( | gcc::context * | ctxt | ) |
◆ make_pass_ipa_auto_profile_offline()
| simple_ipa_opt_pass * make_pass_ipa_auto_profile_offline | ( | gcc::context * | ctxt | ) |
◆ maybe_hot_afdo_count_p()
| bool maybe_hot_afdo_count_p | ( | profile_count | count | ) |
Return true if COUNT is possibly hot.
References afdo_hot_bb_threshod, count, and gcc_checking_assert.
Referenced by afdo_callsite_hot_enough_for_early_inline(), and end_branch_prob().
◆ read_autofdo_file()
| void read_autofdo_file | ( | void | ) |
Read the profile from the profile data file.
References autofdo::afdo_profile_info, DEFAULT_AUTO_PROFILE_FILE, NULL, and autofdo::read_profile().
◆ remove_afdo_speculative_target()
| void remove_afdo_speculative_target | ( | cgraph_edge * | e | ) |
If speculation used during early inline, remove the target so we do not speculate the indirect edge again during afdo pass.
References autofdo::afdo_source_profile.
Referenced by inline_functions_by_afdo().
◆ stmt_loc_used_by_debug_info()
Return true if location of STMT may be expressed by debug info.
References gimple_clobber_p(), gimple_debug_begin_stmt_p(), and is_gimple_debug().
Referenced by autofdo::afdo_set_bb_count(), afdo_vpt_for_early_inline(), and autofdo::autofdo_source_profile::get_count_info().
Variable Documentation
◆ afdo_hot_bb_threshod
| gcov_type afdo_hot_bb_threshod = -1 |
profile counts determined by AFDO smaller than afdo_hot_bb_threshold are considered cols.
Referenced by maybe_hot_afdo_count_p(), and autofdo::autofdo_source_profile::read().
