LCOV - code coverage report
Current view: top level - gcc - cgraphclones.cc (source / functions) Coverage Total Hit
Test: gcc.info Lines: 83.0 % 535 444
Test Date: 2024-03-23 14:05:01 Functions: 91.3 % 23 21
Legend: Lines: hit not hit | Branches: + taken - not taken # not executed Branches: - 0 0

             Branch data     Line data    Source code
       1                 :             : /* Callgraph clones
       2                 :             :    Copyright (C) 2003-2024 Free Software Foundation, Inc.
       3                 :             :    Contributed by Jan Hubicka
       4                 :             : 
       5                 :             : This file is part of GCC.
       6                 :             : 
       7                 :             : GCC is free software; you can redistribute it and/or modify it under
       8                 :             : the terms of the GNU General Public License as published by the Free
       9                 :             : Software Foundation; either version 3, or (at your option) any later
      10                 :             : version.
      11                 :             : 
      12                 :             : GCC is distributed in the hope that it will be useful, but WITHOUT ANY
      13                 :             : WARRANTY; without even the implied warranty of MERCHANTABILITY or
      14                 :             : FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
      15                 :             : for more details.
      16                 :             : 
      17                 :             : You should have received a copy of the GNU General Public License
      18                 :             : along with GCC; see the file COPYING3.  If not see
      19                 :             : <http://www.gnu.org/licenses/>.  */
      20                 :             : 
      21                 :             : /* This module provide facilities for cloning functions.  I.e. creating
      22                 :             :    new functions based on existing functions with simple modifications,
      23                 :             :    such as replacement of parameters.
      24                 :             : 
      25                 :             :    To allow whole program optimization without actual presence of function
      26                 :             :    bodies, an additional infrastructure is provided for so-called virtual
      27                 :             :    clones
      28                 :             : 
      29                 :             :    A virtual clone in the callgraph is a function that has no
      30                 :             :    associated body, just a description of how to create its body based
      31                 :             :    on a different function (which itself may be a virtual clone).
      32                 :             : 
      33                 :             :    The description of function modifications includes adjustments to
      34                 :             :    the function's signature (which allows, for example, removing or
      35                 :             :    adding function arguments), substitutions to perform on the
      36                 :             :    function body, and, for inlined functions, a pointer to the
      37                 :             :    function that it will be inlined into.
      38                 :             : 
      39                 :             :    It is also possible to redirect any edge of the callgraph from a
      40                 :             :    function to its virtual clone.  This implies updating of the call
      41                 :             :    site to adjust for the new function signature.
      42                 :             : 
      43                 :             :    Most of the transformations performed by inter-procedural
      44                 :             :    optimizations can be represented via virtual clones.  For
      45                 :             :    instance, a constant propagation pass can produce a virtual clone
      46                 :             :    of the function which replaces one of its arguments by a
      47                 :             :    constant.  The inliner can represent its decisions by producing a
      48                 :             :    clone of a function whose body will be later integrated into
      49                 :             :    a given function.
      50                 :             : 
      51                 :             :    Using virtual clones, the program can be easily updated
      52                 :             :    during the Execute stage, solving most of pass interactions
      53                 :             :    problems that would otherwise occur during Transform.
      54                 :             : 
      55                 :             :    Virtual clones are later materialized in the LTRANS stage and
      56                 :             :    turned into real functions.  Passes executed after the virtual
      57                 :             :    clone were introduced also perform their Transform stage
      58                 :             :    on new functions, so for a pass there is no significant
      59                 :             :    difference between operating on a real function or a virtual
      60                 :             :    clone introduced before its Execute stage.
      61                 :             : 
      62                 :             :    Optimization passes then work on virtual clones introduced before
      63                 :             :    their Execute stage as if they were real functions.  The
      64                 :             :    only difference is that clones are not visible during the
      65                 :             :    Generate Summary stage.  */
      66                 :             : 
      67                 :             : #include "config.h"
      68                 :             : #include "system.h"
      69                 :             : #include "coretypes.h"
      70                 :             : #include "backend.h"
      71                 :             : #include "target.h"
      72                 :             : #include "rtl.h"
      73                 :             : #include "tree.h"
      74                 :             : #include "gimple.h"
      75                 :             : #include "stringpool.h"
      76                 :             : #include "cgraph.h"
      77                 :             : #include "lto-streamer.h"
      78                 :             : #include "tree-eh.h"
      79                 :             : #include "tree-cfg.h"
      80                 :             : #include "tree-inline.h"
      81                 :             : #include "attribs.h"
      82                 :             : #include "dumpfile.h"
      83                 :             : #include "gimple-pretty-print.h"
      84                 :             : #include "alloc-pool.h"
      85                 :             : #include "symbol-summary.h"
      86                 :             : #include "tree-vrp.h"
      87                 :             : #include "sreal.h"
      88                 :             : #include "ipa-cp.h"
      89                 :             : #include "ipa-prop.h"
      90                 :             : #include "ipa-fnsummary.h"
      91                 :             : #include "symtab-thunks.h"
      92                 :             : #include "symtab-clones.h"
      93                 :             : 
      94                 :             : /* Create clone of edge in the node N represented by CALL_EXPR
      95                 :             :    the callgraph.  */
      96                 :             : 
      97                 :             : cgraph_edge *
      98                 :     5305653 : cgraph_edge::clone (cgraph_node *n, gcall *call_stmt, unsigned stmt_uid,
      99                 :             :                     profile_count num, profile_count den,
     100                 :             :                     bool update_original)
     101                 :             : {
     102                 :     5305653 :   cgraph_edge *new_edge;
     103                 :     5305653 :   profile_count::adjust_for_ipa_scaling (&num, &den);
     104                 :     5305653 :   profile_count prof_count = count.apply_scale (num, den);
     105                 :             : 
     106                 :     5305653 :   if (indirect_unknown_callee)
     107                 :             :     {
     108                 :      160099 :       tree decl;
     109                 :             : 
     110                 :      159864 :       if (call_stmt && (decl = gimple_call_fndecl (call_stmt))
     111                 :             :           /* When the call is speculative, we need to resolve it 
     112                 :             :              via cgraph_resolve_speculation and not here.  */
     113                 :      165828 :           && !speculative)
     114                 :             :         {
     115                 :        5729 :           cgraph_node *callee = cgraph_node::get (decl);
     116                 :        5729 :           gcc_checking_assert (callee);
     117                 :        5729 :           new_edge = n->create_edge (callee, call_stmt, prof_count, true);
     118                 :             :         }
     119                 :             :       else
     120                 :             :         {
     121                 :      308740 :           new_edge = n->create_indirect_edge (call_stmt,
     122                 :      154370 :                                               indirect_info->ecf_flags,
     123                 :             :                                               prof_count, true);
     124                 :      154370 :           *new_edge->indirect_info = *indirect_info;
     125                 :             :         }
     126                 :             :     }
     127                 :             :   else
     128                 :             :     {
     129                 :     5145554 :       new_edge = n->create_edge (callee, call_stmt, prof_count, true);
     130                 :     5145554 :       if (indirect_info)
     131                 :             :         {
     132                 :           0 :           new_edge->indirect_info
     133                 :           0 :             = ggc_cleared_alloc<cgraph_indirect_call_info> ();
     134                 :           0 :           *new_edge->indirect_info = *indirect_info;
     135                 :             :         }
     136                 :             :     }
     137                 :             : 
     138                 :     5305653 :   new_edge->inline_failed = inline_failed;
     139                 :     5305653 :   new_edge->indirect_inlining_edge = indirect_inlining_edge;
     140                 :     5305653 :   if (!call_stmt)
     141                 :      229433 :     new_edge->lto_stmt_uid = stmt_uid;
     142                 :     5305653 :   new_edge->speculative_id = speculative_id;
     143                 :             :   /* Clone flags that depend on call_stmt availability manually.  */
     144                 :     5305653 :   new_edge->can_throw_external = can_throw_external;
     145                 :     5305653 :   new_edge->call_stmt_cannot_inline_p = call_stmt_cannot_inline_p;
     146                 :     5305653 :   new_edge->speculative = speculative;
     147                 :     5305653 :   new_edge->in_polymorphic_cdtor = in_polymorphic_cdtor;
     148                 :             : 
     149                 :             :   /* Update IPA profile.  Local profiles need no updating in original.  */
     150                 :     5305653 :   if (update_original)
     151                 :     4746981 :     count = count.combine_with_ipa_count_within (count.ipa () 
     152                 :     4746981 :                                                  - new_edge->count.ipa (),
     153                 :     4746981 :                                                  caller->count);
     154                 :     5305653 :   symtab->call_edge_duplication_hooks (this, new_edge);
     155                 :     5305653 :   return new_edge;
     156                 :             : }
     157                 :             : 
     158                 :             : /* Set flags of NEW_NODE and its decl.  NEW_NODE is a newly created private
     159                 :             :    clone or its thunk.  */
     160                 :             : 
     161                 :             : static void
     162                 :      107915 : set_new_clone_decl_and_node_flags (cgraph_node *new_node)
     163                 :             : {
     164                 :      107915 :   DECL_EXTERNAL (new_node->decl) = 0;
     165                 :      107915 :   TREE_PUBLIC (new_node->decl) = 0;
     166                 :      107915 :   DECL_COMDAT (new_node->decl) = 0;
     167                 :      107915 :   DECL_WEAK (new_node->decl) = 0;
     168                 :      107915 :   DECL_VIRTUAL_P (new_node->decl) = 0;
     169                 :      107915 :   DECL_STATIC_CONSTRUCTOR (new_node->decl) = 0;
     170                 :      107915 :   DECL_STATIC_DESTRUCTOR (new_node->decl) = 0;
     171                 :      107915 :   DECL_SET_IS_OPERATOR_NEW (new_node->decl, 0);
     172                 :      107915 :   DECL_SET_IS_OPERATOR_DELETE (new_node->decl, 0);
     173                 :      107915 :   DECL_IS_REPLACEABLE_OPERATOR (new_node->decl) = 0;
     174                 :             : 
     175                 :      107915 :   new_node->externally_visible = 0;
     176                 :      107915 :   new_node->local = 1;
     177                 :      107915 :   new_node->lowered = true;
     178                 :      107915 :   new_node->semantic_interposition = 0;
     179                 :      107915 : }
     180                 :             : 
     181                 :             : /* Duplicate thunk THUNK if necessary but make it to refer to NODE.
     182                 :             :    ARGS_TO_SKIP, if non-NULL, determines which parameters should be omitted.
     183                 :             :    Function can return NODE if no thunk is necessary, which can happen when
     184                 :             :    thunk is this_adjusting but we are removing this parameter.  */
     185                 :             : 
     186                 :             : static cgraph_node *
     187                 :          99 : duplicate_thunk_for_node (cgraph_node *thunk, cgraph_node *node)
     188                 :             : {
     189                 :          99 :   cgraph_node *new_thunk, *thunk_of;
     190                 :          99 :   thunk_of = thunk->callees->callee->ultimate_alias_target ();
     191                 :             : 
     192                 :          99 :   if (thunk_of->thunk)
     193                 :           0 :     node = duplicate_thunk_for_node (thunk_of, node);
     194                 :             : 
     195                 :          99 :   if (!DECL_ARGUMENTS (thunk->decl))
     196                 :           1 :     thunk->get_untransformed_body ();
     197                 :             : 
     198                 :          99 :   thunk_info *i = thunk_info::get (thunk);
     199                 :          99 :   cgraph_edge *cs;
     200                 :         123 :   for (cs = node->callers; cs; cs = cs->next_caller)
     201                 :          47 :     if (cs->caller->thunk)
     202                 :             :       {
     203                 :          23 :         thunk_info *i2 = thunk_info::get (cs->caller);
     204                 :          23 :         if (*i2 == *i)
     205                 :          23 :           return cs->caller;
     206                 :             :       }
     207                 :             : 
     208                 :          76 :   tree new_decl;
     209                 :          76 :   clone_info *info = clone_info::get (node);
     210                 :          76 :   if (info && info->param_adjustments)
     211                 :             :     {
     212                 :             :       /* We do not need to duplicate this_adjusting thunks if we have removed
     213                 :             :          this.  */
     214                 :          73 :       if (i->this_adjusting
     215                 :          73 :           && !info->param_adjustments->first_param_intact_p ())
     216                 :          36 :         return node;
     217                 :             : 
     218                 :          37 :       new_decl = copy_node (thunk->decl);
     219                 :          37 :       ipa_param_body_adjustments body_adj (info->param_adjustments,
     220                 :          37 :                                            new_decl);
     221                 :          37 :       body_adj.modify_formal_parameters ();
     222                 :          37 :     }
     223                 :             :   else
     224                 :             :     {
     225                 :           3 :       new_decl = copy_node (thunk->decl);
     226                 :           3 :       for (tree *arg = &DECL_ARGUMENTS (new_decl);
     227                 :           9 :            *arg; arg = &DECL_CHAIN (*arg))
     228                 :             :         {
     229                 :           6 :           tree next = DECL_CHAIN (*arg);
     230                 :           6 :           *arg = copy_node (*arg);
     231                 :           6 :           DECL_CONTEXT (*arg) = new_decl;
     232                 :           6 :           DECL_CHAIN (*arg) = next;
     233                 :             :         }
     234                 :             :     }
     235                 :             : 
     236                 :          40 :   gcc_checking_assert (!DECL_STRUCT_FUNCTION (new_decl));
     237                 :          40 :   gcc_checking_assert (!DECL_INITIAL (new_decl));
     238                 :          40 :   gcc_checking_assert (!DECL_RESULT (new_decl));
     239                 :          40 :   gcc_checking_assert (!DECL_RTL_SET_P (new_decl));
     240                 :             : 
     241                 :          40 :   DECL_NAME (new_decl) = clone_function_name_numbered (thunk->decl,
     242                 :             :                                                        "artificial_thunk");
     243                 :          40 :   SET_DECL_ASSEMBLER_NAME (new_decl, DECL_NAME (new_decl));
     244                 :             : 
     245                 :             :   /* We need to force DECL_IGNORED_P because the new thunk is created after
     246                 :             :      early debug was run.  */
     247                 :          40 :   DECL_IGNORED_P (new_decl) = 1;
     248                 :             : 
     249                 :          40 :   new_thunk = cgraph_node::create (new_decl);
     250                 :          40 :   set_new_clone_decl_and_node_flags (new_thunk);
     251                 :          40 :   new_thunk->definition = true;
     252                 :          40 :   new_thunk->can_change_signature = node->can_change_signature;
     253                 :          40 :   new_thunk->thunk = thunk->thunk;
     254                 :          40 :   new_thunk->unique_name = in_lto_p;
     255                 :          40 :   new_thunk->former_clone_of = thunk->decl;
     256                 :          40 :   if (info && info->param_adjustments)
     257                 :          37 :     clone_info::get_create (new_thunk)->param_adjustments
     258                 :          37 :            = info->param_adjustments;
     259                 :          40 :   new_thunk->unit_id = thunk->unit_id;
     260                 :          40 :   new_thunk->merged_comdat = thunk->merged_comdat;
     261                 :          40 :   new_thunk->merged_extern_inline = thunk->merged_extern_inline;
     262                 :             : 
     263                 :          40 :   cgraph_edge *e = new_thunk->create_edge (node, NULL, new_thunk->count);
     264                 :          40 :   symtab->call_edge_duplication_hooks (thunk->callees, e);
     265                 :          40 :   symtab->call_cgraph_duplication_hooks (thunk, new_thunk);
     266                 :          40 :   return new_thunk;
     267                 :             : }
     268                 :             : 
     269                 :             : /* If E does not lead to a thunk, simply redirect it to N.  Otherwise create
     270                 :             :    one or more equivalent thunks for N and redirect E to the first in the
     271                 :             :    chain.  Note that it is then necessary to call
     272                 :             :    n->expand_all_artificial_thunks once all callers are redirected.  */
     273                 :             : 
     274                 :             : void
     275                 :      256886 : cgraph_edge::redirect_callee_duplicating_thunks (cgraph_node *n)
     276                 :             : {
     277                 :      256886 :   cgraph_node *orig_to = callee->ultimate_alias_target ();
     278                 :      256886 :   if (orig_to->thunk)
     279                 :          99 :     n = duplicate_thunk_for_node (orig_to, n);
     280                 :             : 
     281                 :      256886 :   redirect_callee (n);
     282                 :      256886 : }
     283                 :             : 
     284                 :             : /* Call expand_thunk on all callers that are thunks and if analyze those nodes
     285                 :             :    that were expanded.  */
     286                 :             : 
     287                 :             : void
     288                 :     2452836 : cgraph_node::expand_all_artificial_thunks ()
     289                 :             : {
     290                 :     2452836 :   cgraph_edge *e;
     291                 :     2709967 :   for (e = callers; e;)
     292                 :      257131 :     if (e->caller->thunk)
     293                 :             :       {
     294                 :          44 :         cgraph_node *thunk = e->caller;
     295                 :             : 
     296                 :          44 :         e = e->next_caller;
     297                 :          44 :         if (expand_thunk (thunk, false, false))
     298                 :             :           {
     299                 :           0 :             thunk->thunk = false;
     300                 :           0 :             thunk->analyze ();
     301                 :           0 :             ipa_analyze_node (thunk);
     302                 :           0 :             inline_analyze_function (thunk);
     303                 :             :           }
     304                 :          44 :         thunk->expand_all_artificial_thunks ();
     305                 :             :       }
     306                 :             :     else
     307                 :      257087 :       e = e->next_caller;
     308                 :     2452836 : }
     309                 :             : 
     310                 :             : void
     311                 :     3724744 : dump_callgraph_transformation (const cgraph_node *original,
     312                 :             :                                const cgraph_node *clone,
     313                 :             :                                const char *suffix)
     314                 :             : {
     315                 :     3724744 :   if (symtab->ipa_clones_dump_file)
     316                 :             :     {
     317                 :           0 :       fprintf (symtab->ipa_clones_dump_file,
     318                 :             :                "Callgraph clone;%s;%d;%s;%d;%d;%s;%d;%s;%d;%d;%s\n",
     319                 :           0 :                original->asm_name (), original->order,
     320                 :           0 :                DECL_SOURCE_FILE (original->decl),
     321                 :           0 :                DECL_SOURCE_LINE (original->decl),
     322                 :           0 :                DECL_SOURCE_COLUMN (original->decl), clone->asm_name (),
     323                 :           0 :                clone->order, DECL_SOURCE_FILE (clone->decl),
     324                 :           0 :                DECL_SOURCE_LINE (clone->decl), DECL_SOURCE_COLUMN (clone->decl),
     325                 :             :                suffix);
     326                 :             : 
     327                 :           0 :       symtab->cloned_nodes.add (original);
     328                 :           0 :       symtab->cloned_nodes.add (clone);
     329                 :             :     }
     330                 :     3724744 : }
     331                 :             : 
     332                 :             : /* Turn profile of N to local profile.   */
     333                 :             : 
     334                 :             : static void
     335                 :           0 : localize_profile (cgraph_node *n)
     336                 :             : {
     337                 :           0 :   n->count = n->count.guessed_local ();
     338                 :           0 :   for (cgraph_edge *e = n->callees; e; e=e->next_callee)
     339                 :             :     {
     340                 :           0 :       e->count = e->count.guessed_local ();
     341                 :           0 :       if (!e->inline_failed)
     342                 :           0 :         localize_profile (e->callee);
     343                 :             :     }
     344                 :           0 :   for (cgraph_edge *e = n->indirect_calls; e; e=e->next_callee)
     345                 :           0 :     e->count = e->count.guessed_local ();
     346                 :           0 : }
     347                 :             : 
     348                 :             : /* Create node representing clone of N executed COUNT times.  Decrease
     349                 :             :    the execution counts from original node too.
     350                 :             :    The new clone will have decl set to DECL that may or may not be the same
     351                 :             :    as decl of N.
     352                 :             : 
     353                 :             :    When UPDATE_ORIGINAL is true, the counts are subtracted from the original
     354                 :             :    function's profile to reflect the fact that part of execution is handled
     355                 :             :    by node.  
     356                 :             :    When CALL_DUPLICATION_HOOK is true, the ipa passes are acknowledged about
     357                 :             :    the new clone. Otherwise the caller is responsible for doing so later.
     358                 :             : 
     359                 :             :    If the new node is being inlined into another one, NEW_INLINED_TO should be
     360                 :             :    the outline function the new one is (even indirectly) inlined to.  All hooks
     361                 :             :    will see this in node's inlined_to, when invoked.  Can be NULL if the
     362                 :             :    node is not inlined.
     363                 :             : 
     364                 :             :    If PARAM_ADJUSTMENTS is non-NULL, the parameter manipulation information
     365                 :             :    will be overwritten by the new structure.  Otherwise the new node will
     366                 :             :    share parameter manipulation information with the original node.  */
     367                 :             : 
     368                 :             : cgraph_node *
     369                 :     2452660 : cgraph_node::create_clone (tree new_decl, profile_count prof_count,
     370                 :             :                            bool update_original,
     371                 :             :                            vec<cgraph_edge *> redirect_callers,
     372                 :             :                            bool call_duplication_hook,
     373                 :             :                            cgraph_node *new_inlined_to,
     374                 :             :                            ipa_param_adjustments *param_adjustments,
     375                 :             :                            const char *suffix)
     376                 :             : {
     377                 :     2452660 :   cgraph_node *new_node = symtab->create_empty ();
     378                 :     2452660 :   cgraph_edge *e;
     379                 :     2452660 :   unsigned i;
     380                 :     2452660 :   profile_count old_count = count;
     381                 :     2452660 :   bool nonzero = count.ipa ().nonzero_p ();
     382                 :             : 
     383                 :     2452660 :   if (new_inlined_to)
     384                 :     2320973 :     dump_callgraph_transformation (this, new_inlined_to, "inlining to");
     385                 :             : 
     386                 :             :   /* When inlining we scale precisely to prof_count, when cloning we can
     387                 :             :      preserve local profile.  */
     388                 :     2320973 :   if (!new_inlined_to)
     389                 :      131687 :     prof_count = count.combine_with_ipa_count (prof_count);
     390                 :     2452660 :   new_node->count = prof_count;
     391                 :     2452660 :   new_node->calls_declare_variant_alt = this->calls_declare_variant_alt;
     392                 :             : 
     393                 :             :   /* Update IPA profile.  Local profiles need no updating in original.  */
     394                 :     2452660 :   if (update_original)
     395                 :             :     {
     396                 :     2304438 :       if (inlined_to)
     397                 :      289631 :         count = count.combine_with_ipa_count_within (count.ipa ()
     398                 :      579262 :                                                      - prof_count.ipa (),
     399                 :             :                                                      inlined_to->count);
     400                 :             :       else
     401                 :     2014807 :         count = count.combine_with_ipa_count (count.ipa () - prof_count.ipa ());
     402                 :             :     }
     403                 :     2452660 :   new_node->decl = new_decl;
     404                 :     2452660 :   new_node->register_symbol ();
     405                 :     2452660 :   new_node->lto_file_data = lto_file_data;
     406                 :     2452660 :   new_node->analyzed = analyzed;
     407                 :     2452660 :   new_node->definition = definition;
     408                 :     2452660 :   new_node->versionable = versionable;
     409                 :     2452660 :   new_node->can_change_signature = can_change_signature;
     410                 :     2452660 :   new_node->redefined_extern_inline = redefined_extern_inline;
     411                 :     2452660 :   new_node->semantic_interposition = semantic_interposition;
     412                 :     2452660 :   new_node->tm_may_enter_irr = tm_may_enter_irr;
     413                 :     2452660 :   new_node->externally_visible = false;
     414                 :     2452660 :   new_node->no_reorder = no_reorder;
     415                 :     2452660 :   new_node->local = true;
     416                 :     2452660 :   new_node->inlined_to = new_inlined_to;
     417                 :     2452660 :   new_node->rtl = rtl;
     418                 :     2452660 :   new_node->frequency = frequency;
     419                 :     2452660 :   new_node->tp_first_run = tp_first_run;
     420                 :     2452660 :   new_node->tm_clone = tm_clone;
     421                 :     2452660 :   new_node->icf_merged = icf_merged;
     422                 :     2452660 :   new_node->thunk = thunk;
     423                 :     2452660 :   new_node->unit_id = unit_id;
     424                 :     2452660 :   new_node->merged_comdat = merged_comdat;
     425                 :     2452660 :   new_node->merged_extern_inline = merged_extern_inline;
     426                 :     2452660 :   clone_info *info = clone_info::get (this);
     427                 :             : 
     428                 :     2452660 :   if (param_adjustments)
     429                 :      102798 :     clone_info::get_create (new_node)->param_adjustments = param_adjustments;
     430                 :     2349862 :   else if (info && info->param_adjustments)
     431                 :      188318 :     clone_info::get_create (new_node)->param_adjustments
     432                 :      188318 :          = info->param_adjustments;
     433                 :     2452660 :   new_node->split_part = split_part;
     434                 :             : 
     435                 :     2709327 :   FOR_EACH_VEC_ELT (redirect_callers, i, e)
     436                 :             :     {
     437                 :             :       /* Redirect calls to the old version node to point to its new
     438                 :             :          version.  The only exception is when the edge was proved to
     439                 :             :          be unreachable during the cloning procedure.  */
     440                 :      256667 :       if (!e->callee
     441                 :      256667 :           || !fndecl_built_in_p (e->callee->decl, BUILT_IN_UNREACHABLE,
     442                 :             :                                                   BUILT_IN_UNREACHABLE_TRAP))
     443                 :      256667 :         e->redirect_callee_duplicating_thunks (new_node);
     444                 :             :     }
     445                 :     2452660 :   new_node->expand_all_artificial_thunks ();
     446                 :             : 
     447                 :     4651588 :   for (e = callees;e; e=e->next_callee)
     448                 :     2198928 :     e->clone (new_node, e->call_stmt, e->lto_stmt_uid, new_node->count, old_count,
     449                 :             :               update_original);
     450                 :             : 
     451                 :     2523811 :   for (e = indirect_calls; e; e = e->next_callee)
     452                 :       71151 :     e->clone (new_node, e->call_stmt, e->lto_stmt_uid,
     453                 :             :               new_node->count, old_count, update_original);
     454                 :     2452660 :   new_node->clone_references (this);
     455                 :             : 
     456                 :     2452660 :   new_node->next_sibling_clone = clones;
     457                 :     2452660 :   if (clones)
     458                 :     1180624 :     clones->prev_sibling_clone = new_node;
     459                 :     2452660 :   clones = new_node;
     460                 :     2452660 :   new_node->clone_of = this;
     461                 :             : 
     462                 :     2452660 :   if (call_duplication_hook)
     463                 :     2322418 :     symtab->call_cgraph_duplication_hooks (this, new_node);
     464                 :             :   /* With partial train run we do not want to assume that original's
     465                 :             :      count is zero whenever we redurect all executed edges to clone.
     466                 :             :      Simply drop profile to local one in this case.  */
     467                 :     2452660 :   if (update_original
     468                 :     2304438 :       && opt_for_fn (decl, flag_profile_partial_training)
     469                 :           0 :       && nonzero
     470                 :           0 :       && count.ipa_p ()
     471                 :     2452660 :       && !count.ipa ().nonzero_p ()
     472                 :     2452660 :       && !inlined_to)
     473                 :           0 :     localize_profile (this);
     474                 :             : 
     475                 :     2452660 :   if (!new_inlined_to)
     476                 :      131687 :     dump_callgraph_transformation (this, new_node, suffix);
     477                 :             : 
     478                 :     2452660 :   return new_node;
     479                 :             : }
     480                 :             : 
     481                 :             : static GTY(()) hash_map<const char *, unsigned> *clone_fn_ids;
     482                 :             : 
     483                 :             : /* Return a new assembler name for a clone of decl named NAME.  Apart
     484                 :             :    from the string SUFFIX, the new name will end with a unique (for
     485                 :             :    each NAME) unspecified number.  If clone numbering is not needed
     486                 :             :    then the two argument clone_function_name should be used instead.
     487                 :             :    Should not be called directly except for by
     488                 :             :    lto-partition.cc:privatize_symbol_name_1.  */
     489                 :             : 
     490                 :             : tree
     491                 :      101701 : clone_function_name_numbered (const char *name, const char *suffix)
     492                 :             : {
     493                 :             :   /* Initialize the function->counter mapping the first time it's
     494                 :             :      needed.  */
     495                 :      101701 :   if (!clone_fn_ids)
     496                 :       19516 :     clone_fn_ids = hash_map<const char *, unsigned int>::create_ggc (64);
     497                 :      305103 :   unsigned int &suffix_counter = clone_fn_ids->get_or_insert (
     498                 :      101701 :                                    IDENTIFIER_POINTER (get_identifier (name)));
     499                 :      101701 :   return clone_function_name (name, suffix, suffix_counter++);
     500                 :             : }
     501                 :             : 
     502                 :             : /* Return a new assembler name for a clone of DECL.  Apart from string
     503                 :             :    SUFFIX, the new name will end with a unique (for each DECL
     504                 :             :    assembler name) unspecified number.  If clone numbering is not
     505                 :             :    needed then the two argument clone_function_name should be used
     506                 :             :    instead.  */
     507                 :             : 
     508                 :             : tree
     509                 :      101701 : clone_function_name_numbered (tree decl, const char *suffix)
     510                 :             : {
     511                 :      101701 :   tree name = DECL_ASSEMBLER_NAME (decl);
     512                 :      101701 :   return clone_function_name_numbered (IDENTIFIER_POINTER (name),
     513                 :      101701 :                                        suffix);
     514                 :             : }
     515                 :             : 
     516                 :             : /* Return a new assembler name for a clone of decl named NAME.  Apart
     517                 :             :    from the string SUFFIX, the new name will end with the specified
     518                 :             :    NUMBER.  If clone numbering is not needed then the two argument
     519                 :             :    clone_function_name should be used instead.  */
     520                 :             : 
     521                 :             : tree
     522                 :      209910 : clone_function_name (const char *name, const char *suffix,
     523                 :             :                      unsigned long number)
     524                 :             : {
     525                 :      209910 :   size_t len = strlen (name);
     526                 :      209910 :   char *tmp_name, *prefix;
     527                 :             : 
     528                 :      209910 :   prefix = XALLOCAVEC (char, len + strlen (suffix) + 2);
     529                 :      209910 :   memcpy (prefix, name, len);
     530                 :      209910 :   strcpy (prefix + len + 1, suffix);
     531                 :      209910 :   prefix[len] = symbol_table::symbol_suffix_separator ();
     532                 :      209910 :   ASM_FORMAT_PRIVATE_NAME (tmp_name, prefix, number);
     533                 :      209910 :   return get_identifier (tmp_name);
     534                 :             : }
     535                 :             : 
     536                 :             : /* Return a new assembler name for a clone of DECL.  Apart from the
     537                 :             :    string SUFFIX, the new name will end with the specified NUMBER.  If
     538                 :             :    clone numbering is not needed then the two argument
     539                 :             :    clone_function_name should be used instead.  */
     540                 :             : 
     541                 :             : tree
     542                 :      107875 : clone_function_name (tree decl, const char *suffix,
     543                 :             :                      unsigned long number)
     544                 :             : {
     545                 :      215750 :   return clone_function_name (
     546                 :      107875 :            IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (decl)), suffix, number);
     547                 :             : }
     548                 :             : 
     549                 :             : /* Return a new assembler name ending with the string SUFFIX for a
     550                 :             :    clone of DECL.  */
     551                 :             : 
     552                 :             : tree
     553                 :       61524 : clone_function_name (tree decl, const char *suffix)
     554                 :             : {
     555                 :       61524 :   tree identifier = DECL_ASSEMBLER_NAME (decl);
     556                 :             :   /* For consistency this needs to behave the same way as
     557                 :             :      ASM_FORMAT_PRIVATE_NAME does, but without the final number
     558                 :             :      suffix.  */
     559                 :       61524 :   char *separator = XALLOCAVEC (char, 2);
     560                 :       61524 :   separator[0] = symbol_table::symbol_suffix_separator ();
     561                 :       61524 :   separator[1] = 0;
     562                 :             : #if defined (NO_DOT_IN_LABEL) && defined (NO_DOLLAR_IN_LABEL)
     563                 :             :   const char *prefix = "__";
     564                 :             : #else
     565                 :       61524 :   const char *prefix = "";
     566                 :             : #endif
     567                 :       61524 :   char *result = ACONCAT ((prefix,
     568                 :             :                            IDENTIFIER_POINTER (identifier),
     569                 :             :                            separator,
     570                 :             :                            suffix,
     571                 :             :                            (char*)0));
     572                 :       61524 :   return get_identifier (result);
     573                 :             : }
     574                 :             : 
     575                 :             : 
     576                 :             : /* Create callgraph node clone with new declaration.  The actual body will be
     577                 :             :    copied later at compilation stage.  The name of the new clone will be
     578                 :             :    constructed from the name of the original node, SUFFIX and NUM_SUFFIX.
     579                 :             : 
     580                 :             :    TODO: after merging in ipa-sra use function call notes instead of args_to_skip
     581                 :             :    bitmap interface.
     582                 :             :    */
     583                 :             : cgraph_node *
     584                 :      107875 : cgraph_node::create_virtual_clone (const vec<cgraph_edge *> &redirect_callers,
     585                 :             :                                    vec<ipa_replace_map *, va_gc> *tree_map,
     586                 :             :                                    ipa_param_adjustments *param_adjustments,
     587                 :             :                                    const char * suffix, unsigned num_suffix)
     588                 :             : {
     589                 :      107875 :   tree old_decl = decl;
     590                 :      107875 :   cgraph_node *new_node = NULL;
     591                 :      107875 :   tree new_decl;
     592                 :      107875 :   size_t len, i;
     593                 :      107875 :   ipa_replace_map *map;
     594                 :      107875 :   char *name;
     595                 :             : 
     596                 :      107875 :   gcc_checking_assert (versionable);
     597                 :             :   /* TODO: It would be nice if we could recognize that param_adjustments do not
     598                 :             :      actually perform any changes, but at the moment let's require it simply
     599                 :             :      does not exist.  */
     600                 :      107875 :   gcc_assert (can_change_signature || !param_adjustments);
     601                 :             : 
     602                 :             :   /* Make a new FUNCTION_DECL tree node */
     603                 :      106791 :   if (!param_adjustments)
     604                 :        5077 :     new_decl = copy_node (old_decl);
     605                 :             :   else
     606                 :      102798 :     new_decl = param_adjustments->adjust_decl (old_decl);
     607                 :             : 
     608                 :             :   /* These pointers represent function body and will be populated only when clone
     609                 :             :      is materialized.  */
     610                 :      107875 :   gcc_assert (new_decl != old_decl);
     611                 :      107875 :   DECL_STRUCT_FUNCTION (new_decl) = NULL;
     612                 :      107875 :   DECL_ARGUMENTS (new_decl) = NULL;
     613                 :      107875 :   DECL_INITIAL (new_decl) = NULL;
     614                 :      107875 :   DECL_RESULT (new_decl) = NULL; 
     615                 :             :   /* We cannot do DECL_RESULT (new_decl) = NULL; here because of LTO partitioning
     616                 :             :      sometimes storing only clone decl instead of original.  */
     617                 :             : 
     618                 :             :   /* Generate a new name for the new version. */
     619                 :      107875 :   len = IDENTIFIER_LENGTH (DECL_NAME (old_decl));
     620                 :      107875 :   name = XALLOCAVEC (char, len + strlen (suffix) + 2);
     621                 :      107875 :   memcpy (name, IDENTIFIER_POINTER (DECL_NAME (old_decl)), len);
     622                 :      107875 :   strcpy (name + len + 1, suffix);
     623                 :      107875 :   name[len] = '.';
     624                 :      107875 :   DECL_NAME (new_decl) = get_identifier (name);
     625                 :      107875 :   SET_DECL_ASSEMBLER_NAME (new_decl,
     626                 :             :                            clone_function_name (old_decl, suffix, num_suffix));
     627                 :      107875 :   SET_DECL_RTL (new_decl, NULL);
     628                 :             : 
     629                 :      107875 :   new_node = create_clone (new_decl, count, false,
     630                 :             :                            redirect_callers, false, NULL, param_adjustments,
     631                 :             :                            suffix);
     632                 :             : 
     633                 :             :   /* Update the properties.
     634                 :             :      Make clone visible only within this translation unit.  Make sure
     635                 :             :      that is not weak also.
     636                 :             :      ??? We cannot use COMDAT linkage because there is no
     637                 :             :      ABI support for this.  */
     638                 :      107875 :   set_new_clone_decl_and_node_flags (new_node);
     639                 :      107875 :   new_node->ipcp_clone = ipcp_clone;
     640                 :      107875 :   if (tree_map)
     641                 :       11783 :     clone_info::get_create (new_node)->tree_map = tree_map;
     642                 :      107875 :   if (!implicit_section)
     643                 :      107834 :     new_node->set_section (*this);
     644                 :             : 
     645                 :             :   /* Clones of global symbols or symbols with unique names are unique.  */
     646                 :      107875 :   if ((TREE_PUBLIC (old_decl)
     647                 :       77667 :        && !DECL_EXTERNAL (old_decl)
     648                 :       70759 :        && !DECL_WEAK (old_decl)
     649                 :        2107 :        && !DECL_COMDAT (old_decl))
     650                 :      183435 :       || in_lto_p)
     651                 :        5713 :     new_node->unique_name = true;
     652                 :      126983 :   FOR_EACH_VEC_SAFE_ELT (tree_map, i, map)
     653                 :             :     {
     654                 :       19108 :       tree repl = map->new_tree;
     655                 :       19108 :       if (map->force_load_ref)
     656                 :             :         {
     657                 :         284 :           gcc_assert (TREE_CODE (repl) == ADDR_EXPR);
     658                 :         284 :           repl = get_base_address (TREE_OPERAND (repl, 0));
     659                 :             :         }
     660                 :       19108 :       new_node->maybe_create_reference (repl, NULL);
     661                 :             :     }
     662                 :             : 
     663                 :      107875 :   if (ipa_transforms_to_apply.exists ())
     664                 :       89720 :     new_node->ipa_transforms_to_apply
     665                 :       89720 :       = ipa_transforms_to_apply.copy ();
     666                 :             : 
     667                 :      107875 :   symtab->call_cgraph_duplication_hooks (this, new_node);
     668                 :             : 
     669                 :      107875 :   return new_node;
     670                 :             : }
     671                 :             : 
     672                 :             : /* callgraph node being removed from symbol table; see if its entry can be
     673                 :             :    replaced by other inline clone. 
     674                 :             :    INFO is clone info to attach to the new root.  */
     675                 :             : cgraph_node *
     676                 :    82422390 : cgraph_node::find_replacement (clone_info *info)
     677                 :             : {
     678                 :    82422390 :   cgraph_node *next_inline_clone, *replacement;
     679                 :             : 
     680                 :    82422390 :   for (next_inline_clone = clones;
     681                 :             :        next_inline_clone
     682                 :    82422390 :        && next_inline_clone->decl != decl;
     683                 :           0 :        next_inline_clone = next_inline_clone->next_sibling_clone)
     684                 :             :     ;
     685                 :             : 
     686                 :             :   /* If there is inline clone of the node being removed, we need
     687                 :             :      to put it into the position of removed node and reorganize all
     688                 :             :      other clones to be based on it.  */
     689                 :    82422390 :   if (next_inline_clone)
     690                 :             :     {
     691                 :      344247 :       cgraph_node *n;
     692                 :      344247 :       cgraph_node *new_clones;
     693                 :             : 
     694                 :      344247 :       replacement = next_inline_clone;
     695                 :             : 
     696                 :             :       /* Unlink inline clone from the list of clones of removed node.  */
     697                 :      344247 :       if (next_inline_clone->next_sibling_clone)
     698                 :      212528 :         next_inline_clone->next_sibling_clone->prev_sibling_clone
     699                 :      212528 :           = next_inline_clone->prev_sibling_clone;
     700                 :      344247 :       if (next_inline_clone->prev_sibling_clone)
     701                 :             :         {
     702                 :           0 :           gcc_assert (clones != next_inline_clone);
     703                 :           0 :           next_inline_clone->prev_sibling_clone->next_sibling_clone
     704                 :           0 :             = next_inline_clone->next_sibling_clone;
     705                 :             :         }
     706                 :             :       else
     707                 :             :         {
     708                 :      344247 :           gcc_assert (clones == next_inline_clone);
     709                 :      344247 :           clones = next_inline_clone->next_sibling_clone;
     710                 :             :         }
     711                 :             : 
     712                 :      344247 :       new_clones = clones;
     713                 :      344247 :       clones = NULL;
     714                 :             : 
     715                 :             :       /* Copy clone info.  */
     716                 :      344247 :       if (info)
     717                 :       52068 :         *clone_info::get_create (next_inline_clone) = *info;
     718                 :             : 
     719                 :             :       /* Now place it into clone tree at same level at NODE.  */
     720                 :      344247 :       next_inline_clone->clone_of = clone_of;
     721                 :      344247 :       next_inline_clone->prev_sibling_clone = NULL;
     722                 :      344247 :       next_inline_clone->next_sibling_clone = NULL;
     723                 :      344247 :       if (clone_of)
     724                 :             :         {
     725                 :        2337 :           if (clone_of->clones)
     726                 :        2337 :             clone_of->clones->prev_sibling_clone = next_inline_clone;
     727                 :        2337 :           next_inline_clone->next_sibling_clone = clone_of->clones;
     728                 :        2337 :           clone_of->clones = next_inline_clone;
     729                 :             :         }
     730                 :             : 
     731                 :             :       /* Merge the clone list.  */
     732                 :      344247 :       if (new_clones)
     733                 :             :         {
     734                 :      212528 :           if (!next_inline_clone->clones)
     735                 :      210477 :             next_inline_clone->clones = new_clones;
     736                 :             :           else
     737                 :             :             {
     738                 :             :               n = next_inline_clone->clones;
     739                 :        9669 :               while (n->next_sibling_clone)
     740                 :             :                 n = n->next_sibling_clone;
     741                 :        2051 :               n->next_sibling_clone = new_clones;
     742                 :        2051 :               new_clones->prev_sibling_clone = n;
     743                 :             :             }
     744                 :             :         }
     745                 :             : 
     746                 :             :       /* Update clone_of pointers.  */
     747                 :             :       n = new_clones;
     748                 :     2664772 :       while (n)
     749                 :             :         {
     750                 :     2320525 :           n->clone_of = next_inline_clone;
     751                 :     2320525 :           n = n->next_sibling_clone;
     752                 :             :         }
     753                 :             : 
     754                 :             :       /* Update order in order to be able to find a LTO section
     755                 :             :          with function body.  */
     756                 :      344247 :       replacement->order = order;
     757                 :             : 
     758                 :      344247 :       return replacement;
     759                 :             :     }
     760                 :             :   else
     761                 :             :     return NULL;
     762                 :             : }
     763                 :             : 
     764                 :             : /* Like cgraph_set_call_stmt but walk the clone tree and update all
     765                 :             :    clones sharing the same function body.  
     766                 :             :    When WHOLE_SPECULATIVE_EDGES is true, all three components of
     767                 :             :    speculative edge gets updated.  Otherwise we update only direct
     768                 :             :    call.  */
     769                 :             : 
     770                 :             : void
     771                 :     1294566 : cgraph_node::set_call_stmt_including_clones (gimple *old_stmt,
     772                 :             :                                              gcall *new_stmt,
     773                 :             :                                              bool update_speculative)
     774                 :             : {
     775                 :     1294566 :   cgraph_node *node;
     776                 :     1294566 :   cgraph_edge *master_edge = get_edge (old_stmt);
     777                 :             : 
     778                 :     1294566 :   if (master_edge)
     779                 :     1195730 :     cgraph_edge::set_call_stmt (master_edge, new_stmt, update_speculative);
     780                 :             : 
     781                 :     1294566 :   node = clones;
     782                 :     1294566 :   if (node)
     783                 :      870163 :     while (node != this)
     784                 :             :       {
     785                 :      673914 :         cgraph_edge *edge = node->get_edge (old_stmt);
     786                 :      673914 :         if (edge)
     787                 :             :           {
     788                 :      672115 :             edge = cgraph_edge::set_call_stmt (edge, new_stmt,
     789                 :             :                                                update_speculative);
     790                 :             :             /* If UPDATE_SPECULATIVE is false, it means that we are turning
     791                 :             :                speculative call into a real code sequence.  Update the
     792                 :             :                callgraph edges.  */
     793                 :      672115 :             if (edge->speculative && !update_speculative)
     794                 :             :               {
     795                 :           0 :                 cgraph_edge *indirect = edge->speculative_call_indirect_edge ();
     796                 :             : 
     797                 :           0 :                 for (cgraph_edge *next, *direct
     798                 :           0 :                         = edge->first_speculative_call_target ();
     799                 :           0 :                      direct;
     800                 :           0 :                      direct = next)
     801                 :             :                   {
     802                 :           0 :                     next = direct->next_speculative_call_target ();
     803                 :           0 :                     direct->speculative_call_target_ref ()->speculative = false;
     804                 :           0 :                     direct->speculative = false;
     805                 :             :                   }
     806                 :           0 :                 indirect->speculative = false;
     807                 :             :               }
     808                 :             :           }
     809                 :      673914 :         if (node->clones)
     810                 :             :           node = node->clones;
     811                 :      653860 :         else if (node->next_sibling_clone)
     812                 :             :           node = node->next_sibling_clone;
     813                 :             :         else
     814                 :             :           {
     815                 :      423601 :             while (node != this && !node->next_sibling_clone)
     816                 :      216303 :               node = node->clone_of;
     817                 :      207298 :             if (node != this)
     818                 :       11049 :               node = node->next_sibling_clone;
     819                 :             :           }
     820                 :             :       }
     821                 :     1294566 : }
     822                 :             : 
     823                 :             : /* Like cgraph_create_edge walk the clone tree and update all clones sharing
     824                 :             :    same function body.  If clones already have edge for OLD_STMT; only
     825                 :             :    update the edge same way as cgraph_set_call_stmt_including_clones does.
     826                 :             : 
     827                 :             :    TODO: COUNT and LOOP_DEPTH should be properly distributed based on relative
     828                 :             :    frequencies of the clones.  */
     829                 :             : 
     830                 :             : void
     831                 :           0 : cgraph_node::create_edge_including_clones (cgraph_node *callee,
     832                 :             :                                            gimple *old_stmt, gcall *stmt,
     833                 :             :                                            profile_count count,
     834                 :             :                                            cgraph_inline_failed_t reason)
     835                 :             : {
     836                 :           0 :   cgraph_node *node;
     837                 :             : 
     838                 :           0 :   if (!get_edge (stmt))
     839                 :             :     {
     840                 :           0 :       cgraph_edge *edge = create_edge (callee, stmt, count);
     841                 :           0 :       edge->inline_failed = reason;
     842                 :             :     }
     843                 :             : 
     844                 :           0 :   node = clones;
     845                 :           0 :   if (node)
     846                 :           0 :     while (node != this)
     847                 :             :       /* Thunk clones do not get updated while copying inline function body.  */
     848                 :           0 :       if (!node->thunk)
     849                 :             :         {
     850                 :           0 :           cgraph_edge *edge = node->get_edge (old_stmt);
     851                 :             : 
     852                 :             :           /* It is possible that clones already contain the edge while
     853                 :             :              master didn't.  Either we promoted indirect call into direct
     854                 :             :              call in the clone or we are processing clones of unreachable
     855                 :             :              master where edges has been removed.  */
     856                 :           0 :           if (edge)
     857                 :           0 :             edge = cgraph_edge::set_call_stmt (edge, stmt);
     858                 :           0 :           else if (! node->get_edge (stmt))
     859                 :             :             {
     860                 :           0 :               edge = node->create_edge (callee, stmt, count);
     861                 :           0 :               edge->inline_failed = reason;
     862                 :             :             }
     863                 :             : 
     864                 :           0 :           if (node->clones)
     865                 :             :             node = node->clones;
     866                 :           0 :           else if (node->next_sibling_clone)
     867                 :             :             node = node->next_sibling_clone;
     868                 :             :           else
     869                 :             :             {
     870                 :           0 :               while (node != this && !node->next_sibling_clone)
     871                 :           0 :                 node = node->clone_of;
     872                 :           0 :               if (node != this)
     873                 :           0 :                 node = node->next_sibling_clone;
     874                 :             :             }
     875                 :             :         }
     876                 :           0 : }
     877                 :             : 
     878                 :             : /* Remove the node from cgraph and all inline clones inlined into it.
     879                 :             :    Skip however removal of FORBIDDEN_NODE and return true if it needs to be
     880                 :             :    removed.  This allows to call the function from outer loop walking clone
     881                 :             :    tree.  */
     882                 :             : 
     883                 :             : bool
     884                 :          36 : cgraph_node::remove_symbol_and_inline_clones (cgraph_node *forbidden_node)
     885                 :             : {
     886                 :          36 :   cgraph_edge *e, *next;
     887                 :          36 :   bool found = false;
     888                 :             : 
     889                 :          36 :   if (this == forbidden_node)
     890                 :             :     {
     891                 :           0 :       cgraph_edge::remove (callers);
     892                 :           0 :       return true;
     893                 :             :     }
     894                 :          46 :   for (e = callees; e; e = next)
     895                 :             :     {
     896                 :          10 :       next = e->next_callee;
     897                 :          10 :       if (!e->inline_failed)
     898                 :           0 :         found |= e->callee->remove_symbol_and_inline_clones (forbidden_node);
     899                 :             :     }
     900                 :          36 :   remove ();
     901                 :          36 :   return found;
     902                 :             : }
     903                 :             : 
     904                 :             : /* The edges representing the callers of the NEW_VERSION node were
     905                 :             :    fixed by cgraph_function_versioning (), now the call_expr in their
     906                 :             :    respective tree code should be updated to call the NEW_VERSION.  */
     907                 :             : 
     908                 :             : static void
     909                 :       47671 : update_call_expr (cgraph_node *new_version)
     910                 :             : {
     911                 :       47671 :   cgraph_edge *e;
     912                 :             : 
     913                 :       47671 :   gcc_assert (new_version);
     914                 :             : 
     915                 :             :   /* Update the call expr on the edges to call the new version.  */
     916                 :       47671 :   for (e = new_version->callers; e; e = e->next_caller)
     917                 :             :     {
     918                 :           0 :       function *inner_function = DECL_STRUCT_FUNCTION (e->caller->decl);
     919                 :           0 :       gimple_call_set_fndecl (e->call_stmt, new_version->decl);
     920                 :           0 :       maybe_clean_eh_stmt_fn (inner_function, e->call_stmt);
     921                 :             :     }
     922                 :       47671 : }
     923                 :             : 
     924                 :             : 
     925                 :             : /* Create a new cgraph node which is the new version of
     926                 :             :    callgraph node.  REDIRECT_CALLERS holds the callers
     927                 :             :    edges which should be redirected to point to
     928                 :             :    NEW_VERSION.  ALL the callees edges of the node
     929                 :             :    are cloned to the new version node.  Return the new
     930                 :             :    version node. 
     931                 :             : 
     932                 :             :    If non-NULL BLOCK_TO_COPY determine what basic blocks 
     933                 :             :    was copied to prevent duplications of calls that are dead
     934                 :             :    in the clone.  */
     935                 :             : 
     936                 :             : cgraph_node *
     937                 :       50905 : cgraph_node::create_version_clone (tree new_decl,
     938                 :             :                                   vec<cgraph_edge *> redirect_callers,
     939                 :             :                                   bitmap bbs_to_copy,
     940                 :             :                                   const char *suffix)
     941                 :             :  {
     942                 :       50905 :    cgraph_node *new_version;
     943                 :       50905 :    cgraph_edge *e;
     944                 :       50905 :    unsigned i;
     945                 :             : 
     946                 :       50905 :    new_version = cgraph_node::create (new_decl);
     947                 :             : 
     948                 :       50905 :    new_version->analyzed = analyzed;
     949                 :       50905 :    new_version->definition = definition;
     950                 :       50905 :    new_version->local = local;
     951                 :       50905 :    new_version->externally_visible = false;
     952                 :       50905 :    new_version->no_reorder = no_reorder;
     953                 :       50905 :    new_version->local = new_version->definition;
     954                 :       50905 :    new_version->inlined_to = inlined_to;
     955                 :       50905 :    new_version->rtl = rtl;
     956                 :       50905 :    new_version->count = count;
     957                 :       50905 :    new_version->unit_id = unit_id;
     958                 :       50905 :    new_version->merged_comdat = merged_comdat;
     959                 :       50905 :    new_version->merged_extern_inline = merged_extern_inline;
     960                 :             : 
     961                 :      282524 :    for (e = callees; e; e=e->next_callee)
     962                 :      231619 :      if (!bbs_to_copy
     963                 :      231619 :          || bitmap_bit_p (bbs_to_copy, gimple_bb (e->call_stmt)->index))
     964                 :      184461 :        e->clone (new_version, e->call_stmt,
     965                 :             :                  e->lto_stmt_uid, count, count,
     966                 :             :                  true);
     967                 :       58142 :    for (e = indirect_calls; e; e=e->next_callee)
     968                 :        7237 :      if (!bbs_to_copy
     969                 :        7237 :          || bitmap_bit_p (bbs_to_copy, gimple_bb (e->call_stmt)->index))
     970                 :        4870 :        e->clone (new_version, e->call_stmt,
     971                 :             :                  e->lto_stmt_uid, count, count,
     972                 :             :                  true);
     973                 :       50905 :    FOR_EACH_VEC_ELT (redirect_callers, i, e)
     974                 :             :      {
     975                 :             :        /* Redirect calls to the old version node to point to its new
     976                 :             :           version.  */
     977                 :           0 :        e->redirect_callee (new_version);
     978                 :             :      }
     979                 :             : 
     980                 :       50905 :    dump_callgraph_transformation (this, new_version, suffix);
     981                 :             : 
     982                 :       50905 :    return new_version;
     983                 :             :  }
     984                 :             : 
     985                 :             : /* Perform function versioning.
     986                 :             :    Function versioning includes copying of the tree and
     987                 :             :    a callgraph update (creating a new cgraph node and updating
     988                 :             :    its callees and callers).
     989                 :             : 
     990                 :             :    REDIRECT_CALLERS varray includes the edges to be redirected
     991                 :             :    to the new version.
     992                 :             : 
     993                 :             :    TREE_MAP is a mapping of tree nodes we want to replace with
     994                 :             :    new ones (according to results of prior analysis).
     995                 :             : 
     996                 :             :    If non-NULL ARGS_TO_SKIP determine function parameters to remove
     997                 :             :    from new version.
     998                 :             :    If SKIP_RETURN is true, the new version will return void.
     999                 :             :    If non-NULL BLOCK_TO_COPY determine what basic blocks to copy.
    1000                 :             :    If non_NULL NEW_ENTRY determine new entry BB of the clone.
    1001                 :             : 
    1002                 :             :    If TARGET_ATTRIBUTES is non-null, when creating a new declaration,
    1003                 :             :    add the attributes to DECL_ATTRIBUTES.  And call valid_attribute_p
    1004                 :             :    that will promote value of the attribute DECL_FUNCTION_SPECIFIC_TARGET
    1005                 :             :    of the declaration.
    1006                 :             : 
    1007                 :             :    If VERSION_DECL is set true, use clone_function_name_numbered for the
    1008                 :             :    function clone.  Otherwise, use clone_function_name.
    1009                 :             : 
    1010                 :             :    Return the new version's cgraph node.  */
    1011                 :             : 
    1012                 :             : cgraph_node *
    1013                 :       47672 : cgraph_node::create_version_clone_with_body
    1014                 :             :   (vec<cgraph_edge *> redirect_callers,
    1015                 :             :    vec<ipa_replace_map *, va_gc> *tree_map,
    1016                 :             :    ipa_param_adjustments *param_adjustments,
    1017                 :             :    bitmap bbs_to_copy, basic_block new_entry_block, const char *suffix,
    1018                 :             :    tree target_attributes, bool version_decl)
    1019                 :             : {
    1020                 :       47672 :   tree old_decl = decl;
    1021                 :       47672 :   cgraph_node *new_version_node = NULL;
    1022                 :       47672 :   tree new_decl;
    1023                 :             : 
    1024                 :       47672 :   if (!tree_versionable_function_p (old_decl))
    1025                 :             :     return NULL;
    1026                 :             : 
    1027                 :             :   /* TODO: Restore an assert that we do not change signature if
    1028                 :             :      can_change_signature is false.  We cannot just check that
    1029                 :             :      param_adjustments is NULL because unfortunately ipa-split removes return
    1030                 :             :      values from such functions.  */
    1031                 :             : 
    1032                 :             :   /* Make a new FUNCTION_DECL tree node for the new version. */
    1033                 :       47672 :   if (param_adjustments)
    1034                 :       22983 :     new_decl = param_adjustments->adjust_decl (old_decl);
    1035                 :             :   else
    1036                 :       24689 :     new_decl = copy_node (old_decl);
    1037                 :             : 
    1038                 :             :   /* Generate a new name for the new version. */
    1039                 :       47672 :   tree fnname = (version_decl ? clone_function_name_numbered (old_decl, suffix)
    1040                 :         102 :                 : clone_function_name (old_decl, suffix));
    1041                 :       47672 :   DECL_NAME (new_decl) = fnname;
    1042                 :       47672 :   SET_DECL_ASSEMBLER_NAME (new_decl, fnname);
    1043                 :       47672 :   SET_DECL_RTL (new_decl, NULL);
    1044                 :             : 
    1045                 :       47672 :   DECL_VIRTUAL_P (new_decl) = 0;
    1046                 :             : 
    1047                 :       47672 :   if (target_attributes)
    1048                 :             :     {
    1049                 :         102 :       DECL_ATTRIBUTES (new_decl) = target_attributes;
    1050                 :             : 
    1051                 :         102 :       location_t saved_loc = input_location;
    1052                 :         102 :       tree v = TREE_VALUE (target_attributes);
    1053                 :         102 :       input_location = DECL_SOURCE_LOCATION (new_decl);
    1054                 :         102 :       bool r;
    1055                 :         102 :       tree name_id = get_attribute_name (target_attributes);
    1056                 :         102 :       const char *name_str = IDENTIFIER_POINTER (name_id);
    1057                 :         102 :       if (strcmp (name_str, "target") == 0)
    1058                 :         102 :         r = targetm.target_option.valid_attribute_p (new_decl, name_id, v, 1);
    1059                 :           0 :       else if (strcmp (name_str, "target_version") == 0)
    1060                 :           0 :         r = targetm.target_option.valid_version_attribute_p (new_decl, name_id,
    1061                 :             :                                                              v, 1);
    1062                 :             :       else
    1063                 :           0 :         gcc_unreachable();
    1064                 :             : 
    1065                 :         102 :       input_location = saved_loc;
    1066                 :         102 :       if (!r)
    1067                 :             :         return NULL;
    1068                 :             :     }
    1069                 :             : 
    1070                 :             :   /* When the old decl was a con-/destructor make sure the clone isn't.  */
    1071                 :       47671 :   DECL_STATIC_CONSTRUCTOR (new_decl) = 0;
    1072                 :       47671 :   DECL_STATIC_DESTRUCTOR (new_decl) = 0;
    1073                 :       47671 :   DECL_SET_IS_OPERATOR_NEW (new_decl, 0);
    1074                 :       47671 :   DECL_SET_IS_OPERATOR_DELETE (new_decl, 0);
    1075                 :       47671 :   DECL_IS_REPLACEABLE_OPERATOR (new_decl) = 0;
    1076                 :             : 
    1077                 :             :   /* Create the new version's call-graph node.
    1078                 :             :      and update the edges of the new node. */
    1079                 :       47671 :   new_version_node = create_version_clone (new_decl, redirect_callers,
    1080                 :             :                                           bbs_to_copy, suffix);
    1081                 :             : 
    1082                 :       47671 :   if (ipa_transforms_to_apply.exists ())
    1083                 :           0 :     new_version_node->ipa_transforms_to_apply
    1084                 :           0 :       = ipa_transforms_to_apply.copy ();
    1085                 :             :   /* Copy the OLD_VERSION_NODE function tree to the new version.  */
    1086                 :       47671 :   tree_function_versioning (old_decl, new_decl, tree_map, param_adjustments,
    1087                 :             :                             false, bbs_to_copy, new_entry_block);
    1088                 :             : 
    1089                 :             :   /* Update the new version's properties.
    1090                 :             :      Make The new version visible only within this translation unit.  Make sure
    1091                 :             :      that is not weak also.
    1092                 :             :      ??? We cannot use COMDAT linkage because there is no
    1093                 :             :      ABI support for this.  */
    1094                 :       47671 :   new_version_node->make_decl_local ();
    1095                 :       47671 :   DECL_VIRTUAL_P (new_version_node->decl) = 0;
    1096                 :       47671 :   new_version_node->externally_visible = 0;
    1097                 :       47671 :   new_version_node->local = 1;
    1098                 :       47671 :   new_version_node->lowered = true;
    1099                 :       47671 :   if (!implicit_section)
    1100                 :       47651 :     new_version_node->set_section (*this);
    1101                 :             :   /* Clones of global symbols or symbols with unique names are unique.  */
    1102                 :       47671 :   if ((TREE_PUBLIC (old_decl)
    1103                 :       44505 :        && !DECL_EXTERNAL (old_decl)
    1104                 :       37173 :        && !DECL_WEAK (old_decl)
    1105                 :       12769 :        && !DECL_COMDAT (old_decl))
    1106                 :       79407 :       || in_lto_p)
    1107                 :       12809 :     new_version_node->unique_name = true;
    1108                 :             : 
    1109                 :             :   /* Update the call_expr on the edges to call the new version node. */
    1110                 :       47671 :   update_call_expr (new_version_node);
    1111                 :             : 
    1112                 :       47671 :   symtab->call_cgraph_insertion_hooks (new_version_node);
    1113                 :       47671 :   return new_version_node;
    1114                 :             : }
    1115                 :             : 
    1116                 :             : /* Remove the node from the tree of virtual and inline clones and make it a
    1117                 :             :    standalone node - not a clone any more.  */
    1118                 :             : 
    1119                 :      107107 : void cgraph_node::remove_from_clone_tree ()
    1120                 :             : {
    1121                 :      107107 :   if (next_sibling_clone)
    1122                 :         301 :     next_sibling_clone->prev_sibling_clone = prev_sibling_clone;
    1123                 :      107107 :   if (prev_sibling_clone)
    1124                 :         517 :     prev_sibling_clone->next_sibling_clone = next_sibling_clone;
    1125                 :             :   else
    1126                 :      106590 :     clone_of->clones = next_sibling_clone;
    1127                 :      107107 :   next_sibling_clone = NULL;
    1128                 :      107107 :   prev_sibling_clone = NULL;
    1129                 :      107107 :   clone_of = NULL;
    1130                 :      107107 : }
    1131                 :             : 
    1132                 :             : /* Given virtual clone, turn it into actual clone.  */
    1133                 :             : 
    1134                 :             : void
    1135                 :      107107 : cgraph_node::materialize_clone ()
    1136                 :             : {
    1137                 :      107107 :   clone_info *info = clone_info::get (this);
    1138                 :      107107 :   clone_of->get_untransformed_body ();
    1139                 :      107107 :   former_clone_of = clone_of->decl;
    1140                 :      107107 :   if (clone_of->former_clone_of)
    1141                 :        3447 :     former_clone_of = clone_of->former_clone_of;
    1142                 :      107107 :   if (symtab->dump_file)
    1143                 :             :     {
    1144                 :           0 :       fprintf (symtab->dump_file, "cloning %s to %s\n",
    1145                 :           0 :                clone_of->dump_name (),
    1146                 :             :                dump_name ());
    1147                 :           0 :       if (info && info->tree_map)
    1148                 :             :         {
    1149                 :           0 :           fprintf (symtab->dump_file, "    replace map:");
    1150                 :           0 :           for (unsigned int i = 0;
    1151                 :           0 :                i < vec_safe_length (info->tree_map);
    1152                 :             :                i++)
    1153                 :             :             {
    1154                 :           0 :               ipa_replace_map *replace_info;
    1155                 :           0 :               replace_info = (*info->tree_map)[i];
    1156                 :           0 :               fprintf (symtab->dump_file, "%s %i -> ",
    1157                 :             :                        i ? "," : "", replace_info->parm_num);
    1158                 :           0 :               print_generic_expr (symtab->dump_file,
    1159                 :             :                                   replace_info->new_tree);
    1160                 :             :             }
    1161                 :           0 :           fprintf (symtab->dump_file, "\n");
    1162                 :             :         }
    1163                 :           0 :       if (info && info->param_adjustments)
    1164                 :           0 :         info->param_adjustments->dump (symtab->dump_file);
    1165                 :             :     }
    1166                 :      107107 :   clear_stmts_in_references ();
    1167                 :             :   /* Copy the OLD_VERSION_NODE function tree to the new version.  */
    1168                 :      107107 :   tree_function_versioning (clone_of->decl, decl,
    1169                 :             :                             info ? info->tree_map : NULL,
    1170                 :             :                             info ? info->param_adjustments : NULL,
    1171                 :             :                             true, NULL, NULL);
    1172                 :      107107 :   if (symtab->dump_file)
    1173                 :             :     {
    1174                 :           0 :       dump_function_to_file (clone_of->decl, symtab->dump_file,
    1175                 :             :                              dump_flags);
    1176                 :           0 :       dump_function_to_file (decl, symtab->dump_file, dump_flags);
    1177                 :             :     }
    1178                 :             : 
    1179                 :      107107 :   cgraph_node *this_clone_of = clone_of;
    1180                 :             :   /* Function is no longer clone.  */
    1181                 :      107107 :   remove_from_clone_tree ();
    1182                 :      107107 :   if (!this_clone_of->analyzed && !this_clone_of->clones)
    1183                 :      104381 :     this_clone_of->release_body ();
    1184                 :      107107 : }
    1185                 :             : 
    1186                 :             : #include "gt-cgraphclones.h"
        

Generated by: LCOV version 2.0-1

LCOV profile is generated on x86_64 machine using following configure options: configure --disable-bootstrap --enable-coverage=opt --enable-languages=c,c++,fortran,go,jit,lto,rust,m2 --enable-host-shared. GCC test suite is run with the built compiler.