#include "insn-codes.h"#include "insn-opinit.h"
Go to the source code of this file.
Data Structures | |
| struct | direct_internal_fn_info |
Macros | |
| #define | IFN_UNIQUE_CODES |
| #define | DEF(X) |
| #define | IFN_GOACC_LOOP_CODES DEF(CHUNKS), DEF(STEP), DEF(OFFSET), DEF(BOUND) |
| #define | DEF(X) |
| #define | IFN_GOACC_REDUCTION_CODES DEF(SETUP), DEF(INIT), DEF(FINI), DEF(TEARDOWN) |
| #define | DEF(X) |
| #define | MASK_LOAD_ELSE_ZERO -1 |
| #define | MASK_LOAD_ELSE_M1 -2 |
| #define | MASK_LOAD_ELSE_UNDEFINED -3 |
| #define | VECT_PARTIAL_BIAS_UNSUPPORTED 127 |
Enumerations | |
| enum | ifn_unique_kind { IFN_UNIQUE_CODES } |
| enum | ifn_goacc_loop_kind { IFN_GOACC_LOOP_CODES } |
| enum | ifn_goacc_reduction_kind { IFN_GOACC_REDUCTION_CODES } |
| enum | { ATOMIC_OP_FETCH_CMP_0_EQ = 0 , ATOMIC_OP_FETCH_CMP_0_NE = 1 , ATOMIC_OP_FETCH_CMP_0_LT = 2 , ATOMIC_OP_FETCH_CMP_0_LE = 3 , ATOMIC_OP_FETCH_CMP_0_GT = 4 , ATOMIC_OP_FETCH_CMP_0_GE = 5 } |
Variables | |
| const char *const | internal_fn_name_array [] |
| const int | internal_fn_flags_array [] |
| const_tree | internal_fn_fnspec_array [IFN_LAST+1] |
| const direct_internal_fn_info | direct_internal_fn_array [IFN_LAST+1] |
| #define DEF | ( | X | ) |
| #define DEF | ( | X | ) |
| #define DEF | ( | X | ) |
INTEGER_CST values for IFN_GOACC_LOOP arg-0. Allows the precise stepping of the compute geometry over the loop iterations to be deferred until it is known which compiler is generating the code. The action is encoded in a constant first argument. CHUNK_MAX = LOOP (CODE_CHUNKS, DIR, RANGE, STEP, CHUNK_SIZE, MASK) STEP = LOOP (CODE_STEP, DIR, RANGE, STEP, CHUNK_SIZE, MASK) OFFSET = LOOP (CODE_OFFSET, DIR, RANGE, STEP, CHUNK_SIZE, MASK, CHUNK_NO) BOUND = LOOP (CODE_BOUND, DIR, RANGE, STEP, CHUNK_SIZE, MASK, OFFSET) DIR - +1 for up loop, -1 for down loop RANGE - Range of loop (END - BASE) STEP - iteration step size CHUNKING - size of chunking, (constant zero for no chunking) CHUNK_NO - chunk number MASK - partitioning mask.
Referenced by dump_gimple_call_args().
The GOACC_REDUCTION function defines a generic interface to support
gang, worker and vector reductions. All calls are of the following
form:
V = REDUCTION (CODE, REF_TO_RES, LOCAL_VAR, LEVEL, OP, OFFSET)
REF_TO_RES - is a reference to the original reduction varl, may be NULL
LOCAL_VAR is the intermediate reduction variable
LEVEL corresponds to the GOMP_DIM of the reduction
OP is the tree code of the reduction operation
OFFSET may be used as an offset into a reduction array for the
reductions occuring at this level.
In general the return value is LOCAL_VAR, which creates a data
dependency between calls operating on the same reduction.
Referenced by dump_gimple_call_args().
| #define IFN_UNIQUE_CODES |
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
FORK and JOIN mark the points at which OpenACC partitioned
execution is entered or exited.
DEP_VAR = UNIQUE ({FORK,JOIN}, DEP_VAR, AXIS)
HEAD_MARK and TAIL_MARK are used to demark the sequence entering
or leaving partitioned execution.
DEP_VAR = UNIQUE ({HEAD,TAIL}_MARK, REMAINING_MARKS, ...PRIMARY_FLAGS)
The PRIMARY_FLAGS only occur on the first HEAD_MARK of a sequence.
PRIVATE captures variables to be made private at the surrounding parallelism
level.
Referenced by dump_gimple_call_args().
| #define MASK_LOAD_ELSE_M1 -2 |
Referenced by get_supported_else_vals(), supported_else_val_p(), and vect_get_mask_load_else().
| #define MASK_LOAD_ELSE_UNDEFINED -3 |
Referenced by get_supported_else_vals(), supported_else_val_p(), and vect_get_mask_load_else().
| #define MASK_LOAD_ELSE_ZERO -1 |
Integer constants representing which else value is supported for masked load functions.
Referenced by get_supported_else_vals(), predicate_load_or_store(), read_vector_array(), supported_else_val_p(), vect_get_mask_load_else(), vect_recog_gather_scatter_pattern(), and vectorizable_load().
| #define VECT_PARTIAL_BIAS_UNSUPPORTED 127 |
Referenced by internal_len_load_store_bias(), and vect_verify_loop_lens().
| anonymous enum |
| enum ifn_goacc_loop_kind |
| enum ifn_unique_kind |
|
extern |
Return true if FN is an associative binary operation.
Referenced by associative_binary_op_p().
|
extern |
Return true if STMT can be interpreted as a conditional tree code
operation of the form:
LHS = COND ? OP (RHS1, ...) : ELSE;
operating elementwise if the operands are vectors. This includes
the case of an all-true COND, so that the operation always happens.
There is an alternative approach to interpret the STMT when the operands
are vectors which is the operation predicated by both conditional mask
and loop control length, the equivalent C code:
for (int i = 0; i < NUNTIS; i++)
{
if (i < LEN + BIAS && COND[i])
LHS[i] = A[i] CODE B[i];
else
LHS[i] = ELSE[i];
}
When returning true, set:
- *COND_OUT to the condition COND, or to NULL_TREE if the condition
is known to be all-true
- *CODE_OUT to the tree code
- OPS[I] to operand I of *CODE_OUT
- *ELSE_OUT to the fallback value ELSE, or to NULL_TREE if the
condition is known to be all true.
- *LEN to the len argument if it COND_LEN_* operations or to NULL_TREE.
- *BIAS to the bias argument if it COND_LEN_* operations or to NULL_TREE.
References conditional_internal_fn_code(), dyn_cast(), gimple_assign_rhs1(), gimple_assign_rhs2(), gimple_assign_rhs3(), gimple_assign_rhs_code(), gimple_call_arg(), gimple_call_internal_fn(), gimple_call_internal_p(), gimple_call_num_args(), i, integer_truep(), internal_fn_len_index(), and NULL_TREE.
Referenced by convert_mult_to_fma(), and convert_mult_to_fma_1().
|
extern |
Return true if FN is a binary operation and if FN is commutative.
Referenced by commutative_binary_op_p(), and first_commutative_argument().
|
extern |
Return true if FN is a ternary operation and if its first two arguments are commutative.
Referenced by commutative_ternary_op_p(), and first_commutative_argument().
|
extern |
If IFN implements the conditional form of a tree code, return that tree code, otherwise return ERROR_MARK.
References CASE, and FOR_EACH_CODE_MAPPING.
Referenced by can_interpret_as_conditional_op_p(), check_reduction_path(), internal_fn_mask_index(), try_conditional_simplification(), vect_transform_loop(), vect_transform_reduction(), vectorizable_reduction(), and vectorize_fold_left_reduction().
|
inline |
Return optab information about internal function FN. Only meaningful if direct_internal_fn_p (FN).
References direct_internal_fn_array, direct_internal_fn_p(), and gcc_checking_assert.
Referenced by direct_internal_fn_supported_p(), direct_internal_fn_types(), direct_internal_fn_types(), and vectorizable_internal_function().
|
extern |
Return the optab used by internal function FN.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References gcc_unreachable.
Referenced by internal_check_ptrs_fn_supported_p(), internal_gather_scatter_fn_supported_p(), internal_len_load_store_bias(), and supportable_widening_operation().
|
inline |
Return true if FN is mapped directly to an optab.
References direct_internal_fn_array.
Referenced by build_call_internal(), direct_internal_fn(), directly_supported_p(), directly_supported_p(), maybe_build_call_expr_loc(), ssa_is_replaceable_p(), try_vectorize_loop_1(), and vectorizable_internal_function().
|
extern |
Return true if the STMT is supported when the optimization type is OPT_TYPE, given that STMT is a call to a direct internal function.
References direct_internal_fn_supported_p(), direct_internal_fn_types(), and gimple_call_internal_fn().
|
inline |
Return true if FN is supported for types TYPE0 and TYPE1 when the optimization type is OPT_TYPE. The types are those associated with the "type0" and "type1" fields of FN's direct_internal_fn_info structure.
References direct_internal_fn_supported_p().
|
extern |
Return true if FN is supported for type TYPE when the optimization type is OPT_TYPE. The caller knows that the "type0" and "type1" fields of FN's direct_internal_fn_info structure are the same.
References direct_internal_fn(), direct_internal_fn_supported_p(), gcc_checking_assert, direct_internal_fn_info::type0, and direct_internal_fn_info::type1.
|
extern |
Return true if FN is supported for the types in TYPES when the optimization type is OPT_TYPE. The types are those associated with the "type0" and "type1" fields of FN's direct_internal_fn_info structure.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References gcc_unreachable, and type_pair_strictly_matches_mode_p().
Referenced by build_call_internal(), build_cltz_expr(), build_popcount_expr(), build_saturation_binary_arith_call_and_insert(), build_saturation_binary_arith_call_and_replace(), can_log2(), can_produce_all_loop_masks_p(), cond_removal_in_builtin_zero_pattern(), convert_mult_to_fma(), direct_internal_fn_supported_p(), direct_internal_fn_supported_p(), direct_internal_fn_supported_p(), directly_supported_p(), directly_supported_p(), expand_crc_optab_fn(), fold_builtin_bit_query(), get_masked_reduction_fn(), gimple_expand_vec_cond_expr(), internal_fn_reciprocal(), match_saturation_trunc(), match_single_bit_test(), match_unsigned_saturation_trunc(), maybe_build_call_expr_loc(), optimize_count_trailing_zeroes(), reassociate_bb(), replacement_internal_fn(), try_vectorize_loop_1(), use_mask_by_cond_expr_p(), vect_analyze_loop_2(), vect_gen_while(), vect_get_strided_load_store_ops(), vect_pattern_validate_optab(), vect_recog_abd_pattern(), vect_recog_average_pattern(), vect_recog_build_binary_gimple_stmt(), vect_recog_ctz_ffs_pattern(), vect_recog_divmod_pattern(), vect_recog_mulhs_pattern(), vect_recog_popcount_clz_ctz_ffs_pattern(), vect_recog_pow_pattern(), vect_recog_sat_sub_pattern_transform(), vect_recog_sat_trunc_pattern(), vect_reduction_update_partial_vector_usage(), vectorizable_bb_reduc_epilogue(), vectorizable_call(), vectorizable_condition(), vectorizable_early_exit(), vectorizable_internal_function(), vectorizable_live_operation(), vectorizable_operation(), vectorizable_reduction(), and vectorized_internal_fn_supported_p().
|
extern |
CALL is a call whose return type and arguments are in principle compatible with FN (which satisfies direct_internal_fn_p). Return the types that should be used to determine whether the target supports FN.
References direct_internal_fn(), gimple_call_arg(), gimple_call_lhs(), TREE_TYPE, direct_internal_fn_info::type0, and direct_internal_fn_info::type1.
|
extern |
RETURN_TYPE and ARGS are a return type and argument list that are in principle compatible with FN (which satisfies direct_internal_fn_p). Return the types that should be used to determine whether the target supports FN.
References direct_internal_fn(), TREE_TYPE, direct_internal_fn_info::type0, and direct_internal_fn_info::type1.
Referenced by build_call_internal(), direct_internal_fn_supported_p(), expand_convert_optab_fn(), expand_direct_optab_fn(), internal_fn_reciprocal(), maybe_build_call_expr_loc(), and replacement_internal_fn().
|
extern |
Add sub/add overflow checking to the statement STMT. CODE says whether the operation is +, or -.
References add_reg_br_prob_note(), any_condjump_p(), BITS_PER_WORD, cfun, const0_rtx, CONST_SCALAR_INT_P, create_fixed_operand(), create_input_operand(), create_output_operand(), delete_insns_since(), do_compare_rtx_and_jump(), do_pending_stack_adjust(), emit_jump(), emit_label(), error_mark_node, profile_probability::even(), expand_arith_overflow_result_store(), expand_arith_set_overflow(), expand_binop(), expand_expr(), expand_normal(), expand_ubsan_result_store(), expand_unop(), EXPAND_WRITE, find_reg_note(), gcc_assert, gen_label_rtx(), gen_reg_rtx(), get_last_insn(), GET_MODE_PRECISION(), get_range_pos_neg(), immed_wide_int_const(), JUMP_P, last, maybe_expand_insn(), wi::min_value(), expand_operand::mode, NULL, NULL_RTX, optab_handler(), OPTAB_LIB_WIDEN, pop_temp_slots(), PROFILE_ABSENT, profile_status_for_fn, push_temp_slots(), REG_P, SCALAR_INT_TYPE_MODE, SIGNED, expand_operand::target, TREE_TYPE, ubsan_build_overflow_builtin(), profile_probability::very_likely(), profile_probability::very_unlikely(), and write_complex_part().
Referenced by expand_arith_overflow(), expand_doubleword_mod(), expand_UBSAN_CHECK_ADD(), expand_UBSAN_CHECK_SUB(), and expand_vector_ubsan_overflow().
|
extern |
|
extern |
|
extern |
References expand_bitquery(), and expand_unary_optab_fn.
|
extern |
References expand_bitquery(), and expand_unary_optab_fn.
|
extern |
References expand_bitquery(), and expand_unary_optab_fn.
|
extern |
References emit_library_call_value_1(), expand_normal(), gimple_call_arg(), i, init_one_libfunc(), LCT_NORMAL, NULL_RTX, and ptr_mode.
|
extern |
References expand_bitquery(), and expand_unary_optab_fn.
|
extern |
|
extern |
Expand STMT, which is a call to internal function FN.
References expand_internal_call(), and gimple_call_internal_fn().
|
extern |
Expand STMT as though it were a call to internal function FN.
References internal_fn_expanders.
Referenced by expand_call_stmt(), expand_expr_real_1(), and expand_internal_call().
|
extern |
References gcc_unreachable.
|
extern |
References emit_library_call_value_1(), expand_normal(), gimple_call_arg(), i, init_one_libfunc(), LCT_NORMAL, NULL_RTX, and ptr_mode.
|
extern |
References expand_bitquery(), and expand_unary_optab_fn.
|
extern |
References gcc_unreachable.
|
extern |
References const0_rtx, const1_rtx, const2_rtx, constm1_rtx, convert_to_mode(), do_pending_stack_adjust(), dump_file, dump_flags, emit_insn(), emit_move_insn(), emit_store_flag(), end_sequence(), expand_bitquery(), expand_normal(), expand_simple_binop(), expand_unary_optab_fn, GET_MODE, gimple_call_arg(), gimple_call_lhs(), gimple_call_num_args(), integer_minus_onep(), integer_zerop(), expand_operand::mode, NULL_RTX, OPTAB_WIDEN, optimize_insn_for_speed_p(), seq_cost(), start_sequence(), TDF_DETAILS, TREE_TYPE, and TYPE_MODE.
|
extern |
References gcc_unreachable.
|
extern |
|
extern |
|
extern |
If FN is commutative in two consecutive arguments, return the index of the first, otherwise return -1.
References commutative_binary_fn_p(), and commutative_ternary_fn_p().
|
extern |
Return a function that only performs internal function FN when a certain condition is met and that uses a given fallback value otherwise. In other words, the returned function FN' is such that: LHS = FN' (COND, A1, ... An, ELSE) is equivalent to the C expression: LHS = COND ? FN (A1, ..., An) : ELSE; operating elementwise if the operands are vectors. Return IFN_LAST if no such function exists.
References CASE, and FOR_EACH_COND_FN_PAIR.
|
extern |
Return a function that only performs CODE when a certain condition is met and that uses a given fallback value otherwise. For example, if CODE is a binary operation associated with conditional function FN: LHS = FN (COND, A, B, ELSE) is equivalent to the C expression: LHS = COND ? A CODE B : ELSE; operating elementwise if the operands are vectors. Return IFN_LAST if no such function exists.
References CASE, and FOR_EACH_CODE_MAPPING.
|
extern |
Like get_conditional_internal_fn, but return a function that
additionally restricts the operation to the leading elements
of a vector. The number of elements to process is given by a length
and bias pair, as for IFN_LOAD_LEN. The values of the remaining
elements are taken from the fallback ("else") argument.
For example, if CODE is a binary operation associated with FN:
LHS = FN (COND, A, B, ELSE, LEN, BIAS)
is equivalent to the C code:
for (int i = 0; i < NUNITS; i++)
{
if (i < LEN + BIAS && COND[i])
LHS[i] = A[i] CODE B[i];
else
LHS[i] = ELSE[i];
}
References CASE, and FOR_EACH_CODE_MAPPING.
Referenced by vectorizable_operation().
|
extern |
If there exists an internal function like IFN that operates on vectors, but with additional length and bias parameters, return the internal_fn for that function, otherwise return IFN_LAST.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
Referenced by convert_conditional_op(), and vectorizable_call().
|
extern |
Store all supported else values for the optab referred to by ICODE in ELSE_VALS. The index of the else operand must be specified in ELSE_INDEX.
References CONST0_RTX, CONSTM1_RTX, GET_MODE_CLASS, insn_data, insn_operand_matches(), MASK_LOAD_ELSE_M1, MASK_LOAD_ELSE_UNDEFINED, MASK_LOAD_ELSE_ZERO, operand_data::mode, data::n_operands, and data::operand.
Referenced by internal_gather_scatter_fn_supported_p(), supported_else_val_p(), supports_vec_gather_load_p(), target_supports_len_load_store_p(), target_supports_mask_load_store_p(), and vect_lanes_optab_supported_p().
|
extern |
If IFN implements the conditional form of an unconditional internal function, return that unconditional function, otherwise return IFN_LAST.
References CASE, and FOR_EACH_COND_FN_PAIR.
Referenced by internal_fn_mask_index(), and try_conditional_simplification().
|
extern |
Initialize internal function tables.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References internal_fn_fnspec_array.
Referenced by build_common_builtin_nodes().
|
extern |
Return true if the target supports IFN_CHECK_{RAW,WAR}_PTRS function IFN
for pointers of type TYPE when the accesses have LENGTH bytes and their
common byte alignment is ALIGN.
References direct_internal_fn_optab(), direct_optab_handler(), GEN_INT, immed_wide_int_const(), insn_operand_matches(), and TYPE_MODE.
Referenced by create_ifn_alias_checks().
|
extern |
If FN is an IFN_COND_* or IFN_COND_LEN_* function, return the index of the argument that is used when the condition is false. Return -1 otherwise.
Referenced by add_mask_else_and_len_args(), check_reduction_path(), exist_non_indexing_operands_for_use_p(), internal_gather_scatter_fn_supported_p(), supports_vec_gather_load_p(), target_supports_len_load_store_p(), target_supports_mask_load_store_p(), vect_lanes_optab_supported_p(), vect_transform_reduction(), and vectorizable_load().
|
inline |
References internal_fn_flags_array.
Referenced by call_expr_flags(), gimple_call_flags(), and maybe_push_res_to_seq().
|
inline |
References internal_fn_fnspec_array.
Referenced by gimple_call_fnspec().
|
extern |
If FN takes a vector len argument, return the index of that argument, otherwise return -1.
Referenced by add_mask_else_and_len_args(), can_interpret_as_conditional_op_p(), gimple_fold_partial_load_store_mem_ref(), initialize_ao_ref_for_dse(), try_conditional_simplification(), vectorizable_call(), and vn_reference_lookup_3().
|
extern |
If FN takes a vector mask argument, return the index of that argument, otherwise return -1.
References conditional_internal_fn_code(), and get_unconditional_internal_fn().
Referenced by add_mask_else_and_len_args(), exist_non_indexing_operands_for_use_p(), expand_partial_load_optab_fn(), expand_partial_store_optab_fn(), gimple_fold_partial_load_store_mem_ref(), ifcvt_can_hoist(), vect_analyze_slp(), vect_build_slp_instance(), vect_check_gather_scatter(), vect_get_load_store_mask(), vect_get_smallest_scalar_type(), vect_mark_slp_stmts(), vect_recog_mask_conversion_pattern(), vect_reduction_update_partial_vector_usage(), vectorizable_call(), vectorizable_load(), vectorizable_store(), and vn_reference_lookup_3().
|
inline |
References internal_fn_name_array.
Referenced by combined_fn_name(), dump_generic_node(), dump_gimple_call(), lookup_internal_fn(), expr_hash_elt::print(), print_vn_reference_ops(), and vect_pattern_validate_optab().
|
extern |
If FN takes a value that should be stored to memory, return the index of that argument, otherwise return -1.
Referenced by call_may_clobber_ref_p_1(), exist_non_indexing_operands_for_use_p(), expand_partial_store_optab_fn(), expand_scatter_store_optab_fn(), expand_strided_store_optab_fn(), get_mem_type_for_internal_fn(), gimple_fold_partial_store(), initialize_ao_ref_for_dse(), vect_check_store_rhs(), vect_get_smallest_scalar_type(), vect_get_store_rhs(), vect_recog_mask_conversion_pattern(), and vn_reference_lookup_3().
|
extern |
Return true if IFN is some form of gather load or scatter store.
Referenced by exist_non_indexing_operands_for_use_p(), and vect_check_gather_scatter().
|
extern |
Return true if the target supports gather load or scatter store function IFN. For loads, VECTOR_TYPE is the vector type of the load result, while for stores it is the vector type of the stored data argument. MEMORY_ELEMENT_TYPE is the type of the memory elements being loaded or stored. OFFSET_VECTOR_TYPE is the vector type that holds the offset from the shared base address of each loaded or stored element. SCALE is the amount by which these offsets should be multiplied *after* they have been extended to address width. If the target supports the gather load the supported else values will be added to the vector ELSVAL points to if it is nonzero.
References convert_optab_handler(), direct_internal_fn_optab(), GEN_INT, get_supported_else_vals(), insn_operand_matches(), internal_fn_else_index(), internal_load_fn_p(), tree_int_cst_equal(), TREE_TYPE, TYPE_MODE, TYPE_SIZE, TYPE_UNSIGNED, and TYPE_VECTOR_SUBPARTS().
Referenced by check_load_store_for_partial_vectors(), and vect_gather_scatter_fn_p().
|
extern |
Return the supported bias for IFN which is either IFN_{LEN_,MASK_LEN_,}LOAD
or IFN_{LEN_,MASK_LEN_,}STORE. For now we only support the biases of 0 and
-1 (in case 0 is not an allowable length for {len_,mask_len_}load or
{len_,mask_len_}store). If none of the biases match what the backend
provides, return VECT_PARTIAL_BIAS_UNSUPPORTED.
References convert_optab_handler(), direct_internal_fn_optab(), direct_optab_handler(), GEN_INT, insn_operand_matches(), targetm, and VECT_PARTIAL_BIAS_UNSUPPORTED.
Referenced by vect_verify_loop_lens().
|
extern |
Return true if IFN is some form of load from memory.
Referenced by internal_gather_scatter_fn_supported_p(), vect_get_smallest_scalar_type(), vectorizable_call(), and vectorizable_load().
|
extern |
Return true if IFN is some form of store to memory.
Referenced by vect_build_slp_tree_1(), vect_check_store_rhs(), vect_get_smallest_scalar_type(), vect_get_vector_types_for_stmt(), vect_recog_mask_conversion_pattern(), vectorizable_call(), vectorizable_store(), and vn_reference_lookup_3().
|
extern |
Given an internal_fn IFN that is a widening function, return its corresponding _EVEN and _ODD internal_fns in *EVEN and *ODD.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References gcc_assert, gcc_unreachable, and widening_fn_p().
Referenced by supportable_widening_operation().
|
extern |
Geven an internal_fn IFN that is a widening function, return its corresponding LO and HI internal_fns.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References gcc_assert, gcc_unreachable, and widening_fn_p().
Referenced by supportable_widening_operation().
|
extern |
Return the internal function called NAME, or IFN_LAST if there's no such function.
References i, and internal_fn_name().
|
extern |
Return true if IFN_SET_EDOM is supported.
Referenced by use_internal_fn().
|
extern |
Return true if the else value ELSE_VAL (one of MASK_LOAD_ELSE_ZERO, MASK_LOAD_ELSE_M1, and MASK_LOAD_ELSE_UNDEFINED) is valid fo the optab referred to by ICODE. The index of the else operand must be specified in ELSE_INDEX.
References gcc_unreachable, get_supported_else_vals(), MASK_LOAD_ELSE_M1, MASK_LOAD_ELSE_UNDEFINED, and MASK_LOAD_ELSE_ZERO.
|
inline |
Return true if FN is a direct internal function that can be vectorized by converting the return type and all argument types to vectors of the same number of elements. E.g. we can vectorize an IFN_SQRT on floats as an IFN_SQRT on vectors of N floats.
References direct_internal_fn_array.
Referenced by vect_build_slp_tree_1().
|
extern |
If TYPE is a vector type, return true if IFN is a direct internal function that is supported for that type. If TYPE is a scalar type, return true if IFN is a direct internal function that is supported for the target's preferred vector version of TYPE.
References build_vector_type_for_mode(), direct_internal_fn_supported_p(), is_a(), OPTIMIZE_FOR_SPEED, related_vector_mode(), targetm, TYPE_MODE, VECTOR_MODE_P, and VECTOR_TYPE_P.
Referenced by convert_scalar_cond_reduction(), and ifcvt_can_predicate().
|
extern |
Return true if this CODE describes an internal_fn that returns a vector with elements twice as wide as the element size of the input vectors.
Internal functions. Copyright (C) 2011-2025 Free Software Foundation, Inc. This file is part of GCC. GCC is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. GCC is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see <http://www.gnu.org/licenses/>.
This file specifies a list of internal "functions". These functions
differ from built-in functions in that they have no linkage and cannot
be called directly by the user. They represent operations that are only
synthesised by GCC itself.
Internal functions are used instead of tree codes if the operation
and its operands are more naturally represented as a GIMPLE_CALL
than a GIMPLE_ASSIGN.
Each entry in this file has one of the forms:
DEF_INTERNAL_FN (NAME, FLAGS, FNSPEC)
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_OPTAB_FN (NAME, FLAGS, SELECTOR, SIGNED_OPTAB,
UNSIGNED_OPTAB, TYPE)
DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_FLT_FLOATN_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_INT_EXT_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_SIGNED_COND_FN (NAME, FLAGS, OPTAB, TYPE)
DEF_INTERNAL_WIDENING_OPTAB_FN (NAME, FLAGS, SELECTOR, SOPTAB, UOPTAB,
TYPE)
where NAME is the name of the function, FLAGS is a set of
ECF_* flags and FNSPEC is a string describing functions fnspec.
DEF_INTERNAL_OPTAB_FN defines an internal function that maps to a
direct optab. The function should only be called with a given
set of types if the associated optab is available for the modes
of those types. OPTAB says what optab to use (without the trailing
"_optab") and TYPE categorizes the optab based on its inputs and
outputs. The possible types of optab are:
- mask_load: currently just maskload
- load_lanes: currently just vec_load_lanes
- mask_load_lanes: currently just vec_mask_load_lanes
- mask_len_load_lanes: currently just vec_mask_len_load_lanes
- gather_load: used for {mask_,mask_len_,}gather_load
- strided_load: currently just mask_len_strided_load
- len_load: currently just len_load
- mask_len_load: currently just mask_len_load
- mask_store: currently just maskstore
- store_lanes: currently just vec_store_lanes
- mask_store_lanes: currently just vec_mask_store_lanes
- mask_len_store_lanes: currently just vec_mask_len_store_lanes
- scatter_store: used for {mask_,mask_len_,}scatter_store
- strided_store: currently just mask_len_strided_store
- len_store: currently just len_store
- mask_len_store: currently just mask_len_store
- unary: a normal unary optab, such as vec_reverse_<mode>
- binary: a normal binary optab, such as vec_interleave_lo_<mode>
- ternary: a normal ternary optab, such as fma<mode>4
- unary_convert: a single-input conversion optab, such as
lround<srcmode><dstmode>2.
- cond_binary: a conditional binary optab, such as cond_add<mode>
- cond_unary: a conditional unary optab, such as cond_neg<mode>
- cond_ternary: a conditional ternary optab, such as cond_fma_rev<mode>
- fold_left: for scalar = FN (scalar, vector), keyed off the vector mode
- check_ptrs: used for check_{raw,war}_ptrs
- cond_len_unary: a conditional unary optab, such as cond_len_neg<mode>
- cond_len_binary: a conditional binary optab, such as cond_len_add<mode>
- cond_len_ternary: a conditional ternary optab, such as
cond_len_fma_rev<mode>
DEF_INTERNAL_SIGNED_OPTAB_FN defines an internal function that
maps to one of two optabs, depending on the signedness of an input.
SIGNED_OPTAB and UNSIGNED_OPTAB are the optabs for signed and
unsigned inputs respectively, both without the trailing "_optab".
SELECTOR says which type in the tree_pair determines the signedness.
DEF_INTERNAL_FLT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition,
the function implements the computational part of a built-in math
function BUILT_IN_<NAME>{F,,L}. Unlike some built-in functions,
these internal functions never set errno.
DEF_INTERNAL_INT_FN is like DEF_INTERNAL_OPTAB_FN, but in addition
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
DEF_INTERNAL_INT_EXT_FN is like DEF_INTERNAL_INT_FN, except that it
has expand_##NAME defined in internal-fn.cc to override the
DEF_INTERNAL_INT_FN expansion behavior.
DEF_INTERNAL_WIDENING_OPTAB_FN is a wrapper that defines five internal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN:
- one that describes a widening operation with the same number of elements
in the output and input vectors,
- two that describe a pair of high-low widening operations where the output
vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the top half and
bottom half, these have the suffixes _HI and _LO,
- and two that describe a pair of even-odd widening operations where the
output vectors each have half the number of elements of the input vectors,
corresponding to the result of the widening operation on the even and odd
elements, these have the suffixes _EVEN and _ODD.
These five internal functions will require two optabs each, a SIGNED_OPTAB
and an UNSIGNED_OTPAB.
DEF_INTERNAL_COND_FN is a wrapper that defines 2 internal functions with
DEF_INTERNAL_OPTAB_FN:
- One is COND_* operations that are predicated by mask only. Such operations
make sense for both vectors and scalars.
- The other is COND_LEN_* operations that are predicated by mask and len
both. Such operations only make sense for vectors.
DEF_INTERNAL_SIGNED_COND_FN is like DEF_INTERNAL_COND_FN but defines intenal
functions with DEF_INTERNAL_SIGNED_OPTAB_FN.
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
where STMT is the statement that performs the call. These are generated
automatically for optab functions and call out to a function or macro
called expand_<TYPE>_optab_fn. Extract the last active element from a vector.
Same, but return the first argument if no elements are active.
Unary math functions.
Floating-point to integer conversions. ??? Here we preserve the I/L/LL prefix convention from the corresponding built-in functions, rather than make the internal functions polymorphic in both the argument and the return types. Perhaps an alternative would be to pass a zero of the required return type as a second parameter.
FP rounding.
Binary math functions.
FP scales.
Ternary math functions.
Unary integer ops.
An unduplicable, uncombinable function. Generally used to preserve a CFG property in the face of jump threading, tail merging or other such optimizations. The first argument distinguishes between uses. See internal-fn.h for usage.
A function to represent an artifical initialization to an uninitialized automatic variable.
A function to associate the access size and access mode information with the corresponding reference to an object. It only reads from the 2nd argument.
DIM_SIZE and DIM_POS return the size of a particular compute dimension and the executing thread's position within that dimension. DIM_POS is pure (and not const) so that it isn't thought to clobber memory and can be gcse'd within a single parallel region, but not across FORK/JOIN boundaries. They take a single INTEGER_CST argument. This might be overly conservative.
OpenACC looping abstraction. See internal-fn.h for usage.
OpenACC reduction abstraction. See internal-fn.h for usage.
Openacc tile abstraction. Describes the spans of the element loop. GOACC_TILE (num-loops, loop-no, tile-arg, tile-mask, element-mask).
Set errno to EDOM, if GCC knows how to do that directly for the current target.
Atomic functions. These don't have ECF_NOTHROW because for -fnon-call-exceptions they can throw, otherwise we set gimple_call_nothrow_p on it.
To implement [[fallthrough]]. If the TREE_NOTHROW or GF_CALL_NOTHROW flag is set on the call (normally redundant with ECF_NOTHROW), it marks [[fallthrough]] at the end of C++ loop body.
To implement __builtin_launder.
Divmod function.
For coroutines.
A NOP function with arbitrary arguments and return value.
Temporary vehicle for __builtin_shufflevector.
<=> optimization.
[[assume (cond)]].
For if-conversion of inbranch SIMD clones.
_BitInt support.
Bitwise functions.
References as_internal_fn(), internal_fn_p(), and code_helper::is_fn_code().
Referenced by lookup_evenodd_internal_fn(), lookup_hilo_internal_fn(), supportable_widening_operation(), and vectorizable_conversion().
|
extern |
Referenced by direct_internal_fn(), direct_internal_fn_p(), and vectorizable_internal_fn_p().
|
extern |
Return the ECF_* flags for function FN.
The ECF_* flags of each internal function, indexed by function number.
Referenced by internal_fn_flags().
|
extern |
Return fnspec for function FN.
Fnspec of each internal function, indexed by function number.
Referenced by init_internal_fns(), and internal_fn_fnspec().
|
extern |
Return the name of internal function FN. The name is only meaningful for dumps; it has no linkage.
For lang_hooks.types.type_for_mode.
The names of each internal function, indexed by function number.
Referenced by internal_fn_name().
1.13.2