From patchwork Mon Apr 18 09:26:05 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1069 Return-Path: Delivered-To: unknown Received: from imap.gmail.com (74.125.159.109) by localhost6.localdomain6 with IMAP4-SSL; 08 Jun 2011 14:48:48 -0000 Delivered-To: patches@linaro.org Received: by 10.224.67.148 with SMTP id r20cs28387qai; Mon, 18 Apr 2011 02:26:10 -0700 (PDT) Received: by 10.227.11.146 with SMTP id t18mr4815470wbt.104.1303118768922; Mon, 18 Apr 2011 02:26:08 -0700 (PDT) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx.google.com with ESMTPS id bc6si9698371wbb.52.2011.04.18.02.26.08 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 18 Apr 2011 02:26:08 -0700 (PDT) Received-SPF: neutral (google.com: 74.125.82.50 is neither permitted nor denied by best guess record for domain of richard.sandiford@linaro.org) client-ip=74.125.82.50; Authentication-Results: mx.google.com; spf=neutral (google.com: 74.125.82.50 is neither permitted nor denied by best guess record for domain of richard.sandiford@linaro.org) smtp.mail=richard.sandiford@linaro.org Received: by wwc33 with SMTP id 33so5288556wwc.31 for ; Mon, 18 Apr 2011 02:26:08 -0700 (PDT) Received: by 10.216.141.225 with SMTP id g75mr4706509wej.10.1303118768293; Mon, 18 Apr 2011 02:26:08 -0700 (PDT) Received: from richards-thinkpad (gbibp9ph1--blueice2n1.emea.ibm.com [195.212.29.75]) by mx.google.com with ESMTPS id n2sm2471618wej.22.2011.04.18.02.26.06 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 18 Apr 2011 02:26:07 -0700 (PDT) From: Richard Sandiford To: Diego Novillo Mail-Followup-To: Diego Novillo , gcc-patches@gcc.gnu.org, patches@linaro.org, richard.sandiford@linaro.org Cc: gcc-patches@gcc.gnu.org, patches@linaro.org Subject: Re: RFA: Gimple calls to "internal" functions References: Date: Mon, 18 Apr 2011 10:26:05 +0100 In-Reply-To: (Diego Novillo's message of "Thu, 14 Apr 2011 09:53:18 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) MIME-Version: 1.0 Diego Novillo writes: > On Thu, Apr 14, 2011 at 09:43, Richard Sandiford > wrote: >> +/* This file specifies a list of internal "functions".  These functions >> +   differ from built-in functions in that they have no linkage and cannot >> +   be called directly by the user.  They represent operations that are only >> +   synthesised by GCC itself. >> + >> +   Internal functions are used instead of tree codes if the operation >> +   and its operands are more naturally represented as a GIMPLE_CALL >> +   than a GIMPLE_ASSIGN. >> + >> +   Each entry in this file has the form: >> + >> +     DEF_INTERNAL_FN (NAME, FLAGS) >> + >> +   where NAME is the name of the function and FLAGS is a set of >> +   ECF_* flags.  */ > > Could you add a short description specifying how these internal > functions are later expanded into RTL? Sure, added. > So, this patch adds no internal functions at all? Yeah. The first ones are defined by: http://gcc.gnu.org/ml/gcc-patches/2011-04/msg00881.html > We need to make sure that internal functions are properly streamed > in/out for LTO. So, we should have some test cases for it. Gah, I'd even thought about streaming for the first (subcode-based) implementation. Streaming just worked then though. It just doesn't work now. As far as tests go, I'd like to run the main vectoriser tests with -flto, which would trap all uses of the first batch of functions. How does this updated patch look? Bootstrapped & regression-tested on x86_64-linux-gnu, and tested on arm-linux-gnueabi. Richard gcc/ * Makefile.in (INTERNAL_FN_DEF, INTERNAL_FN_H): Define. (GIMPLE_H): Include $(INTERNAL_FN_H). (OBJS-common): Add internal-fn.o. (internal-fn.o): New rule. * internal-fn.def: New file. * internal-fn.h: Likewise. * internal-fn.c: Likewise. * gimple.h: Include internal-fn.h. (GF_CALL_INTERNAL): New gf_mask. (gimple_statement_call): Put fntype into a union with a new internal_fn field. (gimple_build_call_internal): Declare. (gimple_build_call_internal_vec): Likewise. (gimple_call_same_target_p): Likewise. (gimple_call_internal_p): New function. (gimple_call_internal_fn): Likewise. (gimple_call_fntype): Assert that the call is not internal. (gimple_call_fn): Assert that the function is nonnull. (gimple_call_set_internal_fn): New function. (gimple_call_set_fn): Clear GF_CALL_INTERNAL field. (gimple_call_fndecl): Return null for calls to internal functions. (gimple_call_return_type): Use the type of the lhs for internal calls. * gimple.c (gimple_build_call_1): Use gimple_call_set_fn rather than gimple_set_op. (gimple_build_call_internal_1): New function. (gimple_build_call_internal): Likewise. (gimple_build_call_internal_vec): Likewise. (walk_gimple_op): Skip the GIMPLE_CALL function for calls to internal functions. (gimple_call_same_target_p): New function. (gimple_call_flags): Handle calls to internal functions. (gimple_call_arg_flags): Likewise. (gimple_call_return_flags): Likewise. (gimple_has_side_effects): Likewise. (gimple_call_copy_skip_args): Likewise. * cfgexpand.c (expand_gimple_call_internal): New function. (expand_call_stmt): Use it. * expr.c (expand_expr_real_1): Don't call promote_function_mode for internal functions. * gimple-fold.c (gimple_fold_stmt_to_constant_1): Don't fold calls to internal functions. * gimple-low.c (gimple_check_call_args): Handle calls to internal functions. * gimple-pretty-print.c (dump_gimple_call): Likewise. * ipa-prop.c (ipa_analyze_call_uses): Likewise. * tree-cfg.c (verify_gimple_call): Likewise. (do_warn_unused_result): Likewise. * tree-eh.c (same_handler_p): Use gimple_call_same_target_p. * tree-ssa-ccp.c (ccp_fold_stmt): Handle calls to internal functions. * tree-ssa-dom.c (hashable_expr): Use the gimple statement to record the target of a call. (initialize_hash_element): Update accordingly. (hashable_expr_equal_p): Use gimple_call_same_target_p. (iterative_hash_hashable_expr): Handle calls to internal functions. (print_expr_hash_elt): Likewise. * tree-ssa-pre.c (eliminate): Likewise. * tree-ssa-sccvn.c (copy_reference_ops_from_call): Likewise. * tree-ssa-structalias.c (get_fi_for_callee): Likewise. (find_func_aliases): Likewise. * value-prof.c (gimple_ic_transform): Likewise. (gimple_indirect_call_to_profile): Likewise. * lto-streamer-in.c (input_gimple_stmt): Likewise. * lto-streamer-out.c (output_gimple_stmt): Likewise. Index: gcc/Makefile.in =================================================================== --- gcc/Makefile.in 2011-04-18 10:18:49.000000000 +0100 +++ gcc/Makefile.in 2011-04-18 10:18:54.000000000 +0100 @@ -893,6 +893,8 @@ RTL_ERROR_H = $(RTL_H) $(DIAGNOSTIC_CORE READ_MD_H = $(OBSTACK_H) $(HASHTAB_H) read-md.h PARAMS_H = params.h params.def BUILTINS_DEF = builtins.def sync-builtins.def omp-builtins.def +INTERNAL_FN_DEF = internal-fn.def +INTERNAL_FN_H = internal-fn.h $(INTERNAL_FN_DEF) TREE_H = tree.h all-tree.def tree.def c-family/c-common.def \ $(lang_tree_files) $(MACHMODE_H) tree-check.h $(BUILTINS_DEF) \ $(INPUT_H) statistics.h $(VEC_H) treestruct.def $(HASHTAB_H) \ @@ -901,8 +903,8 @@ TREE_H = tree.h all-tree.def tree.def c- REGSET_H = regset.h $(BITMAP_H) hard-reg-set.h BASIC_BLOCK_H = basic-block.h $(PREDICT_H) $(VEC_H) $(FUNCTION_H) cfghooks.h GIMPLE_H = gimple.h gimple.def gsstruct.def pointer-set.h $(VEC_H) \ - $(GGC_H) $(BASIC_BLOCK_H) $(TARGET_H) tree-ssa-operands.h \ - tree-ssa-alias.h vecir.h + vecir.h $(GGC_H) $(BASIC_BLOCK_H) $(TARGET_H) tree-ssa-operands.h \ + tree-ssa-alias.h $(INTERNAL_FN_H) GCOV_IO_H = gcov-io.h gcov-iov.h auto-host.h COVERAGE_H = coverage.h $(GCOV_IO_H) DEMANGLE_H = $(srcdir)/../include/demangle.h @@ -1274,6 +1276,7 @@ OBJS-common = \ init-regs.o \ input.o \ integrate.o \ + internal-fn.o \ intl.o \ ira.o \ ira-build.o \ @@ -2759,6 +2762,8 @@ tree-object-size.o: tree-object-size.c $ $(TM_H) $(TREE_H) $(DIAGNOSTIC_CORE_H) $(DIAGNOSTIC_H) $(TREE_FLOW_H) \ $(TREE_PASS_H) tree-ssa-propagate.h tree-pretty-print.h \ gimple-pretty-print.h +internal-fn.o : internal-fn.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \ + $(INTERNAL_FN_H) $(TREE_H) $(EXPR_H) $(OPTABS_H) gimple.o : gimple.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TREE_H) \ $(GGC_H) $(GIMPLE_H) $(DIAGNOSTIC_CORE_H) $(DIAGNOSTIC_H) gt-gimple.h \ $(TREE_FLOW_H) value-prof.h $(FLAGS_H) $(DEMANGLE_H) \ Index: gcc/internal-fn.def =================================================================== --- /dev/null 2011-03-23 08:42:11.268792848 +0000 +++ gcc/internal-fn.def 2011-04-18 10:20:35.000000000 +0100 @@ -0,0 +1,39 @@ +/* Internal functions. + Copyright (C) 2011 Free Software Foundation, Inc. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +/* This file specifies a list of internal "functions". These functions + differ from built-in functions in that they have no linkage and cannot + be called directly by the user. They represent operations that are only + synthesised by GCC itself. + + Internal functions are used instead of tree codes if the operation + and its operands are more naturally represented as a GIMPLE_CALL + than a GIMPLE_ASSIGN. + + Each entry in this file has the form: + + DEF_INTERNAL_FN (NAME, FLAGS) + + where NAME is the name of the function and FLAGS is a set of + ECF_* flags. Each entry must have a corresponding expander + of the form: + + void expand_NAME (tree lhs, tree *args) + + where LHS and ARGS are as for expand_internal_call. */ Index: gcc/internal-fn.h =================================================================== --- /dev/null 2011-03-23 08:42:11.268792848 +0000 +++ gcc/internal-fn.h 2011-04-18 10:18:54.000000000 +0100 @@ -0,0 +1,51 @@ +/* Internal functions. + Copyright (C) 2011 Free Software Foundation, Inc. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#ifndef GCC_INTERNAL_FN_H +#define GCC_INTERNAL_FN_H + +enum internal_fn { +#define DEF_INTERNAL_FN(CODE, FLAGS) IFN_##CODE, +#include "internal-fn.def" +#undef DEF_INTERNAL_FN + IFN_LAST +}; + +/* Return the name of internal function FN. The name is only meaningful + for dumps; it has no linkage. */ + +static inline const char * +internal_fn_name (enum internal_fn fn) +{ + extern const char *const internal_fn_name_array[]; + return internal_fn_name_array[(int) fn]; +} + +/* Return the ECF_* flags for function FN. */ + +static inline int +internal_fn_flags (enum internal_fn fn) +{ + extern const int internal_fn_flags_array[]; + return internal_fn_flags_array[(int) fn]; +} + +extern void expand_internal_call (enum internal_fn, tree, tree *); + +#endif Index: gcc/internal-fn.c =================================================================== --- /dev/null 2011-03-23 08:42:11.268792848 +0000 +++ gcc/internal-fn.c 2011-04-18 10:18:54.000000000 +0100 @@ -0,0 +1,64 @@ +/* Internal functions. + Copyright (C) 2011 Free Software Foundation, Inc. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "system.h" +#include "coretypes.h" +#include "internal-fn.h" +#include "tree.h" +#include "expr.h" +#include "optabs.h" + +/* The names of each internal function, indexed by function number. */ +const char *const internal_fn_name_array[] = { +#define DEF_INTERNAL_FN(CODE, FLAGS) #CODE, +#include "internal-fn.def" +#undef DEF_INTERNAL_FN + "" +}; + +/* The ECF_* flags of each internal function, indexed by function number. */ +const int internal_fn_flags_array[] = { +#define DEF_INTERNAL_FN(CODE, FLAGS) FLAGS, +#include "internal-fn.def" +#undef DEF_INTERNAL_FN + 0 +}; + +/* Routines to expand each internal function, indexed by function number. + Each routine has the prototype: + + expand_ (tree lhs, tree *args) + + where LHS and ARGS are as for expand_internal_call. */ +static void (*const internal_fn_expanders[]) (tree, tree *) = { +#define DEF_INTERNAL_FN(CODE, FLAGS) expand_##CODE, +#include "internal-fn.def" +#undef DEF_INTERNAL_FN + 0 +}; + +/* Expand a call to internal function FN. ARGS is an array of the + function's arguments and LHS is where the result should be stored. */ + +void +expand_internal_call (enum internal_fn fn, tree lhs, tree *args) +{ + internal_fn_expanders[(int) fn] (lhs, args); +} Index: gcc/gimple.h =================================================================== --- gcc/gimple.h 2011-04-18 10:18:49.000000000 +0100 +++ gcc/gimple.h 2011-04-18 10:18:54.000000000 +0100 @@ -30,6 +30,7 @@ #define GCC_GIMPLE_H #include "basic-block.h" #include "tree-ssa-operands.h" #include "tree-ssa-alias.h" +#include "internal-fn.h" struct gimple_seq_node_d; typedef struct gimple_seq_node_d *gimple_seq_node; @@ -102,6 +103,7 @@ enum gf_mask { GF_CALL_TAILCALL = 1 << 3, GF_CALL_VA_ARG_PACK = 1 << 4, GF_CALL_NOTHROW = 1 << 5, + GF_CALL_INTERNAL = 1 << 6, GF_OMP_PARALLEL_COMBINED = 1 << 0, /* True on an GIMPLE_OMP_RETURN statement if the return does not require @@ -406,7 +408,10 @@ struct GTY(()) gimple_statement_call struct pt_solution call_clobbered; /* [ WORD 13 ] */ - tree fntype; + union GTY ((desc ("%1.membase.opbase.gsbase.subcode & GF_CALL_INTERNAL"))) { + tree GTY ((tag ("0"))) fntype; + enum internal_fn GTY ((tag ("GF_CALL_INTERNAL"))) internal_fn; + } u; /* [ WORD 14 ] Operand vector. NOTE! This must always be the last field @@ -820,6 +825,8 @@ #define gimple_build_debug_bind(var,val, gimple gimple_build_call_vec (tree, VEC(tree, heap) *); gimple gimple_build_call (tree, unsigned, ...); +gimple gimple_build_call_internal (enum internal_fn, unsigned, ...); +gimple gimple_build_call_internal_vec (enum internal_fn, VEC(tree, heap) *); gimple gimple_build_call_from_tree (tree); gimple gimplify_assign (tree, tree, gimple_seq *); gimple gimple_build_cond (enum tree_code, tree, tree, tree, tree); @@ -864,6 +871,7 @@ gimple_seq gimple_seq_alloc (void); void gimple_seq_free (gimple_seq); void gimple_seq_add_seq (gimple_seq *, gimple_seq); gimple_seq gimple_seq_copy (gimple_seq); +bool gimple_call_same_target_p (const_gimple, const_gimple); int gimple_call_flags (const_gimple); int gimple_call_return_flags (const_gimple); int gimple_call_arg_flags (const_gimple, unsigned); @@ -2004,13 +2012,35 @@ gimple_call_set_lhs (gimple gs, tree lhs } +/* Return true if call GS calls an internal-only function, as enumerated + by internal_fn. */ + +static inline bool +gimple_call_internal_p (const_gimple gs) +{ + GIMPLE_CHECK (gs, GIMPLE_CALL); + return (gs->gsbase.subcode & GF_CALL_INTERNAL) != 0; +} + + +/* Return the target of internal call GS. */ + +static inline enum internal_fn +gimple_call_internal_fn (const_gimple gs) +{ + gcc_gimple_checking_assert (gimple_call_internal_p (gs)); + return gs->gimple_call.u.internal_fn; +} + + /* Return the function type of the function called by GS. */ static inline tree gimple_call_fntype (const_gimple gs) { GIMPLE_CHECK (gs, GIMPLE_CALL); - return gs->gimple_call.fntype; + gcc_gimple_checking_assert (!gimple_call_internal_p (gs)); + return gs->gimple_call.u.fntype; } /* Set the type of the function called by GS to FNTYPE. */ @@ -2019,7 +2049,8 @@ gimple_call_fntype (const_gimple gs) gimple_call_set_fntype (gimple gs, tree fntype) { GIMPLE_CHECK (gs, GIMPLE_CALL); - gs->gimple_call.fntype = fntype; + gcc_gimple_checking_assert (!gimple_call_internal_p (gs)); + gs->gimple_call.u.fntype = fntype; } @@ -2029,8 +2060,12 @@ gimple_call_set_fntype (gimple gs, tree static inline tree gimple_call_fn (const_gimple gs) { + tree op; + GIMPLE_CHECK (gs, GIMPLE_CALL); - return gimple_op (gs, 1); + op = gimple_op (gs, 1); + gcc_gimple_checking_assert (op != NULL_TREE); + return op; } /* Return a pointer to the tree node representing the function called by call @@ -2044,12 +2079,25 @@ gimple_call_fn_ptr (const_gimple gs) } +/* Set internal function FN to be the function called by call statement GS. */ + +static inline void +gimple_call_set_internal_fn (gimple gs, enum internal_fn fn) +{ + GIMPLE_CHECK (gs, GIMPLE_CALL); + gs->gsbase.subcode |= GF_CALL_INTERNAL; + gimple_set_op (gs, 1, NULL_TREE); + gs->gimple_call.u.internal_fn = fn; +} + + /* Set FN to be the function called by call statement GS. */ static inline void gimple_call_set_fn (gimple gs, tree fn) { GIMPLE_CHECK (gs, GIMPLE_CALL); + gs->gsbase.subcode &= ~GF_CALL_INTERNAL; gimple_set_op (gs, 1, fn); } @@ -2071,7 +2119,12 @@ gimple_call_set_fndecl (gimple gs, tree static inline tree gimple_call_fndecl (const_gimple gs) { - tree addr = gimple_call_fn (gs); + tree addr; + + if (gimple_call_internal_p (gs)) + return NULL_TREE; + + addr = gimple_call_fn (gs); if (TREE_CODE (addr) == ADDR_EXPR) { tree fndecl = TREE_OPERAND (addr, 0); @@ -2094,7 +2147,12 @@ gimple_call_fndecl (const_gimple gs) static inline tree gimple_call_return_type (const_gimple gs) { - tree type = gimple_call_fntype (gs); + tree type; + + if (gimple_call_internal_p (gs)) + return TREE_TYPE (gimple_call_lhs (gs)); + + type = gimple_call_fntype (gs); /* The type returned by a function is the type of its function type. */ Index: gcc/gimple.c =================================================================== --- gcc/gimple.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/gimple.c 2011-04-18 10:18:54.000000000 +0100 @@ -230,7 +230,7 @@ gimple_build_call_1 (tree fn, unsigned n gimple s = gimple_build_with_ops (GIMPLE_CALL, ERROR_MARK, nargs + 3); if (TREE_CODE (fn) == FUNCTION_DECL) fn = build_fold_addr_expr (fn); - gimple_set_op (s, 1, fn); + gimple_call_set_fn (s, fn); gimple_call_set_fntype (s, TREE_TYPE (TREE_TYPE (fn))); gimple_call_reset_alias_info (s); return s; @@ -277,6 +277,58 @@ gimple_build_call (tree fn, unsigned nar } +/* Helper for gimple_build_call_internal and gimple_build_call_internal_vec. + Build the basic components of a GIMPLE_CALL statement to internal + function FN with NARGS arguments. */ + +static inline gimple +gimple_build_call_internal_1 (enum internal_fn fn, unsigned nargs) +{ + gimple s = gimple_build_with_ops (GIMPLE_CALL, ERROR_MARK, nargs + 3); + gimple_call_set_internal_fn (s, fn); + gimple_call_reset_alias_info (s); + return s; +} + + +/* Build a GIMPLE_CALL statement to internal function FN. NARGS is + the number of arguments. The ... are the arguments. */ + +gimple +gimple_build_call_internal (enum internal_fn fn, unsigned nargs, ...) +{ + va_list ap; + gimple call; + unsigned i; + + call = gimple_build_call_internal_1 (fn, nargs); + va_start (ap, nargs); + for (i = 0; i < nargs; i++) + gimple_call_set_arg (call, i, va_arg (ap, tree)); + va_end (ap); + + return call; +} + + +/* Build a GIMPLE_CALL statement to internal function FN with the arguments + specified in vector ARGS. */ + +gimple +gimple_build_call_internal_vec (enum internal_fn fn, VEC(tree, heap) *args) +{ + unsigned i, nargs; + gimple call; + + nargs = VEC_length (tree, args); + call = gimple_build_call_internal_1 (fn, nargs); + for (i = 0; i < nargs; i++) + gimple_call_set_arg (call, i, VEC_index (tree, args, i)); + + return call; +} + + /* Build a GIMPLE_CALL statement from CALL_EXPR T. Note that T is assumed to be in GIMPLE form already. Minimal checking is done of this fact. */ @@ -1399,9 +1451,12 @@ walk_gimple_op (gimple stmt, walk_tree_f if (ret) return ret; - ret = walk_tree (gimple_call_fn_ptr (stmt), callback_op, wi, pset); - if (ret) - return ret; + if (!gimple_call_internal_p (stmt)) + { + ret = walk_tree (gimple_call_fn_ptr (stmt), callback_op, wi, pset); + if (ret) + return ret; + } for (i = 0; i < gimple_call_num_args (stmt); i++) { @@ -1773,6 +1828,19 @@ gimple_has_body_p (tree fndecl) return (gimple_body (fndecl) || (fn && fn->cfg)); } +/* Return true if calls C1 and C2 are known to go to the same function. */ + +bool +gimple_call_same_target_p (const_gimple c1, const_gimple c2) +{ + if (gimple_call_internal_p (c1)) + return (gimple_call_internal_p (c2) + && gimple_call_internal_fn (c1) == gimple_call_internal_fn (c2)); + else + return (!gimple_call_internal_p (c2) + && operand_equal_p (gimple_call_fn (c1), gimple_call_fn (c2), 0)); +} + /* Detect flags from a GIMPLE_CALL. This is just like call_expr_flags, but for gimple tuples. */ @@ -1784,6 +1852,8 @@ gimple_call_flags (const_gimple stmt) if (decl) flags = flags_from_decl_or_type (decl); + else if (gimple_call_internal_p (stmt)) + flags = internal_fn_flags (gimple_call_internal_fn (stmt)); else flags = flags_from_decl_or_type (gimple_call_fntype (stmt)); @@ -1798,8 +1868,14 @@ gimple_call_flags (const_gimple stmt) int gimple_call_arg_flags (const_gimple stmt, unsigned arg) { - tree type = gimple_call_fntype (stmt); - tree attr = lookup_attribute ("fn spec", TYPE_ATTRIBUTES (type)); + tree type; + tree attr; + + if (gimple_call_internal_p (stmt)) + return 0; + + type = gimple_call_fntype (stmt); + attr = lookup_attribute ("fn spec", TYPE_ATTRIBUTES (type)); if (!attr) return 0; @@ -1839,6 +1915,9 @@ gimple_call_return_flags (const_gimple s tree type; tree attr = NULL_TREE; + if (gimple_call_internal_p (stmt)) + return 0; + if (gimple_call_flags (stmt) & ECF_MALLOC) return ERF_NOALIAS; @@ -2287,7 +2366,8 @@ gimple_has_side_effects (const_gimple s) return true; } - if (TREE_SIDE_EFFECTS (gimple_call_fn (s))) + if (!gimple_call_internal_p (s) + && TREE_SIDE_EFFECTS (gimple_call_fn (s))) return true; for (i = 0; i < nargs; i++) @@ -2332,8 +2412,9 @@ gimple_rhs_has_side_effects (const_gimpl /* We cannot use gimple_has_volatile_ops here, because we must ignore a volatile LHS. */ - if (TREE_SIDE_EFFECTS (gimple_call_fn (s)) - || TREE_THIS_VOLATILE (gimple_call_fn (s))) + if (!gimple_call_internal_p (s) + && (TREE_SIDE_EFFECTS (gimple_call_fn (s)) + || TREE_THIS_VOLATILE (gimple_call_fn (s)))) { gcc_assert (gimple_has_volatile_ops (s)); return true; @@ -3089,7 +3170,6 @@ canonicalize_cond_expr_cond (tree t) gimple_call_copy_skip_args (gimple stmt, bitmap args_to_skip) { int i; - tree fn = gimple_call_fn (stmt); int nargs = gimple_call_num_args (stmt); VEC(tree, heap) *vargs = VEC_alloc (tree, heap, nargs); gimple new_stmt; @@ -3098,7 +3178,11 @@ gimple_call_copy_skip_args (gimple stmt, if (!bitmap_bit_p (args_to_skip, i)) VEC_quick_push (tree, vargs, gimple_call_arg (stmt, i)); - new_stmt = gimple_build_call_vec (fn, vargs); + if (gimple_call_internal_p (stmt)) + new_stmt = gimple_build_call_internal_vec (gimple_call_internal_fn (stmt), + vargs); + else + new_stmt = gimple_build_call_vec (gimple_call_fn (stmt), vargs); VEC_free (tree, heap, vargs); if (gimple_call_lhs (stmt)) gimple_call_set_lhs (new_stmt, gimple_call_lhs (stmt)); Index: gcc/cfgexpand.c =================================================================== --- gcc/cfgexpand.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/cfgexpand.c 2011-04-18 10:18:54.000000000 +0100 @@ -1831,16 +1831,39 @@ expand_gimple_cond (basic_block bb, gimp return new_bb; } +/* A subroutine of expand_call_stmt. Expand GIMPLE_CALL statement STMT, + which is known to be to an internal function. */ + +static void +expand_gimple_call_internal (gimple stmt) +{ + tree lhs; + tree *args; + size_t i; + + lhs = gimple_call_lhs (stmt); + args = XALLOCAVEC (tree, gimple_call_num_args (stmt)); + for (i = 0; i < gimple_call_num_args (stmt); i++) + args[i] = gimple_call_arg (stmt, i); + expand_internal_call (gimple_call_internal_fn (stmt), lhs, args); +} + /* A subroutine of expand_gimple_stmt_1, expanding one GIMPLE_CALL statement STMT. */ static void expand_call_stmt (gimple stmt) { - tree exp, decl, lhs = gimple_call_lhs (stmt); + tree exp, decl, lhs; bool builtin_p; size_t i; + if (gimple_call_internal_p (stmt)) + { + expand_gimple_call_internal (stmt); + return; + } + exp = build_vl_exp (CALL_EXPR, gimple_call_num_args (stmt) + 3); CALL_EXPR_FN (exp) = gimple_call_fn (stmt); @@ -1885,6 +1908,7 @@ expand_call_stmt (gimple stmt) SET_EXPR_LOCATION (exp, gimple_location (stmt)); TREE_BLOCK (exp) = gimple_block (stmt); + lhs = gimple_call_lhs (stmt); if (lhs) expand_assignment (lhs, exp, false); else Index: gcc/expr.c =================================================================== --- gcc/expr.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/expr.c 2011-04-18 10:18:54.000000000 +0100 @@ -8521,10 +8521,15 @@ expand_expr_real_1 (tree exp, rtx target enum machine_mode pmode; /* Get the signedness to be used for this variable. Ensure we get - the same mode we got when the variable was declared. */ + the same mode we got when the variable was declared. + + Note that calls to internal functions do not result in a + call instruction, so promote_function_mode is not meaningful + in that case. */ if (code == SSA_NAME && (g = SSA_NAME_DEF_STMT (ssa_name)) - && gimple_code (g) == GIMPLE_CALL) + && gimple_code (g) == GIMPLE_CALL + && !gimple_call_internal_p (g)) pmode = promote_function_mode (type, mode, &unsignedp, gimple_call_fntype (g), 2); Index: gcc/gimple-fold.c =================================================================== --- gcc/gimple-fold.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/gimple-fold.c 2011-04-18 10:18:54.000000000 +0100 @@ -2856,7 +2856,13 @@ gimple_fold_stmt_to_constant_1 (gimple s case GIMPLE_CALL: { - tree fn = (*valueize) (gimple_call_fn (stmt)); + tree fn; + + if (gimple_call_internal_p (stmt)) + /* No folding yet for these functions. */ + return NULL_TREE; + + fn = (*valueize) (gimple_call_fn (stmt)); if (TREE_CODE (fn) == ADDR_EXPR && TREE_CODE (TREE_OPERAND (fn, 0)) == FUNCTION_DECL && DECL_BUILT_IN (TREE_OPERAND (fn, 0))) Index: gcc/gimple-low.c =================================================================== --- gcc/gimple-low.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/gimple-low.c 2011-04-18 10:18:54.000000000 +0100 @@ -224,6 +224,8 @@ gimple_check_call_args (gimple stmt, tre /* Get argument types for verification. */ if (fndecl) parms = TYPE_ARG_TYPES (TREE_TYPE (fndecl)); + else if (gimple_call_internal_p (stmt)) + parms = NULL_TREE; else parms = TYPE_ARG_TYPES (gimple_call_fntype (stmt)); Index: gcc/gimple-pretty-print.c =================================================================== --- gcc/gimple-pretty-print.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/gimple-pretty-print.c 2011-04-18 10:18:54.000000000 +0100 @@ -616,8 +616,12 @@ dump_gimple_call (pretty_printer *buffer if (flags & TDF_RAW) { - dump_gimple_fmt (buffer, spc, flags, "%G <%T, %T", - gs, gimple_call_fn (gs), lhs); + if (gimple_call_internal_p (gs)) + dump_gimple_fmt (buffer, spc, flags, "%G <%s, %T", gs, + internal_fn_name (gimple_call_internal_fn (gs)), lhs); + else + dump_gimple_fmt (buffer, spc, flags, "%G <%T, %T", + gs, gimple_call_fn (gs), lhs); if (gimple_call_num_args (gs) > 0) { pp_string (buffer, ", "); @@ -637,7 +641,10 @@ dump_gimple_call (pretty_printer *buffer pp_space (buffer); } - print_call_name (buffer, gimple_call_fn (gs), flags); + if (gimple_call_internal_p (gs)) + pp_string (buffer, internal_fn_name (gimple_call_internal_fn (gs))); + else + print_call_name (buffer, gimple_call_fn (gs), flags); pp_string (buffer, " ("); dump_gimple_call_args (buffer, gs, flags); pp_character (buffer, ')'); Index: gcc/ipa-prop.c =================================================================== --- gcc/ipa-prop.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/ipa-prop.c 2011-04-18 10:18:54.000000000 +0100 @@ -1416,8 +1416,12 @@ ipa_analyze_call_uses (struct cgraph_nod struct ipa_node_params *info, struct param_analysis_info *parms_info, gimple call) { - tree target = gimple_call_fn (call); + tree target; + + if (gimple_call_internal_p (call)) + return; + target = gimple_call_fn (call); if (TREE_CODE (target) == SSA_NAME) ipa_analyze_indirect_call_uses (node, info, parms_info, call, target); else if (TREE_CODE (target) == OBJ_TYPE_REF) Index: gcc/tree-cfg.c =================================================================== --- gcc/tree-cfg.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-cfg.c 2011-04-18 10:18:54.000000000 +0100 @@ -3042,23 +3042,43 @@ valid_fixed_convert_types_p (tree type1, static bool verify_gimple_call (gimple stmt) { - tree fn = gimple_call_fn (stmt); + tree fn; tree fntype, fndecl; unsigned i; - if (!is_gimple_call_addr (fn)) + if (!gimple_call_internal_p (stmt)) { - error ("invalid function in gimple call"); - debug_generic_stmt (fn); - return true; - } - - if (!POINTER_TYPE_P (TREE_TYPE (fn)) - || (TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != FUNCTION_TYPE - && TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != METHOD_TYPE)) - { - error ("non-function in gimple call"); - return true; + fn = gimple_call_fn (stmt); + if (!is_gimple_call_addr (fn)) + { + error ("invalid function in gimple call"); + debug_generic_stmt (fn); + return true; + } + if (!POINTER_TYPE_P (TREE_TYPE (fn)) + || (TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != FUNCTION_TYPE + && TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != METHOD_TYPE)) + { + error ("non-function in gimple call"); + return true; + } + fntype = gimple_call_fntype (stmt); + if (gimple_call_lhs (stmt) + && !useless_type_conversion_p (TREE_TYPE (gimple_call_lhs (stmt)), + TREE_TYPE (fntype)) + /* ??? At least C++ misses conversions at assignments from + void * call results. + ??? Java is completely off. Especially with functions + returning java.lang.Object. + For now simply allow arbitrary pointer type conversions. */ + && !(POINTER_TYPE_P (TREE_TYPE (gimple_call_lhs (stmt))) + && POINTER_TYPE_P (TREE_TYPE (fntype)))) + { + error ("invalid conversion in gimple call"); + debug_generic_stmt (TREE_TYPE (gimple_call_lhs (stmt))); + debug_generic_stmt (TREE_TYPE (fntype)); + return true; + } } fndecl = gimple_call_fndecl (stmt); @@ -3086,24 +3106,6 @@ verify_gimple_call (gimple stmt) return true; } - fntype = gimple_call_fntype (stmt); - if (gimple_call_lhs (stmt) - && !useless_type_conversion_p (TREE_TYPE (gimple_call_lhs (stmt)), - TREE_TYPE (fntype)) - /* ??? At least C++ misses conversions at assignments from - void * call results. - ??? Java is completely off. Especially with functions - returning java.lang.Object. - For now simply allow arbitrary pointer type conversions. */ - && !(POINTER_TYPE_P (TREE_TYPE (gimple_call_lhs (stmt))) - && POINTER_TYPE_P (TREE_TYPE (fntype)))) - { - error ("invalid conversion in gimple call"); - debug_generic_stmt (TREE_TYPE (gimple_call_lhs (stmt))); - debug_generic_stmt (TREE_TYPE (fntype)); - return true; - } - if (gimple_call_chain (stmt) && !is_gimple_val (gimple_call_chain (stmt))) { @@ -3121,7 +3123,7 @@ verify_gimple_call (gimple stmt) error ("static chain in indirect gimple call"); return true; } - fn = TREE_OPERAND (fn, 0); + fn = TREE_OPERAND (gimple_call_fn (stmt), 0); if (!DECL_STATIC_CHAIN (fn)) { @@ -7436,6 +7438,8 @@ do_warn_unused_result (gimple_seq seq) case GIMPLE_CALL: if (gimple_call_lhs (g)) break; + if (gimple_call_internal_p (g)) + break; /* This is a naked call, as opposed to a GIMPLE_CALL with an LHS. All calls whose value is ignored should be Index: gcc/tree-eh.c =================================================================== --- gcc/tree-eh.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-eh.c 2011-04-18 10:18:54.000000000 +0100 @@ -2743,7 +2743,7 @@ same_handler_p (gimple_seq oneh, gimple_ || gimple_call_lhs (twos) || gimple_call_chain (ones) || gimple_call_chain (twos) - || !operand_equal_p (gimple_call_fn (ones), gimple_call_fn (twos), 0) + || !gimple_call_same_target_p (ones, twos) || gimple_call_num_args (ones) != gimple_call_num_args (twos)) return false; Index: gcc/tree-ssa-ccp.c =================================================================== --- gcc/tree-ssa-ccp.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-ssa-ccp.c 2011-04-18 10:18:54.000000000 +0100 @@ -1723,6 +1723,11 @@ ccp_fold_stmt (gimple_stmt_iterator *gsi return true; } + /* Internal calls provide no argument types, so the extra laxity + for normal calls does not apply. */ + if (gimple_call_internal_p (stmt)) + return false; + /* Propagate into the call arguments. Compared to replace_uses_in this can use the argument slot types for type verification instead of the current argument type. We also can safely Index: gcc/tree-ssa-dom.c =================================================================== --- gcc/tree-ssa-dom.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-ssa-dom.c 2011-04-18 10:18:54.000000000 +0100 @@ -64,7 +64,7 @@ struct hashable_expr struct { enum tree_code op; tree opnd; } unary; struct { enum tree_code op; tree opnd0, opnd1; } binary; struct { enum tree_code op; tree opnd0, opnd1, opnd2; } ternary; - struct { tree fn; bool pure; size_t nargs; tree *args; } call; + struct { gimple fn_from; bool pure; size_t nargs; tree *args; } call; } ops; }; @@ -258,7 +258,7 @@ initialize_hash_element (gimple stmt, tr expr->type = TREE_TYPE (gimple_call_lhs (stmt)); expr->kind = EXPR_CALL; - expr->ops.call.fn = gimple_call_fn (stmt); + expr->ops.call.fn_from = stmt; if (gimple_call_flags (stmt) & (ECF_CONST | ECF_PURE)) expr->ops.call.pure = true; @@ -422,8 +422,8 @@ hashable_expr_equal_p (const struct hash /* If the calls are to different functions, then they clearly cannot be equal. */ - if (! operand_equal_p (expr0->ops.call.fn, - expr1->ops.call.fn, 0)) + if (!gimple_call_same_target_p (expr0->ops.call.fn_from, + expr1->ops.call.fn_from)) return false; if (! expr0->ops.call.pure) @@ -503,9 +503,15 @@ iterative_hash_hashable_expr (const stru { size_t i; enum tree_code code = CALL_EXPR; + gimple fn_from; val = iterative_hash_object (code, val); - val = iterative_hash_expr (expr->ops.call.fn, val); + fn_from = expr->ops.call.fn_from; + if (gimple_call_internal_p (fn_from)) + val = iterative_hash_hashval_t + ((hashval_t) gimple_call_internal_fn (fn_from), val); + else + val = iterative_hash_expr (gimple_call_fn (fn_from), val); for (i = 0; i < expr->ops.call.nargs; i++) val = iterative_hash_expr (expr->ops.call.args[i], val); } @@ -565,8 +571,14 @@ print_expr_hash_elt (FILE * stream, cons { size_t i; size_t nargs = element->expr.ops.call.nargs; + gimple fn_from; - print_generic_expr (stream, element->expr.ops.call.fn, 0); + fn_from = element->expr.ops.call.fn_from; + if (gimple_call_internal_p (fn_from)) + fputs (internal_fn_name (gimple_call_internal_fn (fn_from)), + stream); + else + print_generic_expr (stream, gimple_call_fn (fn_from), 0); fprintf (stream, " ("); for (i = 0; i < nargs; i++) { Index: gcc/tree-ssa-pre.c =================================================================== --- gcc/tree-ssa-pre.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-ssa-pre.c 2011-04-18 10:18:54.000000000 +0100 @@ -4381,6 +4381,7 @@ eliminate (void) /* Visit indirect calls and turn them into direct calls if possible. */ if (is_gimple_call (stmt) + && !gimple_call_internal_p (stmt) && TREE_CODE (gimple_call_fn (stmt)) == SSA_NAME) { tree orig_fn = gimple_call_fn (stmt); Index: gcc/tree-ssa-sccvn.c =================================================================== --- gcc/tree-ssa-sccvn.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-ssa-sccvn.c 2011-04-18 10:18:54.000000000 +0100 @@ -911,7 +911,10 @@ copy_reference_ops_from_call (gimple cal memset (&temp, 0, sizeof (temp)); temp.type = gimple_call_return_type (call); temp.opcode = CALL_EXPR; - temp.op0 = gimple_call_fn (call); + if (gimple_call_internal_p (call)) + temp.op0 = NULL_TREE; + else + temp.op0 = gimple_call_fn (call); temp.op1 = gimple_call_chain (call); temp.off = -1; VEC_safe_push (vn_reference_op_s, heap, *result, &temp); Index: gcc/tree-ssa-structalias.c =================================================================== --- gcc/tree-ssa-structalias.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/tree-ssa-structalias.c 2011-04-18 10:18:54.000000000 +0100 @@ -4026,6 +4026,8 @@ get_fi_for_callee (gimple call) { tree decl; + gcc_assert (!gimple_call_internal_p (call)); + /* If we can directly resolve the function being called, do so. Otherwise, it must be some sort of indirect expression that we should still be able to handle. */ @@ -4319,6 +4321,7 @@ find_func_aliases (gimple origt) /* Fallthru to general call handling. */; } if (!in_ipa_mode + || gimple_call_internal_p (t) || (fndecl && (!(fi = lookup_vi_for_tree (fndecl)) || !fi->is_fn_info))) Index: gcc/value-prof.c =================================================================== --- gcc/value-prof.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/value-prof.c 2011-04-18 10:18:54.000000000 +0100 @@ -1258,6 +1258,9 @@ gimple_ic_transform (gimple stmt) if (gimple_call_fndecl (stmt) != NULL_TREE) return false; + if (gimple_call_internal_p (stmt)) + return false; + histogram = gimple_histogram_value_of_type (cfun, stmt, HIST_TYPE_INDIR_CALL); if (!histogram) return false; @@ -1649,6 +1652,7 @@ gimple_indirect_call_to_profile (gimple tree callee; if (gimple_code (stmt) != GIMPLE_CALL + || gimple_call_internal_p (stmt) || gimple_call_fndecl (stmt) != NULL_TREE) return; Index: gcc/lto-streamer-in.c =================================================================== --- gcc/lto-streamer-in.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/lto-streamer-in.c 2011-04-18 10:18:54.000000000 +0100 @@ -1063,7 +1063,13 @@ input_gimple_stmt (struct lto_input_bloc } } if (is_gimple_call (stmt)) - gimple_call_set_fntype (stmt, lto_input_tree (ib, data_in)); + { + if (gimple_call_internal_p (stmt)) + gimple_call_set_internal_fn + (stmt, (enum internal_fn) lto_input_sleb128 (ib)); + else + gimple_call_set_fntype (stmt, lto_input_tree (ib, data_in)); + } break; case GIMPLE_NOP: Index: gcc/lto-streamer-out.c =================================================================== --- gcc/lto-streamer-out.c 2011-04-18 10:18:49.000000000 +0100 +++ gcc/lto-streamer-out.c 2011-04-18 10:18:54.000000000 +0100 @@ -1760,7 +1760,12 @@ output_gimple_stmt (struct output_block lto_output_tree_ref (ob, op); } if (is_gimple_call (stmt)) - lto_output_tree_ref (ob, gimple_call_fntype (stmt)); + { + if (gimple_call_internal_p (stmt)) + output_sleb128 (ob, (int) gimple_call_internal_fn (stmt)); + else + lto_output_tree_ref (ob, gimple_call_fntype (stmt)); + } break; case GIMPLE_NOP: