From patchwork Mon Oct 23 17:26:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 116809 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp4891425qgn; Mon, 23 Oct 2017 10:27:20 -0700 (PDT) X-Received: by 10.99.110.6 with SMTP id j6mr12611670pgc.246.1508779640541; Mon, 23 Oct 2017 10:27:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1508779640; cv=none; d=google.com; s=arc-20160816; b=e7GXXeRu+Fmcq9VDq6B/+5BTiKjywOh8beRRrOR6xR9YgWaS31xT2C0CZjc6gX22r9 Tt3meiGbkAnIj2374qdeX5wQypJ1PpsnxfaDUmFgYfFGmfIWDIY0nBqAJA/grAW5J+a/ 8Fsvx9mEhJ3Ocn6NWgxu5g6M1Tdg42t7DyRurREmu7ZG+PZf2CjnDX/t3ZWOSMBgc4qQ ftkmxXLWPNpROSFuxflK/F7t2QBtyYxu0xY6t03G5E180Ed653uMywSVbQ1gRugkhyV7 9y9dlbRVWONg7bBWS8bq6FiB9PcJYpp1xJXPv4drtLbor+bmNdMWzC5bgEhPqShH0+bV TJmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:mail-followup-to:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=nmz08X1ZP2Me18uuHOYwkVqp9H6x9XWp3z+d7mJ/KYs=; b=FVipTd8zLLcM8GvndPW8M39IrmJx0R60K0gZUGIfMWuwKb3haDjjf0trHTvI15JjmQ qtFIDxM+KCS0TUs+IpkC2jCeQNZxNdYRCpQgdUxzKSBAkmqUAREmxJas4iiT6HLS86Tf YMKLoSaWaan5IW7eLTqR+YHWNsvv/1LWinKLFS5kHiRJK+03rgQw9FzHRz3n80eRVd8x SEB9Q/yQeP2CM3eakLcYIGPWObDHyxdAAyXIgA21MuZ9MRdwIxLHsxA+kdz4b0iP5apj 2sFoEhhhvpPKg1/8oP3SNNiunoGsVLaL1bFC1dc9EhxuiS6+ST04OCBoA5K3uIBLaJRv g7Gg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=iYoREcvC; spf=pass (google.com: domain of gcc-patches-return-464840-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464840-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id g8si4171919plt.780.2017.10.23.10.27.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 10:27:20 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-464840-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=iYoREcvC; spf=pass (google.com: domain of gcc-patches-return-464840-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-464840-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=default; b=vXt25iKoDcaHHyM5alGKjxtBx00DG 7dJUlfHzHAaOzuddb+pWzj9FLZJmGOhL0OBtpFiY+FUOtzuE4T5OogAAKVkGfMEe igXdpz/wzRk/jdsxiZAOIL8LXChIITPg86/tt+bsqzokCj8CIa+fXH6prRmSkOSX 5R9Nx8RkaVcTtQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; s=default; bh=AbO5cXsrsOsCnF2AKvEZvX+AqjQ=; b=iYo REcvCndLHDuHaZ1f79kBalcst1o8IR/CrX79ebF3051g2EpAZXrL0FQ9RvBoguJn A7z13ir3zSy/lUfbIQzcx7wIvFr/JeYjECwlLbXqsVgNK2/b6Yjib7DLufFT99DM 2MXLAKBGsZaVaJ8iK+32vWRBD8ZPABGtBc+zmrM0= Received: (qmail 9302 invoked by alias); 23 Oct 2017 17:26:42 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 9109 invoked by uid 89); 23 Oct 2017 17:26:42 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.7 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=064 X-HELO: mail-wm0-f42.google.com Received: from mail-wm0-f42.google.com (HELO mail-wm0-f42.google.com) (74.125.82.42) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 23 Oct 2017 17:26:39 +0000 Received: by mail-wm0-f42.google.com with SMTP id t69so10980772wmt.2 for ; Mon, 23 Oct 2017 10:26:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=nmz08X1ZP2Me18uuHOYwkVqp9H6x9XWp3z+d7mJ/KYs=; b=hF+5+NaKwDtfa5gt835j+2zK5WwIkCWNt/uHBZbmvAiuPH8orLU1nsbBeMEDr2SZ50 uCMaAJgMBExJSK7Bd8VmR8fBPOlhHNA3T7OYmeugHAfiW81xYJ77N8+W23WKtlD6NdiB 5iEbGbH+bAUxSnso4X/USpubp9+xXJcA27dyXy5CWufnbV4Rbr/WPWtWQ4MEosfTv7cd jR1mO8jsSqbEDYe5Doc54/ldLSApGf6ZRqjDJQSeh8r1iKJEuQniodOCG+5UtPxZZhU8 Th9TwBofjI7ulX1lzqr6QKh1FlJNo2wzWlDelaVe2+kK09sM8rOnCvKNCZV2gVKMoYlw pjCw== X-Gm-Message-State: AMCzsaWJS3RF7+sx7s0sHaGBDbljhZPtwdQdWf2B2bkDCeUUxCp5YrKg iThnwypkkXaFfc3Up4t5HZuVk9j3eck= X-Google-Smtp-Source: ABhQp+Rbog+V2v9rIySCVcJHdfI2TgCJ3x10yBZYNJf4b7uU9/PX1LtnQBxPiDF8YPmx8rOSgEQ+/Q== X-Received: by 10.28.168.203 with SMTP id r194mr5959426wme.2.1508779597512; Mon, 23 Oct 2017 10:26:37 -0700 (PDT) Received: from localhost ([2.26.27.199]) by smtp.gmail.com with ESMTPSA id f19sm1621589wmf.4.2017.10.23.10.26.36 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Oct 2017 10:26:36 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: [064/nnn] poly_int: SLP max_units References: <871sltvm7r.fsf@linaro.org> Date: Mon, 23 Oct 2017 18:26:36 +0100 In-Reply-To: <871sltvm7r.fsf@linaro.org> (Richard Sandiford's message of "Mon, 23 Oct 2017 17:54:32 +0100") Message-ID: <877evlkc6r.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 This match makes tree-vect-slp.c track the maximum number of vector units as a poly_uint64 rather than an unsigned int. 2017-10-23 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vect-slp.c (vect_record_max_nunits, vect_build_slp_tree_1) (vect_build_slp_tree_2, vect_build_slp_tree): Change max_nunits from an unsigned int * to a poly_uint64_pod *. (calculate_unrolling_factor): New function. (vect_analyze_slp_instance): Use it. Track polynomial max_nunits. Index: gcc/tree-vect-slp.c =================================================================== --- gcc/tree-vect-slp.c 2017-10-23 17:22:26.573499378 +0100 +++ gcc/tree-vect-slp.c 2017-10-23 17:22:27.793744215 +0100 @@ -489,7 +489,7 @@ vect_get_and_check_slp_defs (vec_info *v static bool vect_record_max_nunits (vec_info *vinfo, gimple *stmt, unsigned int group_size, - tree vectype, unsigned int *max_nunits) + tree vectype, poly_uint64 *max_nunits) { if (!vectype) { @@ -506,8 +506,11 @@ vect_record_max_nunits (vec_info *vinfo, /* If populating the vector type requires unrolling then fail before adjusting *max_nunits for basic-block vectorization. */ + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); + unsigned HOST_WIDE_INT const_nunits; if (is_a (vinfo) - && TYPE_VECTOR_SUBPARTS (vectype) > group_size) + && (!nunits.is_constant (&const_nunits) + || const_nunits > group_size)) { dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "Build SLP failed: unrolling required " @@ -517,9 +520,7 @@ vect_record_max_nunits (vec_info *vinfo, } /* In case of multiple types we need to detect the smallest type. */ - if (*max_nunits < TYPE_VECTOR_SUBPARTS (vectype)) - *max_nunits = TYPE_VECTOR_SUBPARTS (vectype); - + vect_update_max_nunits (max_nunits, vectype); return true; } @@ -540,7 +541,7 @@ vect_record_max_nunits (vec_info *vinfo, static bool vect_build_slp_tree_1 (vec_info *vinfo, unsigned char *swap, vec stmts, unsigned int group_size, - unsigned nops, unsigned int *max_nunits, + unsigned nops, poly_uint64 *max_nunits, bool *matches, bool *two_operators) { unsigned int i; @@ -966,16 +967,15 @@ bst_traits::equal (value_type existing, static slp_tree vect_build_slp_tree_2 (vec_info *vinfo, vec stmts, unsigned int group_size, - unsigned int *max_nunits, + poly_uint64 *max_nunits, vec *loads, bool *matches, unsigned *npermutes, unsigned *tree_size, unsigned max_tree_size); static slp_tree vect_build_slp_tree (vec_info *vinfo, - vec stmts, unsigned int group_size, - unsigned int *max_nunits, - vec *loads, + vec stmts, unsigned int group_size, + poly_uint64 *max_nunits, vec *loads, bool *matches, unsigned *npermutes, unsigned *tree_size, unsigned max_tree_size) { @@ -1007,12 +1007,13 @@ vect_build_slp_tree (vec_info *vinfo, static slp_tree vect_build_slp_tree_2 (vec_info *vinfo, vec stmts, unsigned int group_size, - unsigned int *max_nunits, + poly_uint64 *max_nunits, vec *loads, bool *matches, unsigned *npermutes, unsigned *tree_size, unsigned max_tree_size) { - unsigned nops, i, this_tree_size = 0, this_max_nunits = *max_nunits; + unsigned nops, i, this_tree_size = 0; + poly_uint64 this_max_nunits = *max_nunits; gimple *stmt; slp_tree node; @@ -1951,6 +1952,15 @@ vect_split_slp_store_group (gimple *firs return group2; } +/* Calculate the unrolling factor for an SLP instance with GROUP_SIZE + statements and a vector of NUNITS elements. */ + +static poly_uint64 +calculate_unrolling_factor (poly_uint64 nunits, unsigned int group_size) +{ + return exact_div (common_multiple (nunits, group_size), group_size); +} + /* Analyze an SLP instance starting from a group of grouped stores. Call vect_build_slp_tree to build a tree of packed stmts if possible. Return FALSE if it's impossible to SLP any stmt in the loop. */ @@ -1962,11 +1972,9 @@ vect_analyze_slp_instance (vec_info *vin slp_instance new_instance; slp_tree node; unsigned int group_size = GROUP_SIZE (vinfo_for_stmt (stmt)); - unsigned int nunits; tree vectype, scalar_type = NULL_TREE; gimple *next; unsigned int i; - unsigned int max_nunits = 0; vec loads; struct data_reference *dr = STMT_VINFO_DATA_REF (vinfo_for_stmt (stmt)); vec scalar_stmts; @@ -2005,7 +2013,7 @@ vect_analyze_slp_instance (vec_info *vin return false; } - nunits = TYPE_VECTOR_SUBPARTS (vectype); + poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (vectype); /* Create a node (a root of the SLP tree) for the packed grouped stores. */ scalar_stmts.create (group_size); @@ -2043,32 +2051,35 @@ vect_analyze_slp_instance (vec_info *vin bool *matches = XALLOCAVEC (bool, group_size); unsigned npermutes = 0; bst_fail = new hash_set , bst_traits> (); + poly_uint64 max_nunits = nunits; node = vect_build_slp_tree (vinfo, scalar_stmts, group_size, - &max_nunits, &loads, matches, &npermutes, + &max_nunits, &loads, matches, &npermutes, NULL, max_tree_size); delete bst_fail; if (node != NULL) { /* Calculate the unrolling factor based on the smallest type. */ poly_uint64 unrolling_factor - = least_common_multiple (max_nunits, group_size) / group_size; + = calculate_unrolling_factor (max_nunits, group_size); if (may_ne (unrolling_factor, 1U) && is_a (vinfo)) { - - if (max_nunits > group_size) - { - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "Build SLP failed: store group " - "size not a multiple of the vector size " - "in basic block SLP\n"); - vect_free_slp_tree (node); - loads.release (); - return false; - } + unsigned HOST_WIDE_INT const_max_nunits; + if (!max_nunits.is_constant (&const_max_nunits) + || const_max_nunits > group_size) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "Build SLP failed: store group " + "size not a multiple of the vector size " + "in basic block SLP\n"); + vect_free_slp_tree (node); + loads.release (); + return false; + } /* Fatal mismatch. */ - matches[group_size/max_nunits * max_nunits] = false; + matches[group_size / const_max_nunits * const_max_nunits] = false; vect_free_slp_tree (node); loads.release (); } @@ -2187,20 +2198,22 @@ vect_analyze_slp_instance (vec_info *vin /* For basic block SLP, try to break the group up into multiples of the vector size. */ + unsigned HOST_WIDE_INT const_nunits; if (is_a (vinfo) && GROUP_FIRST_ELEMENT (vinfo_for_stmt (stmt)) - && STMT_VINFO_GROUPED_ACCESS (vinfo_for_stmt (stmt))) + && STMT_VINFO_GROUPED_ACCESS (vinfo_for_stmt (stmt)) + && nunits.is_constant (&const_nunits)) { /* We consider breaking the group only on VF boundaries from the existing start. */ for (i = 0; i < group_size; i++) if (!matches[i]) break; - if (i >= nunits && i < group_size) + if (i >= const_nunits && i < group_size) { /* Split into two groups at the first vector boundary before i. */ - gcc_assert ((nunits & (nunits - 1)) == 0); - unsigned group1_size = i & ~(nunits - 1); + gcc_assert ((const_nunits & (const_nunits - 1)) == 0); + unsigned group1_size = i & ~(const_nunits - 1); gimple *rest = vect_split_slp_store_group (stmt, group1_size); bool res = vect_analyze_slp_instance (vinfo, stmt, max_tree_size); @@ -2208,9 +2221,9 @@ vect_analyze_slp_instance (vec_info *vin skip the rest of that vector. */ if (group1_size < i) { - i = group1_size + nunits; + i = group1_size + const_nunits; if (i < group_size) - rest = vect_split_slp_store_group (rest, nunits); + rest = vect_split_slp_store_group (rest, const_nunits); } if (i < group_size) res |= vect_analyze_slp_instance (vinfo, rest, max_tree_size);