From patchwork Tue Oct 1 11:14:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel X-Patchwork-Id: 174832 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp8392408ill; Tue, 1 Oct 2019 04:15:08 -0700 (PDT) X-Google-Smtp-Source: APXvYqzia3X1BTlsC4H5onK8raPtqzZ9vODmj6xTDNjUEZ45jD1a3+6Bk64jklPaM5l1F6Cobkne X-Received: by 2002:a50:a939:: with SMTP id l54mr24958282edc.214.1569928508489; Tue, 01 Oct 2019 04:15:08 -0700 (PDT) Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id y22si8833683ejw.144.2019.10.01.04.15.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Oct 2019 04:15:08 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509961-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=NCvWwCOz; dkim=neutral (body hash did not verify) header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=Dv3Wvn7H; dkim=neutral (body hash did not verify) header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=Dv3Wvn7H; arc=fail (body hash mismatch); spf=pass (google.com: domain of gcc-patches-return-509961-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509961-patch=linaro.org@gcc.gnu.org" DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:references:in-reply-to :content-type:mime-version; q=dns; s=default; b=htQP69qNTXKYd3ZP BFUUnk4xytRCFrKJDiTu+ItG4KLn6iejgnCLlDMpldOb8L2bSmfYPvoiBcx+PQ+G kRddIEIzlEQ28GY3if1iHc1na+aaa7kRbXMHW+QeZeS9y/J/r+Db8G2Wjuiovf04 jJmRjXEYLJgZ25vmovTTbAGKwmE= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:references:in-reply-to :content-type:mime-version; s=default; bh=OX4gLWL/5lTUJIRbMjHBBI I//2A=; b=NCvWwCOz6I6WbcpI3aoQCGF8/P8c9elZrw2MFST6h91pk/fQ9pzfRT e54SbddZ3P+NaBgdYTCP3ZvaiB/UWW4J2EECXLWIQPGusy+rItXrKzzGhLSFBBrr q7qc9JchLB82o8msnJ1ze/1CKYN7rpzyr4UzNWXqUIqffugqX7LSA= Received: (qmail 65848 invoked by alias); 1 Oct 2019 11:14:52 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 65840 invoked by uid 89); 1 Oct 2019 11:14:51 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-23.2 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: EUR02-VE1-obe.outbound.protection.outlook.com Received: from mail-eopbgr20049.outbound.protection.outlook.com (HELO EUR02-VE1-obe.outbound.protection.outlook.com) (40.107.2.49) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 01 Oct 2019 11:14:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2b8CivNj2px8KRbdnY0mUXUzd8tgtw/r8wxpwIezCVU=; b=Dv3Wvn7H74J24g5E9+Dw9y9inzpCA2+aNcD8avU6wrK22uzO0csjHQ6RO+pqM3ztCJyRYVy8nYSyLwKKGOaXv+fMhSdbhR0LqTmc4zAPeaHG2jITM/UyDAEBIXubTjGyn7M44DI9rYCePiYAWFJfnA6FCUOtNW0B/wpJryJxxO0= Received: from VI1PR08CA0127.eurprd08.prod.outlook.com (2603:10a6:800:d4::29) by AM0PR08MB3234.eurprd08.prod.outlook.com (2603:10a6:208:5e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 11:14:44 +0000 Received: from DB5EUR03FT005.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::203) by VI1PR08CA0127.outlook.office365.com (2603:10a6:800:d4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.16 via Frontend Transport; Tue, 1 Oct 2019 11:14:44 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT005.mail.protection.outlook.com (10.152.20.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2305.15 via Frontend Transport; Tue, 1 Oct 2019 11:14:43 +0000 Received: ("Tessian outbound 0cf06bf5c60e:v33"); Tue, 01 Oct 2019 11:14:42 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 609305aac5a693c5 X-CR-MTA-TID: 64aa7808 Received: from 032d8e138cd5.1 (ip-172-16-0-2.eu-west-1.compute.internal [104.47.9.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6FA787EC-CEE9-470B-9398-853B1C575B7C.1; Tue, 01 Oct 2019 11:14:37 +0000 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2059.outbound.protection.outlook.com [104.47.9.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 032d8e138cd5.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Tue, 01 Oct 2019 11:14:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dV3xNQTevyCtdSKqZMJS7bVfGUl0mShNqrzyQINelbo5jvDpmxQspQfhUTUJtnsc1OC3Hyo5BYgLmZ5lGTNI2Ah+Ukk2Qv4lYkhzQofO8BhONIPoLvvcVwlXnIRmMX1pyyaNRsyuW2Dau1DUNfc+A+vm/9NhmA3YtPXqZXE4L36FZ7QpOuF3wzPsKvu5W2NNmk0WxQJaTB09DTbqNVoMvihr7T5X3Sh7+hBQMMVIfVlpt/D7OkgUDLJAg+WzTxR59HxGj2iCPudMTdyOygaU+8liZZVgWfeSQzLe7fZdUHi3+vIQCzhqwZU5tkKSWoCHbJ9Nb0i/jg2ifWZzNvmynw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2b8CivNj2px8KRbdnY0mUXUzd8tgtw/r8wxpwIezCVU=; b=DUbRDA8gz9MC4ozeV3uuIZCWvYffVV8QZN8Pe8XsDB1QE/E65eV1dCrNnQ/N8lLtGyCWw3Dt5Lcx0XjthjZZ/GmJNiM8/vSOSLbmjyNrAF6FSqypc/51YE6i5ZEhjJpcnB04+cqwwrw0UqRa4P4SAbnAtAWSzV1omVi2NQTQeNsryMQxQZpESAsviLVkKn2mbkoz8hoKNqLiAQCdIWQASJ5Mh+pbw4TZgYKQyeb+UQ6TjqzfSbFmY/rMeh81A3Lhc+u7lTTpOcHxVKAle2bW3344laBGNem55mzWeBW9yYLbtHy0V9J1TfHRZUIGYVGWYsoyAupkEnE1X9DroFIHyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2b8CivNj2px8KRbdnY0mUXUzd8tgtw/r8wxpwIezCVU=; b=Dv3Wvn7H74J24g5E9+Dw9y9inzpCA2+aNcD8avU6wrK22uzO0csjHQ6RO+pqM3ztCJyRYVy8nYSyLwKKGOaXv+fMhSdbhR0LqTmc4zAPeaHG2jITM/UyDAEBIXubTjGyn7M44DI9rYCePiYAWFJfnA6FCUOtNW0B/wpJryJxxO0= Received: from DB6PR0801MB2054.eurprd08.prod.outlook.com (10.168.86.135) by DB6PR0801MB1750.eurprd08.prod.outlook.com (10.169.226.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 11:14:36 +0000 Received: from DB6PR0801MB2054.eurprd08.prod.outlook.com ([fe80::b92b:84c:8ff2:f917]) by DB6PR0801MB2054.eurprd08.prod.outlook.com ([fe80::b92b:84c:8ff2:f917%6]) with mapi id 15.20.2305.017; Tue, 1 Oct 2019 11:14:36 +0000 From: Joel Hutton To: GCC Patches CC: nd , "rguenther@suse.de" Subject: [RFC][SLP] SLP vectorization: vectorize vector constructors Date: Tue, 1 Oct 2019 11:14:36 +0000 Message-ID: References: <910ff75f-1f42-e8d0-80a0-432c181aa9c8@arm.com> In-Reply-To: <910ff75f-1f42-e8d0-80a0-432c181aa9c8@arm.com> user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Joel.Hutton@arm.com; x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(366004)(346002)(39860400002)(136003)(376002)(396003)(189003)(199004)(66446008)(36756003)(64756008)(25786009)(66476007)(66946007)(66616009)(66556008)(6916009)(5660300002)(478600001)(7736002)(76176011)(58126008)(305945005)(31686004)(54906003)(316002)(52116002)(99286004)(3846002)(81166006)(6436002)(4326008)(53546011)(6506007)(31696002)(14454004)(102836004)(81156014)(8936002)(256004)(5024004)(386003)(26005)(71190400001)(71200400001)(486006)(2906002)(86362001)(8676002)(6116002)(6486002)(99936001)(6512007)(65806001)(65956001)(11346002)(446003)(186003)(66066001)(476003)(2616005); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0801MB1750; H:DB6PR0801MB2054.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: pQdl+XeNh1/0Sh+ohCRDHKUuQ2vBfLFNvkgWJ/sfclbKqUokkOj0f6sBRlELL0094PsVBFFaq01X0wcENz4/EkjG0uFxx57ZJucB27wqeu9ovve+rA4B0bX8ynOhJRGfmnl3KWhcGeo8NwTbhZT8ZBsjcYS/vHfU5s4NamSApsD28zCnosQB1rOj8y60s3B5xIdD/vO1zxAsrukcFBwFliqczGcuVGfA0cPJJyhKnUUBpVS8iN77EEWOOSgvSUXzOaymPI72PV/V+Ca+WoPYxe+vvFxUeWX7JhF3Y+6o1kj2tU4974lIbJhWYUP9FhGnVq5TvGwg69QBSklULtDPqO3lFNh7I0HkgcFyIOIll0h4LTdD94xQ8qRBpi3BSOuGXonVa3ATDMMOc/6YN4bEbP/fnfizFLuUGRs9kSE5+v4= x-ms-exchange-transport-forked: True MIME-Version: 1.0 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Joel.Hutton@arm.com; Return-Path: Joel.Hutton@arm.com X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT005.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 3d5d0060-0061-40ca-92fa-08d746608b02 X-IsSubscribed: yes On 01/10/2019 12:07, Joel wrote: > > SLP vectorization: vectorize vector constructors > > > Currently SLP vectorization can build SLP trees staring from > reductions or from group stores. This patch adds a third starting > point: vector constructors. > > > For the following test case (compiled with -O3 -fno-vect-cost-model): > > > char g_d[1024], g_s1[1024], g_s2[1024]; void test_loop(void) { char /d > = g_d, /s1 = g_s1, *s2 = g_s2; > > > for ( int y = 0; y < 128; y++ ) > { > for ( int x = 0; x < 16; x++ ) > d[x] = s1[x] + s2[x]; > d += 16; > } > > } > > > before patch: test_loop: .LFB0: .cfi_startproc adrp x0, g_s1 adrp x2, > g_s2 add x3, x0, :lo12:g_s1 add x4, x2, :lo12:g_s2 ldrb w7, [x2, > #:lo12:g_s2] ldrb w1, [x0, #:lo12:g_s1] adrp x0, g_d ldrb w6, [x4, 1] > add x0, x0, :lo12:g_d ldrb w5, [x3, 1] add w1, w1, w7 fmov s0, w1 ldrb > w7, [x4, 2] add w5, w5, w6 ldrb w1, [x3, 2] ldrb w6, [x4, 3] add x2, > x0, 2048 ins v0.b[1], w5 add w1, w1, w7 ldrb w7, [x3, 3] ldrb w5, [x4, > 4] add w7, w7, w6 ldrb w6, [x3, 4] ins v0.b[2], w1 ldrb w8, [x4, 5] > add w6, w6, w5 ldrb w5, [x3, 5] ldrb w9, [x4, 6] add w5, w5, w8 ldrb > w1, [x3, 6] ins v0.b[3], w7 ldrb w8, [x4, 7] add w1, w1, w9 ldrb w11, > [x3, 7] ldrb w7, [x4, 8] add w11, w11, w8 ldrb w10, [x3, 8] ins > v0.b[4], w6 ldrb w8, [x4, 9] add w10, w10, w7 ldrb w9, [x3, 9] ldrb > w7, [x4, 10] add w9, w9, w8 ldrb w8, [x3, 10] ins v0.b[5], w5 ldrb w6, > [x4, 11] add w8, w8, w7 ldrb w7, [x3, 11] ldrb w5, [x4, 12] add w7, > w7, w6 ldrb w6, [x3, 12] ins v0.b[6], w1 ldrb w12, [x4, 13] add w6, > w6, w5 ldrb w5, [x3, 13] ldrb w1, [x3, 14] add w5, w5, w12 ldrb w13, > [x4, 14] ins v0.b[7], w11 ldrb w12, [x4, 15] add w4, w1, w13 ldrb w1, > [x3, 15] add w1, w1, w12 ins v0.b[8], w10 ins v0.b[9], w9 ins > v0.b[10], w8 ins v0.b[11], w7 ins v0.b[12], w6 ins v0.b[13], w5 ins > v0.b[14], w4 ins v0.b[15], w1 .p2align 3,,7 .L2: str q0, [x0], 16 cmp > x2, x0 bne .L2 ret .cfi_endproc .LFE0: > > > After patch: > > > test_loop: .LFB0: .cfi_startproc adrp x3, g_s1 adrp x2, g_s2 add x3, > x3, :lo12:g_s1 add x2, x2, :lo12:g_s2 adrp x0, g_d add x0, x0, > :lo12:g_d add x1, x0, 2048 ldr q1, [x2] ldr q0, [x3] add v0.16b, > v0.16b, v1.16b .p2align 3,,7 .L2: str q0, [x0], 16 cmp x0, x1 bne .L2 > ret .cfi_endproc .LFE0: > > > > > bootstrapped and tested on aarch64-none-linux-gnu > Patch attached: >From 7b9e6d02017ffe6f7ab17cbdd48da41ccc5f6db0 Mon Sep 17 00:00:00 2001 From: Joel Hutton Date: Fri, 27 Sep 2019 10:26:00 +0100 Subject: [PATCH] SLP vectorization: vectorize vector constructors --- gcc/tree-vect-slp.c | 98 ++++++++++++++++++++++++++++++++++++------- gcc/tree-vect-stmts.c | 20 +++++++++ 2 files changed, 103 insertions(+), 15 deletions(-) diff --git a/gcc/tree-vect-slp.c b/gcc/tree-vect-slp.c index 9b86b67734ad3e3506e9cee6a532b68decf24ae6..4c715ebe34dbdb8072e15dc9053f53a1949a070d 100644 --- a/gcc/tree-vect-slp.c +++ b/gcc/tree-vect-slp.c @@ -1923,6 +1923,8 @@ vect_analyze_slp_instance (vec_info *vinfo, struct data_reference *dr = STMT_VINFO_DATA_REF (stmt_info); vec scalar_stmts; + bool constructor = false; + if (STMT_VINFO_GROUPED_ACCESS (stmt_info)) { scalar_type = TREE_TYPE (DR_REF (dr)); @@ -1935,6 +1937,17 @@ vect_analyze_slp_instance (vec_info *vinfo, vectype = STMT_VINFO_VECTYPE (stmt_info); group_size = REDUC_GROUP_SIZE (stmt_info); } + else if (is_gimple_assign (stmt_info->stmt) + && TREE_CODE (TREE_TYPE (gimple_assign_lhs (stmt_info->stmt))) + == VECTOR_TYPE + && gimple_assign_rhs_code (stmt_info->stmt) == CONSTRUCTOR) + { + vectype = TREE_TYPE (gimple_assign_rhs1 (stmt_info->stmt)); + group_size = CONSTRUCTOR_NELTS (gimple_assign_rhs1 (stmt_info->stmt)); + constructor = true; + if (TREE_CODE (vectype) != VECTOR_TYPE) + vectype = NULL; + } else { gcc_assert (is_a (vinfo)); @@ -1981,6 +1994,32 @@ vect_analyze_slp_instance (vec_info *vinfo, STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info)) = STMT_VINFO_REDUC_DEF (vect_orig_stmt (scalar_stmts.last ())); } + else if (constructor) + { + tree rhs = gimple_assign_rhs1 (stmt_info->stmt); + tree val; + FOR_EACH_CONSTRUCTOR_VALUE (CONSTRUCTOR_ELTS (rhs), i, val) + { + tree prev_val; + int j; + FOR_EACH_CONSTRUCTOR_VALUE (CONSTRUCTOR_ELTS (rhs), j, prev_val) + { + if (val == prev_val && i!=j) + return false; + } + if (TREE_CODE (val) == SSA_NAME) + { + gimple* def = SSA_NAME_DEF_STMT (val); + stmt_vec_info def_info = vinfo->lookup_stmt (def); + /* Value is defined in another basic block. */ + if (!def_info) + return false; + scalar_stmts.safe_push (def_info); + } + else + return false; + } + } else { /* Collect reduction statements. */ @@ -2227,6 +2266,50 @@ vect_analyze_slp (vec_info *vinfo, unsigned max_tree_size) max_tree_size); } + /* Find SLP sequences starting from a vector constructor. */ + gimple_stmt_iterator gsi; + if (is_a (vinfo)) + { + bb_vec_info bb_vinfo = dyn_cast (vinfo); + + for (gsi = bb_vinfo->region_begin; + gsi_stmt (gsi) != gsi_stmt (bb_vinfo->region_end); gsi_next (&gsi)) + { + gimple *stmt = gsi_stmt (gsi); + bool vectorizable = true; + if ( is_gimple_assign (stmt) + && gimple_assign_rhs_code (stmt) == CONSTRUCTOR + && TREE_CODE (TREE_TYPE (gimple_assign_lhs (stmt))) == VECTOR_TYPE) + { + tree rhs = gimple_assign_rhs1 (stmt); + tree val; + int i; + /* Check that the constructor elements are unique. */ + FOR_EACH_CONSTRUCTOR_VALUE (CONSTRUCTOR_ELTS (rhs), i, val) + { + tree prev_val; + int j; + FOR_EACH_CONSTRUCTOR_VALUE (CONSTRUCTOR_ELTS (rhs), j, prev_val) + { + if (val == prev_val && i!=j) + { + vectorizable = false; + break; + } + } + } + + if (vectorizable) + { + first_element = bb_vinfo->lookup_stmt (stmt); + vect_analyze_slp_instance (vinfo, first_element, + max_tree_size); + } + } + } + } + + return opt_result::success (); } @@ -2908,21 +2991,6 @@ vect_slp_analyze_bb_1 (gimple_stmt_iterator region_begin, return NULL; } - /* If there are no grouped stores in the region there is no need - to continue with pattern recog as vect_analyze_slp will fail - anyway. */ - if (bb_vinfo->grouped_stores.is_empty ()) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "not vectorized: no grouped stores in " - "basic block.\n"); - - delete bb_vinfo; - return NULL; - } - - /* While the rest of the analysis below depends on it in some way. */ fatal = false; vect_pattern_recog (bb_vinfo); diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index b1e97f85d96b352e9e52bbb7c265dda6e1d0f3ad..9e1a762f922d069da4e5bcc3cb05df2bc3d1acff 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -6237,6 +6237,26 @@ vectorizable_operation (stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, gimple_assign_set_lhs (new_stmt, new_temp); new_stmt_info = vect_finish_stmt_generation (stmt_info, new_stmt, gsi); + + /* For vector constructors, the same SSA name must be used, so that + data flow into other basic blocks is maintained. */ + imm_use_iterator use_iter; + gimple *use_stmt; + tree lhs = gimple_get_lhs (stmt_info->stmt); + FOR_EACH_IMM_USE_STMT (use_stmt, use_iter, lhs) + { + if (is_gimple_assign (use_stmt) + && gimple_assign_rhs_code (use_stmt) == CONSTRUCTOR + && use_stmt->bb == stmt_info->stmt->bb) + { + gassign *rstmt + = gimple_build_assign (gimple_get_lhs (use_stmt), + gimple_get_lhs (new_stmt)); + gimple_stmt_iterator rgsi = gsi_for_stmt (use_stmt); + gsi_replace (&rgsi, rstmt, true); + } + } + if (vec_cvt_dest) { new_temp = build1 (VIEW_CONVERT_EXPR, vectype_out, new_temp); -- 2.17.1