From patchwork Thu Oct 26 19:06:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 117252 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp1123701qgn; Thu, 26 Oct 2017 12:07:24 -0700 (PDT) X-Google-Smtp-Source: ABhQp+ShbVlXL1kh0/YX/yaPkyBJYGCusE9L/IewT9SR/r7NN7D8sJqXpei5I1s4FGejY+64QRpR X-Received: by 10.159.253.71 with SMTP id b7mr5251125plx.169.1509044844110; Thu, 26 Oct 2017 12:07:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1509044844; cv=none; d=google.com; s=arc-20160816; b=S+WOzgW9sQupV8nJcCiKj2YKIYndTVnCSuCSSSAwXb0f3l3bQhyfPsrsqz0IIgZnrv 6SrKSVqoH74ZxamvHkdA1K5xWTai8iuGEs5WD7VXgXfDgfCTmow720XspucLmS1Gwvdg f0gvCF/7WT81rzPLrA+xNo/t9J51acGJpFazy7nWboIlr4rtcah1Mi+Gz5FnqlnobmTT Ew1pAer/1dx6unDBpu7YSO3Bil4vvMdRO2uzCX3XuLhNcksVrbVa4EQhVxq52xSOTi87 FMsa6v6mmJNr3PnY4zQKOvBpTvwoUYnaYiMHnLnQ37bHq5f8AvkowIzmD95mSSDUhk1m lmHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from:delivered-to :sender:list-help:list-post:list-archive:list-subscribe :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=8z3Keex4/zfwQyTseFfNbDT7vD5nFbEyPNP7r6A1AhQ=; b=QJ6BvUtm0iuhkf7cMw7Wy04Hyv+JxfYzLTpSVMmgavz9ILE5+nkbgn2iIU2Y57NSOI aF51dpzq6vhPpRMRbkm+yGbUINqJ5ZBUTbh+gS+1vp2e4bNunSYtNF0FHQ7Nk90/P6Ct bAHZm9anR9tH6q2SGE0Vhz8eyRWcbSnf5w0qVoNSOGzf8GIs816yWShjiNweulZQdB9j 5g91gSfdAmUI9cp4cOm7pCVIw58kmcvmNn2ZDq/VW6hrS3ewMCquEQ2uc2BmOsmFfym3 skTcDfZ7jP9ChvIkEojUmjEslDfDK35N+YVHd61eifzBqdOpc2woZ89mo0EWrLYihVgi D+Wg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=hi0bgZAq; spf=pass (google.com: domain of libc-alpha-return-86402-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=libc-alpha-return-86402-patch=linaro.org@sourceware.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id h8si3711005pgs.522.2017.10.26.12.07.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Oct 2017 12:07:24 -0700 (PDT) Received-SPF: pass (google.com: domain of libc-alpha-return-86402-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=hi0bgZAq; spf=pass (google.com: domain of libc-alpha-return-86402-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=libc-alpha-return-86402-patch=linaro.org@sourceware.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; q=dns; s=default; b=CP/vn+mihkWKPI20/w96YpnsFoHavGX e6gpqt2neUkXuflDcYzJLzh2yhIUTuhW2a+Twj25zWMI4/QDmsQhRsEs+PBQ4ioK S0ZThjklOhhMt/Wo4DoUU+vTY+QoIXhCyDI3T2ohmfw7enTezdJppywx1zVeZYXA uCUk0UnQaBs4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; s=default; bh=9a/0GMvp2w9/jRYxYkbuXGKY6po=; b=hi0bg ZAqr20yjz9HTaUJyspett0CWg+K+ZjhK24R+8xWbhTR1U6WHWvJlvT79ew4CTIx/ bVazD+FRog9Yw5dDN2Au526G87lIB8Y7jIXP3ize4KHjekNwbQI/qcusqe4euHm4 xI/3tWa4ylyXrIWupvUYNs5btI8OYjDMsvJsgc= Received: (qmail 96845 invoked by alias); 26 Oct 2017 19:07:06 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 96824 invoked by uid 89); 26 Oct 2017 19:07:05 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_STOCKGEN, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy=sk:armv7l X-HELO: mail-qt0-f193.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=8z3Keex4/zfwQyTseFfNbDT7vD5nFbEyPNP7r6A1AhQ=; b=g7i1dk0R+4buONIIrQWHPGKl/Lqzds0Q4RStxxIFI6Jsu/hFiyl0WV2PD0vtNkb+4E /jfVTgO4uwGhXOP7VcrIGxD+4Qyt7a7GVQN7fGdmQjgJN5M4bnaX5WDGejbYfTcAyc6k SPbT6e9rFWEMApi2lz0beBO46W3ai7emHBjuqIfFpy/X/yIXtMq0oVXkWQV7a4Qwlj7t rrJAmppbsm4cI8w2Sbe9KoQe/O+Qb7Wulxx8/mChbhlG0ngtGrsmb66xIV25PTsUz7jM Le9TDGqlSK7WP+/ZvTJl5XV4lIrKSRJBEDjpji4B4Ymm9urs6cPEFiSTvJd0+xZStz8t s53A== X-Gm-Message-State: AMCzsaUj7QB9Nz/21vx3ga3v9JQu1ClJix95tFOk+Ow3MbX6LN2olaL5 su8k/zg3kQ9+EKnG/L1fbpRXegPV3/g= X-Received: by 10.200.15.118 with SMTP id l51mr39875724qtk.181.1509044821084; Thu, 26 Oct 2017 12:07:01 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org Subject: [PATCH 01/25] arm: Implement memcpy ifunc selection in C Date: Thu, 26 Oct 2017 17:06:29 -0200 Message-Id: <1509044813-9951-2-git-send-email-adhemerval.zanella@linaro.org> In-Reply-To: <1509044813-9951-1-git-send-email-adhemerval.zanella@linaro.org> References: <1509044813-9951-1-git-send-email-adhemerval.zanella@linaro.org> This patch refactor ARM memcpy ifunc selector to a C implementation. No functional change is expected, including ifunc resolution rules. It also adds some cleanup: - Internal memcpy hidden definition (__GI_memcpy) is now a hidden symbol. - No need to create hidden definition for the ifunc variants. Checked on armv7-linux-gnueabihf and with a build for arm-linux-gnueabi, arm-linux-gnueabihf with and without multiarch support and with both GCC 7.1 and GCC mainline. I also checked with the some possible multiarch different configurations that trigger different memcpy buids (__ARM_NEON__ && !__SOFT_FP__, !__ARM_NEON__ && !__SOFT_FP__, and !__ARM_NEON__ && __SOFT_FP__). * sysdeps/arm/arm-ifunc.h: New file. * sysdeps/arm/armv7/multiarch/ifunc-memcpy.h: Likewise. * sysdeps/arm/armv7/multiarch/memcpy.c: Likewise. * sysdeps/arm/armv7/multiarch/memcpy_arm.S: Likewise. * sysdeps/arm/armv7/multiarch/rtld-memcpy.S: Likewise. * sysdeps/arm/armv7/multiarch/memcpy_neon.S [!__ARM_NEON__] (__memcpy_neon): Avoid create hidden alias. * sysdeps/arm/armv7/multiarch/memcpy_vfp.S [!__ARM_NEON_] (__memcpy_vfp): Likewise. * sysdeps/arm/armv7/multiarch/Makefile [$(subdir) = string] (sysdep_routines): Add memcpy_arm. * sysdeps/arm/armv7/multiarch/memcpy.S: Remove file. Signed-off-by: Adhemerval Zanella --- ChangeLog | 15 ++++++ sysdeps/arm/arm-ifunc.h | 33 +++++++++++++ sysdeps/arm/armv7/multiarch/Makefile | 2 +- sysdeps/arm/armv7/multiarch/ifunc-memcpy.h | 37 +++++++++++++++ sysdeps/arm/armv7/multiarch/memcpy.S | 76 ------------------------------ sysdeps/arm/armv7/multiarch/memcpy.c | 35 ++++++++++++++ sysdeps/arm/armv7/multiarch/memcpy_arm.S | 10 ++++ sysdeps/arm/armv7/multiarch/memcpy_neon.S | 8 ++-- sysdeps/arm/armv7/multiarch/memcpy_vfp.S | 4 +- sysdeps/arm/armv7/multiarch/rtld-memcpy.S | 1 + 10 files changed, 139 insertions(+), 82 deletions(-) create mode 100644 sysdeps/arm/arm-ifunc.h create mode 100644 sysdeps/arm/armv7/multiarch/ifunc-memcpy.h delete mode 100644 sysdeps/arm/armv7/multiarch/memcpy.S create mode 100644 sysdeps/arm/armv7/multiarch/memcpy.c create mode 100644 sysdeps/arm/armv7/multiarch/memcpy_arm.S create mode 100644 sysdeps/arm/armv7/multiarch/rtld-memcpy.S -- 2.7.4 diff --git a/sysdeps/arm/arm-ifunc.h b/sysdeps/arm/arm-ifunc.h new file mode 100644 index 0000000..52cb533 --- /dev/null +++ b/sysdeps/arm/arm-ifunc.h @@ -0,0 +1,33 @@ +/* Common definition for ifunc resolvers. Linux/ARM version. + This file is part of the GNU C Library. + Copyright (C) 2017 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#define INIT_ARCH() + +#define arm_libc_ifunc_redirected(redirected_name, name, expr) \ + __ifunc (redirected_name, name, expr(hwcap), int hwcap, INIT_ARCH) + +#if defined SHARED +# define arm_libc_ifunc_hidden_def(redirect_name, name) \ + __hidden_ver1 (name, __GI_##name, redirect_name) \ + __attribute__ ((visibility ("hidden"))) +#else +# define arm_libc_ifunc_hidden_def(redirect_name, name) +#endif diff --git a/sysdeps/arm/armv7/multiarch/Makefile b/sysdeps/arm/armv7/multiarch/Makefile index 9e1e61c..1e62ef9 100644 --- a/sysdeps/arm/armv7/multiarch/Makefile +++ b/sysdeps/arm/armv7/multiarch/Makefile @@ -1,3 +1,3 @@ ifeq ($(subdir),string) -sysdep_routines += memcpy_neon memcpy_vfp memchr_neon +sysdep_routines += memcpy_neon memcpy_vfp memchr_neon memcpy_arm endif diff --git a/sysdeps/arm/armv7/multiarch/ifunc-memcpy.h b/sysdeps/arm/armv7/multiarch/ifunc-memcpy.h new file mode 100644 index 0000000..78cef2a --- /dev/null +++ b/sysdeps/arm/armv7/multiarch/ifunc-memcpy.h @@ -0,0 +1,37 @@ +/* Common definition for memcpy resolver. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifdef __SOFTFP__ +__typeof (REDIRECT_NAME) OPTIMIZE (arm) attribute_hidden; +#endif +__typeof (REDIRECT_NAME) OPTIMIZE (vfp) attribute_hidden; +__typeof (REDIRECT_NAME) OPTIMIZE (neon) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (int hwcap) +{ + if (hwcap & HWCAP_ARM_NEON) + return OPTIMIZE (neon); +#ifdef __SOFTFP__ + if (hwcap & HWCAP_ARM_VFP) + return OPTIMIZE (vfp); + return OPTIMIZE (arm); +#else + return OPTIMIZE (vfp); +#endif +} diff --git a/sysdeps/arm/armv7/multiarch/memcpy.S b/sysdeps/arm/armv7/multiarch/memcpy.S deleted file mode 100644 index 8a53bda..0000000 --- a/sysdeps/arm/armv7/multiarch/memcpy.S +++ /dev/null @@ -1,76 +0,0 @@ -/* Multiple versions of memcpy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2013-2017 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -/* Thumb requires excess IT instructions here. */ -#define NO_THUMB -#include -#include - -#if IS_IN (libc) -/* Under __ARM_NEON__, memcpy_neon.S defines the name memcpy. */ -# ifndef __ARM_NEON__ - .text -ENTRY(memcpy) - .type memcpy, %gnu_indirect_function -# ifdef __SOFTFP__ - ldr r1, .Lmemcpy_arm - tst r0, #HWCAP_ARM_VFP - ldrne r1, .Lmemcpy_vfp -# else - ldr r1, .Lmemcpy_vfp -# endif - tst r0, #HWCAP_ARM_NEON - ldrne r1, .Lmemcpy_neon -1: - add r0, r1, pc - DO_RET(lr) - -# ifdef __SOFTFP__ -.Lmemcpy_arm: - .long C_SYMBOL_NAME(__memcpy_arm) - 1b - PC_OFS -# endif -.Lmemcpy_neon: - .long C_SYMBOL_NAME(__memcpy_neon) - 1b - PC_OFS -.Lmemcpy_vfp: - .long C_SYMBOL_NAME(__memcpy_vfp) - 1b - PC_OFS - -END(memcpy) - -libc_hidden_builtin_def (memcpy) -#endif /* Not __ARM_NEON__. */ - -/* These versions of memcpy are defined not to clobber any VFP or NEON - registers so they must always call the ARM variant of the memcpy code. */ -strong_alias (__memcpy_arm, __aeabi_memcpy) -strong_alias (__memcpy_arm, __aeabi_memcpy4) -strong_alias (__memcpy_arm, __aeabi_memcpy8) -libc_hidden_def (__memcpy_arm) - -#undef libc_hidden_builtin_def -#define libc_hidden_builtin_def(name) -#undef weak_alias -#define weak_alias(x, y) -#undef libc_hidden_def -#define libc_hidden_def(name) - -#define memcpy __memcpy_arm - -#endif - -#include "memcpy_impl.S" diff --git a/sysdeps/arm/armv7/multiarch/memcpy.c b/sysdeps/arm/armv7/multiarch/memcpy.c new file mode 100644 index 0000000..b94a017 --- /dev/null +++ b/sysdeps/arm/armv7/multiarch/memcpy.c @@ -0,0 +1,35 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* For __ARM_NEON__ memchr_neon.S defines memchr directly and ifunc + is not used. */ +#if IS_IN (libc) && !defined (__ARM_NEON__) +# define memcpy __redirect_memcpy +# include +# undef memcpy + +# include + +# define SYMBOL_NAME memcpy +# include "ifunc-memcpy.h" + +arm_libc_ifunc_redirected (__redirect_memcpy, memcpy, IFUNC_SELECTOR); + +arm_libc_ifunc_hidden_def (__redirect_memcpy, memcpy); +#endif diff --git a/sysdeps/arm/armv7/multiarch/memcpy_arm.S b/sysdeps/arm/armv7/multiarch/memcpy_arm.S new file mode 100644 index 0000000..e4a9a68 --- /dev/null +++ b/sysdeps/arm/armv7/multiarch/memcpy_arm.S @@ -0,0 +1,10 @@ +#define memcpy __memcpy_arm +#undef libc_hidden_builtin_def +#define libc_hidden_builtin_def(a) +#include "memcpy_impl.S" + +/* These versions of memcpy are defined not to clobber any VFP or NEON + registers so they must always call the ARM variant of the memcpy code. */ +strong_alias (__memcpy_arm, __aeabi_memcpy) +strong_alias (__memcpy_arm, __aeabi_memcpy4) +strong_alias (__memcpy_arm, __aeabi_memcpy8) diff --git a/sysdeps/arm/armv7/multiarch/memcpy_neon.S b/sysdeps/arm/armv7/multiarch/memcpy_neon.S index e60d1cc..1a8d8bb 100644 --- a/sysdeps/arm/armv7/multiarch/memcpy_neon.S +++ b/sysdeps/arm/armv7/multiarch/memcpy_neon.S @@ -1,8 +1,8 @@ -#ifdef __ARM_NEON__ -/* Under __ARM_NEON__, this file defines memcpy directly. */ -libc_hidden_builtin_def (memcpy) -#else +/* For __ARM_NEON__ this file defines memcpy. */ +#ifndef __ARM_NEON__ # define memcpy __memcpy_neon +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(a) #endif #define MEMCPY_NEON diff --git a/sysdeps/arm/armv7/multiarch/memcpy_vfp.S b/sysdeps/arm/armv7/multiarch/memcpy_vfp.S index e008c04..d1e9ede 100644 --- a/sysdeps/arm/armv7/multiarch/memcpy_vfp.S +++ b/sysdeps/arm/armv7/multiarch/memcpy_vfp.S @@ -1,7 +1,9 @@ -/* Under __ARM_NEON__, memcpy_neon.S defines memcpy directly +/* Under __ARM_NEON__ memcpy_neon.S defines memcpy directly and the __memcpy_vfp code will never be used. */ #ifndef __ARM_NEON__ # define MEMCPY_VFP # define memcpy __memcpy_vfp +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(a) # include "memcpy_impl.S" #endif diff --git a/sysdeps/arm/armv7/multiarch/rtld-memcpy.S b/sysdeps/arm/armv7/multiarch/rtld-memcpy.S new file mode 100644 index 0000000..ca23875 --- /dev/null +++ b/sysdeps/arm/armv7/multiarch/rtld-memcpy.S @@ -0,0 +1 @@ +#include