From patchwork Mon Oct 9 12:08:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9631EE95A97 for ; Mon, 9 Oct 2023 12:11:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346496AbjJIMLD (ORCPT ); Mon, 9 Oct 2023 08:11:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346484AbjJIMLA (ORCPT ); Mon, 9 Oct 2023 08:11:00 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 784DE11F; Mon, 9 Oct 2023 05:10:52 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A42FC433C7; Mon, 9 Oct 2023 12:10:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853452; bh=yZCoppzoGnZPmAd9Kauu4UXsNYRCsI9CR42S+W5etl4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KfaSEuyti9yWaWrEBf3akhOEHiu5eb2yx+UnsZ18vCm+Y1bC5B/UAqx0zLltqo+ip 3fHLpAwZPY9dFL3cvPerE8BiKh4Tf7SSMDY/ry21LIvnJum7TuVkBK1t9+nYWjOmwg PzDZUnCMEWbKgWvT7xDv8SzZK3GYR5ASxUwMXngtHUQ9WtFOp2/K6geP57XBi8noQw gJb0b5s6U9MHI5YpOdBbj/pQhynDfJgbiAmFAyJBPB+8PaaGprZ1DXMCwAmNJ/xxx+ GadTieXWjIHaAn/GKtUmG7bDgn90LmltQnMaPfmoZpAYO1NBhXSwxZF4uPTPY7J2Z7 g7Jb8R0d8hCxw== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:36 +0100 Subject: [PATCH v6 02/38] prctl: arch-agnostic prctl for shadow stack MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-2-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=4712; i=broonie@kernel.org; h=from:subject:message-id; bh=yZCoppzoGnZPmAd9Kauu4UXsNYRCsI9CR42S+W5etl4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2SDvU9dypsYcMutZwOrXCms75ZIsrxzA45vLCi 1QP1N8+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtkgAKCRAk1otyXVSH0FD3B/ 9+eSR3IayfadFWaepYJfA2rkRAM2ngjlHgiL/7/n61CQ9wpdWs9fxUO6kwNxn+agL6L3TD9bZMvpDq zqxzfCD8PeE/IWYexVyudu4VaT1aCv2c5XNk2gJdwlV5rNduptKeYOHOxX2rl2u9h9dmFYclt5Rz6V je6RdJkBXzraM538hH5NNEGTwaLMGKoefu3DWIOjw39uHSvwIx47NvNc0CpKlImqTSlFr16tgHn8wI 3jzVYmQeL5F/VIIXydcFZJ7JxU2MpQyK2J/2kcyacOyrGr/zhzY+3kFfrLoMDa3Q1nwBQHVHCFn1LE yes3qfbGW1Ja8ku/1QEZdHg70UNsH3 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Three architectures (x86, aarch64, riscv) have announced support for shadow stacks with fairly similar functionality. While x86 is using arch_prctl() to control the functionality neither arm64 nor riscv uses that interface so this patch adds arch-agnostic prctl() support to get and set status of shadow stacks and lock the current configuation to prevent further changes, with support for turning on and off individual subfeatures so applications can limit their exposure to features that they do not need. The features are: - PR_SHADOW_STACK_ENABLE: Tracking and enforcement of shadow stacks, including allocation of a shadow stack if one is not already allocated. - PR_SHADOW_STACK_WRITE: Writes to specific addresses in the shadow stack. - PR_SHADOW_STACK_PUSH: Push additional values onto the shadow stack. These features are expected to be inherited by new threads and cleared on exec(), unknown features should be rejected for enable but accepted for locking (in order to allow for future proofing). This is based on a patch originally written by Deepak Gupta but modified fairly heavily, support for indirect landing pads is removed, additional modes added and the locking interface reworked. The set status prctl() is also reworked to just set flags, if setting/reading the shadow stack pointer is required this could be a separate prctl. Signed-off-by: Mark Brown --- include/linux/mm.h | 4 ++++ include/uapi/linux/prctl.h | 22 ++++++++++++++++++++++ kernel/sys.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index bf5d0b1b16f4..1f0d93151a36 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4062,4 +4062,8 @@ static inline void accept_memory(phys_addr_t start, phys_addr_t end) #endif +int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status); +int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); +int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); + #endif /* _LINUX_MM_H */ diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 3c36aeade991..0de3d6ee18e0 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -305,4 +305,26 @@ struct prctl_mm_map { # define PR_RISCV_V_VSTATE_CTRL_NEXT_MASK 0xc # define PR_RISCV_V_VSTATE_CTRL_MASK 0x1f +/* + * Get the current shadow stack configuration for the current thread, + * this will be the value configured via PR_SET_SHADOW_STACK_STATUS. + */ +#define PR_GET_SHADOW_STACK_STATUS 71 + +/* + * Set the current shadow stack configuration. Enabling the shadow + * stack will cause a shadow stack to be allocated for the thread. + */ +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +/* + * Prevent further changes to the specified shadow stack + * configuration. All bits may be locked via this call, including + * undefined bits. + */ +#define PR_LOCK_SHADOW_STACK_STATUS 73 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sys.c b/kernel/sys.c index 2410e3999ebe..b26423a614a9 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2302,6 +2302,21 @@ int __weak arch_prctl_spec_ctrl_set(struct task_struct *t, unsigned long which, return -EINVAL; } +int __weak arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status) +{ + return -EINVAL; +} + +int __weak arch_set_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + +int __weak arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + #define PR_IO_FLUSHER (PF_MEMALLOC_NOIO | PF_LOCAL_THROTTLE) #ifdef CONFIG_ANON_VMA_NAME @@ -2720,6 +2735,21 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, case PR_RISCV_V_GET_CONTROL: error = RISCV_V_GET_CONTROL(); break; + case PR_GET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_get_shadow_stack_status(me, (unsigned long __user *) arg2); + break; + case PR_SET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_set_shadow_stack_status(me, arg2); + break; + case PR_LOCK_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_lock_shadow_stack_status(me, arg2); + break; default: error = -EINVAL; break; From patchwork Mon Oct 9 12:08:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0F87E95A9A for ; Mon, 9 Oct 2023 12:11:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346484AbjJIML0 (ORCPT ); Mon, 9 Oct 2023 08:11:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346482AbjJIMLI (ORCPT ); Mon, 9 Oct 2023 08:11:08 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64B63AF; Mon, 9 Oct 2023 05:11:06 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B81EBC43391; Mon, 9 Oct 2023 12:10:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853465; bh=Z3injfV8td0LqFLeGew+/6AX1Y4kpAJIpTs5MPlZTRs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=L0XZBUaOQXf9y9fsjxiv9eVVR1prSeIrYHJDBabA/g3k+G8CWfM8b33trE7OnKpn0 tJQfJ9ff2252fM07wZUSgLSpa7vHahVvLQ0ecavPAIEALPtz4WXg3avclmJQ4oAQEc ZQaGkO1pG/hwFeXLcXIKsodAVJMQzaf8/WlwBn2af3j909bI30I/fAU+x6FMr/sT5v xAr/OpoWz/xn1Fn88pdwdszlfvdlL0jLFH9368oejjf1kSwcWyjPm9fIwL0fCPqe6G yo36ILgazK0hG9aKH9TqWM7ehe+hvqfq2IWTpacLv+BdZnzHMs0v4G6TRwHCN1GWUb 3D1Fpox6VX5Gw== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:38 +0100 Subject: [PATCH v6 04/38] arm64: Document boot requirements for Guarded Control Stacks MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-4-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1640; i=broonie@kernel.org; h=from:subject:message-id; bh=Z3injfV8td0LqFLeGew+/6AX1Y4kpAJIpTs5MPlZTRs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2U0MFICh6d4ltR364bbYSc0w3Zc0T2/uWHrmEL Sh9o1/CJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtlAAKCRAk1otyXVSH0C6TB/ 418l04NMHU5UE9J1q5yLrG6ZJ2AnKPW9f3cHymnObOAisPsDelPkiMUmjXFq4hNuDacnZhdJOcllux So0ITDMff85io5ycrx6ps5TVQ+4q+hM7uRvQOSLENJOdoWim8AdvhRz5YOz3HoHt29/kemkbOvBRuy YqBfBZrdxTreYRVZXVinGijpDXHFJch6kzmjWNzg9HdLhos+fjGBuN4LYLT8O6sEF5qxehDFNn8ORt XdTDevlSf2Pwa0/MEdKyjS4HcfDwi5v7SixX1a5YRtuO0cwC5RlEvIyTiSRid7hZafvk2o+mhMa/FG m93bsvu0JONpaPuvF7SF8YTGE0o6mn X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org FEAT_GCS introduces a number of new system registers, we require that access to these registers is not trapped when we identify that the feature is detected. Signed-off-by: Mark Brown --- Documentation/arch/arm64/booting.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/Documentation/arch/arm64/booting.rst b/Documentation/arch/arm64/booting.rst index b57776a68f15..de3679770c64 100644 --- a/Documentation/arch/arm64/booting.rst +++ b/Documentation/arch/arm64/booting.rst @@ -411,6 +411,28 @@ Before jumping into the kernel, the following conditions must be met: - HFGRWR_EL2.nPIRE0_EL1 (bit 57) must be initialised to 0b1. + - For features with Guarded Control Stacks (FEAT_GCS): + + - If EL3 is present: + + - SCR_EL3.GCSEn (bit 39) must be initialised to 0b1. + + - If the kernel is entered at EL1 and EL2 is present: + + - HFGITR_EL2.nGCSEPP (bit 59) must be initialised to 0b1. + + - HFGITR_EL2.nGCSSTR_EL1 (bit 58) must be initialised to 0b1. + + - HFGITR_EL2.nGCSPUSHM_EL1 (bit 57) must be initialised to 0b1. + + - HFGRTR_EL2.nGCS_EL1 (bit 53) must be initialised to 0b1. + + - HFGRTR_EL2.nGCS_EL0 (bit 52) must be initialised to 0b1. + + - HFGWTR_EL2.nGCS_EL1 (bit 53) must be initialised to 0b1. + + - HFGWTR_EL2.nGCS_EL0 (bit 52) must be initialised to 0b1. + The requirements described above for CPU mode, caches, MMUs, architected timers, coherency and system registers apply to all CPUs. All CPUs must enter the kernel in the same exception level. Where the values documented From patchwork Mon Oct 9 12:08:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB3B2E95A97 for ; Mon, 9 Oct 2023 12:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346553AbjJIMMG (ORCPT ); Mon, 9 Oct 2023 08:12:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346490AbjJIMLk (ORCPT ); Mon, 9 Oct 2023 08:11:40 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1410138; Mon, 9 Oct 2023 05:11:19 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20477C433CA; Mon, 9 Oct 2023 12:11:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853479; bh=DHXrkEz4uaI5EG4HiHB3b7Wj+xS82Cz4YUQVewuBUzc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=EpeMe9VENZDj/tfvQ0h0WPh3VgDnpa8FPfRzTsdG29A7YqkP/rGRcPp4MReNm5Mqf XL+dh1tkTPB4tmAkGuvvezJZODKRqhlqty3Gtn7cSpzoHSdWiFys190IsLcRrA/qNQ XecFycrJnK5IPz64+hUm1PWC3XJhhkNVwgYteP1WFXK9SUK6ZNVHzk5fT1f7WPjvPE Lpk8pYGgHNsmDAPmrjiXzMhb5PyHYMJNl82tKcWBkPNT5r2VrKXhOHOt33yjDGk6eq spHqVknye6EbUuzvHgibZv9XZod9P3Km4O2q97/kcRhmrkrNvovuJgZXdp2FZqKihC brKSMTrfvdrtQ== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:40 +0100 Subject: [PATCH v6 06/38] arm64/sysreg: Add new system registers for GCS MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-6-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1873; i=broonie@kernel.org; h=from:subject:message-id; bh=DHXrkEz4uaI5EG4HiHB3b7Wj+xS82Cz4YUQVewuBUzc=; b=owGbwMvMwMWocq27KDak/QLjabUkhlTlt1Pdnvex9je8cc6rS7n+7EuY28QjQVenuUqu+v39/67u v8+cOhmNWRgYuRhkxRRZ1j7LWJUeLrF1/qP5r2AGsTKBTGHg4hSAiUS7sP+VElnKb/E8ODlTp/RMWX yO53Ph0HtNP65adFrrXilwX7k1Qt5A02hOr5hpm+XUFWeMHa6cOGYe58P+O4Yn5295ttf7iqUu79+7 n5QLcajt/eKb4X7v90314LuXjRJWROgsEP+dlBBzs6HsQ87ixSujmj8yVfGf0w2PjjueIPGYp9y+uo Hltcxn9cmVJkeO/P7lda1206ejrkdqKqyZn/Ibz2WVYXC6Jlfy//+0o4cCTv/q2K99+/ObppUMhdZn V7uu5r2//UZ/WGlIb+NNjtmG0V7CLkV6mkHXZH2q/68zmbUn5PP0ZV02uRzMHSsdjqhdnFL4zSgoYL 2G0E2bRHdOcQGVI26OP/RjLf0TAA== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org FEAT_GCS introduces a number of new system registers. Add the registers available up to EL2 to sysreg as per DDI0601 2022-12. Signed-off-by: Mark Brown --- arch/arm64/tools/sysreg | 55 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 76ce150e7347..6872bc2c26cd 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -1784,6 +1784,41 @@ Sysreg SMCR_EL1 3 0 1 2 6 Fields SMCR_ELx EndSysreg +SysregFields GCSCR_ELx +Res0 63:10 +Field 9 STREn +Field 8 PUSHMEn +Res0 7 +Field 6 EXLOCKEN +Field 5 RVCHKEN +Res0 4:1 +Field 0 PCRSEL +EndSysregFields + +Sysreg GCSCR_EL1 3 0 2 5 0 +Fields GCSCR_ELx +EndSysreg + +SysregFields GCSPR_ELx +Field 63:3 PTR +Res0 2:0 +EndSysregFields + +Sysreg GCSPR_EL1 3 0 2 5 1 +Fields GCSPR_ELx +EndSysreg + +Sysreg GCSCRE0_EL1 3 0 2 5 2 +Res0 63:11 +Field 10 nTR +Field 9 STREn +Field 8 PUSHMEn +Res0 7:6 +Field 5 RVCHKEN +Res0 4:1 +Field 0 PCRSEL +EndSysreg + Sysreg ALLINT 3 0 4 3 0 Res0 63:14 Field 13 ALLINT @@ -2014,6 +2049,10 @@ Field 4 DZP Field 3:0 BS EndSysreg +Sysreg GCSPR_EL0 3 3 2 5 1 +Fields GCSPR_ELx +EndSysreg + Sysreg SVCR 3 3 4 2 2 Res0 63:2 Field 1 ZA @@ -2342,6 +2381,14 @@ Sysreg SMCR_EL2 3 4 1 2 6 Fields SMCR_ELx EndSysreg +Sysreg GCSCR_EL2 3 4 2 5 0 +Fields GCSCR_ELx +EndSysreg + +Sysreg GCSPR_EL2 3 4 2 5 1 +Fields GCSPR_ELx +EndSysreg + Sysreg DACR32_EL2 3 4 3 0 0 Res0 63:32 Field 31:30 D15 @@ -2401,6 +2448,14 @@ Sysreg SMCR_EL12 3 5 1 2 6 Fields SMCR_ELx EndSysreg +Sysreg GCSCR_EL12 3 5 2 5 0 +Fields GCSCR_ELx +EndSysreg + +Sysreg GCSPR_EL12 3 5 2 5 1 +Fields GCSPR_ELx +EndSysreg + Sysreg FAR_EL12 3 5 6 0 0 Field 63:0 ADDR EndSysreg From patchwork Mon Oct 9 12:08:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82FC3E95A91 for ; Mon, 9 Oct 2023 12:12:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376321AbjJIMMZ (ORCPT ); Mon, 9 Oct 2023 08:12:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346611AbjJIMLo (ORCPT ); Mon, 9 Oct 2023 08:11:44 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D533C1A7; Mon, 9 Oct 2023 05:11:32 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5217BC43140; Mon, 9 Oct 2023 12:11:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853492; bh=sadX6jRs7DdBAThhpdGK2D3phr2V3dLQCMXle4b8GlY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ATlHVM6AZeAPwZROhTDwJAXvT1SullAlI71Vn91fpb40hH0HMjTlCj8nJbcQ3Ud3N vAhVspCDOeDsjgUAUBTNGwGZTHyokcbBzVVVErlZF4gaRCHvA8132tfS0cYpyz7zso 71qgrkpCNYoX1KdrSBTd0nrMqNW0UrLUB3WheJ1j3Kxtod1a3ousFlJ406c1t6piU1 g2ElyABWAZHRc+nRTRI776ggjev9EOJbu2ipxrPSF075+9nWfWD952VGnmTcu1crg0 Z7YMy8L5LgmZwKM9Yix342mpSS9lneIbrD2Co0e1PKXQCoXswDTriYD68tGfTJY1IF 3nsQDM/pqU+/g== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:42 +0100 Subject: [PATCH v6 08/38] arm64/gcs: Add manual encodings of GCS instructions MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-8-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2577; i=broonie@kernel.org; h=from:subject:message-id; bh=sadX6jRs7DdBAThhpdGK2D3phr2V3dLQCMXle4b8GlY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2X6KvSDevFmNU73sRXi3dVSic6kb7utwq77DtU O9AtxbeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtlwAKCRAk1otyXVSH0PJzB/ 9bjtfdH/kreNste9Rhszz++5TtrXwaf+/u3cMsmbyDsfIECxfrMIwsOBr1iJ61TV0lhBla3gTWKzee B7mmzi+ozphCNHbM0MDRz0F62SQzuaeuxTGQWp7rTUbtm2aMqwK3vOlXDbfkp7E1yyMaHz3Z5xh6Tw h8d1vRGZv5sXyZyDBBI2aiEO6lThtTrLozvtuWQ1vgfWue+wjzrq1aMDgiN8IeAW1HmSGSYmliogM6 532osJm9A/FjGeWX/7VE6UK1Q45iesDwGfzySGm0oNzW9Uex8YR3/NTmPzJ2zulrHA/KnIamunKmUR uhxZ729qwQ62A064U8mJ4OD4jGhU9v X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Define C callable functions for GCS instructions used by the kernel. In order to avoid ambitious toolchain requirements for GCS support these are manually encoded, this means we have fixed register numbers which will be a bit limiting for the compiler but none of these should be used in sufficiently fast paths for this to be a problem. Note that GCSSTTR is used to store to EL0. Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 51 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/include/asm/uaccess.h | 22 +++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h new file mode 100644 index 000000000000..7c5e95218db6 --- /dev/null +++ b/arch/arm64/include/asm/gcs.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Ltd. + */ +#ifndef __ASM_GCS_H +#define __ASM_GCS_H + +#include +#include + +static inline void gcsb_dsync(void) +{ + asm volatile(".inst 0xd503227f" : : : "memory"); +} + +static inline void gcsstr(u64 *addr, u64 val) +{ + register u64 *_addr __asm__ ("x0") = addr; + register long _val __asm__ ("x1") = val; + + /* GCSSTTR x1, x0 */ + asm volatile( + ".inst 0xd91f1c01\n" + : + : "rZ" (_val), "r" (_addr) + : "memory"); +} + +static inline void gcsss1(u64 Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static inline u64 gcsss2(void) +{ + u64 Xt; + + asm volatile( + "SYSL %0, #3, C7, C7, #3\n" + : "=r" (Xt) + : + : "memory"); + + return Xt; +} + +#endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 14be5000c5a0..22e10e79f56a 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -425,4 +425,26 @@ static inline size_t probe_subpage_writeable(const char __user *uaddr, #endif /* CONFIG_ARCH_HAS_SUBPAGE_FAULTS */ +#ifdef CONFIG_ARM64_GCS + +static inline int gcssttr(unsigned long __user *addr, unsigned long val) +{ + register unsigned long __user *_addr __asm__ ("x0") = addr; + register unsigned long _val __asm__ ("x1") = val; + int err = 0; + + /* GCSSTTR x1, x0 */ + asm volatile( + "1: .inst 0xd91f1c01\n" + "2: \n" + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0) + : "+r" (err) + : "rZ" (_val), "r" (_addr) + : "memory"); + + return err; +} + +#endif /* CONFIG_ARM64_GCS */ + #endif /* __ASM_UACCESS_H */ From patchwork Mon Oct 9 12:08:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD5B0E95A8E for ; Mon, 9 Oct 2023 12:12:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346613AbjJIMMt (ORCPT ); Mon, 9 Oct 2023 08:12:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346479AbjJIMMG (ORCPT ); Mon, 9 Oct 2023 08:12:06 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 255F210B; Mon, 9 Oct 2023 05:11:46 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9380BC433C8; Mon, 9 Oct 2023 12:11:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853505; bh=XR/JPTU4enAJ4pcvH8p8vc1g83S1au3eNbsnIb4ywnQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rP9e/pVEU1XRIjj2Q9uCE+vVJQiSYLoJDDmvoN3Y+fxV4PDyb0Vs8fWv5dE22EdGz B0vb94hqvciW/TEsRbokha5ZqcprcqYd4Usrcmcr+y93H4dv3DKJAP5z76Exn3fa8W QPI2YjyCvmt6N7kiuo4tZXT3emLIL5NPZKALokimQSPX+EFbD6N5Q642tmTnWEAJKJ 7VgVCJPy4ZM09WcaxWoJv7Zi7JTkaps2cE7IhSdwgsbyuLYN5uaWHbaRut0htOs6RP au6TpCRBbjfRQmVcgQG5R8QCGK7+4xIbwmdWHVFFsdzFe+q++lxMpBhENWv+eDo/c+ wD58DJP+NWIbg== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:44 +0100 Subject: [PATCH v6 10/38] arm64/cpufeature: Runtime detection of Guarded Control Stack (GCS) MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-10-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2903; i=broonie@kernel.org; h=from:subject:message-id; bh=XR/JPTU4enAJ4pcvH8p8vc1g83S1au3eNbsnIb4ywnQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2ZAGHqvxhmUj+PJXN+2VsAYfmovxTrEoIYjBrX 2bzDiHWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtmQAKCRAk1otyXVSH0LouB/ 4jMbLZC7D6Zl9Jb2+lm+LeAvBZcAupYd70rbx+3xkuxxpcymaCTO3zxHXISjX1SuQqDXdZ2Qr7PfGU AD2rdudrC2wAVWtVlIuvD1s8iyD5b5nIqoiQrsYKu1s6UByhZnLIrwDq8BXPLFUNVBkwd0pMBv4DVJ OSnLTLI6pOfw2BRvIdml05y4Axyxu65IveS3S9NiO7r8GE9M/+Yjsv2UB9TfMCca2oJdKaITbH+0ht xPC1Vr58uRUI6DwnycYpnuE4Hl+w2AI0CrMHY3eiChkjo0rYQHBWfhw4qqpUKruKnfZ739/1FkLGqb XWMGY3i9fPM/PuQFxByz8YwPzG371T X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Add a cpufeature for GCS, allowing other code to conditionally support it at runtime. Signed-off-by: Mark Brown --- arch/arm64/include/asm/cpufeature.h | 6 ++++++ arch/arm64/kernel/cpufeature.c | 16 ++++++++++++++++ arch/arm64/tools/cpucaps | 1 + 3 files changed, 23 insertions(+) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 5bba39376055..4a5eea41f8ed 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -831,6 +831,12 @@ static inline bool system_supports_tlb_range(void) cpus_have_const_cap(ARM64_HAS_TLB_RANGE); } +static inline bool system_supports_gcs(void) +{ + return IS_ENABLED(CONFIG_ARM64_GCS) && + cpus_have_const_cap(ARM64_HAS_GCS); +} + int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt); bool try_emulate_mrs(struct pt_regs *regs, u32 isn); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 444a73c2e638..e247dce1759c 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -255,6 +255,8 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { }; static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_GCS), + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_EL1_GCS_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SME), FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_EL1_SME_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_EL1_MPAM_frac_SHIFT, 4, 0), @@ -2220,6 +2222,12 @@ static void cpu_enable_mops(const struct arm64_cpu_capabilities *__unused) sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_MSCEn); } +static void cpu_enable_gcs(const struct arm64_cpu_capabilities *__unused) +{ + /* GCS is not currently used at EL1 */ + write_sysreg_s(0, SYS_GCSCR_EL1); +} + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -2719,6 +2727,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature, ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, EVT, IMP) }, + { + .desc = "Guarded Control Stack (GCS)", + .capability = ARM64_HAS_GCS, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .cpu_enable = cpu_enable_gcs, + .matches = has_cpuid_feature, + ARM64_CPUID_FIELDS(ID_AA64PFR1_EL1, GCS, IMP) + }, {}, }; diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index c3f06fdef609..9b470b311f29 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -27,6 +27,7 @@ HAS_ECV_CNTPOFF HAS_EPAN HAS_EVT HAS_FGT +HAS_GCS HAS_GENERIC_AUTH HAS_GENERIC_AUTH_ARCH_QARMA3 HAS_GENERIC_AUTH_ARCH_QARMA5 From patchwork Mon Oct 9 12:08:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6627E95A96 for ; Mon, 9 Oct 2023 12:13:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376266AbjJIMNQ (ORCPT ); Mon, 9 Oct 2023 08:13:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376293AbjJIMM0 (ORCPT ); Mon, 9 Oct 2023 08:12:26 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01747D7E; Mon, 9 Oct 2023 05:11:59 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0416C433AD; Mon, 9 Oct 2023 12:11:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853519; bh=zDATCw7GkV4as2xIWJbM5lO/ptaSO6aC4Ay36HJrvVc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ENONym6LxarYigzay1mcU79+hI/vAoV9b0fyw/J2UUuwzThWjoEsUpd5lkbA5Z4K9 P0/CV25CwF+cAdlJvnqs4FLOJ71AZpQHGjzgeYsO08ImaCjN2qlN1BUP8PHqIGKOXQ WpPk61fewqYUGFsssDoC9aSSwPZyJhjkUCFZlDhQ8yPqKCJZLKOpL6emCCp5a+Nt0F UiBPnweFH4fMygG0qDGzAA40oBFFjm3hJQIPQuWZaVzH01aIpiymcfVqNdS7u0T4Vv UG1ntzuihR5n3QwqjmpsnuK/iB+fLmz4UFUxHP9y0RRrp66Z2ME/lzF5xhqIOyc6/T Qq+H9HwSESlGQ== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:46 +0100 Subject: [PATCH v6 12/38] mm: Define VM_SHADOW_STACK for arm64 when we support GCS MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-12-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2033; i=broonie@kernel.org; h=from:subject:message-id; bh=zDATCw7GkV4as2xIWJbM5lO/ptaSO6aC4Ay36HJrvVc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2ay7Z6vBAn0o2n8KbXYYUHM/hfs9dGuBEwzhip d4LBRGmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtmgAKCRAk1otyXVSH0EiSB/ 9SbJ/RyLl4aYRdiZA9qvebhW2OnXSpQIYBFctQA1wlLoaPha9i/Ik0Vcc88mJxHtw1nMccHURGYhSU WCzpcB67v4hwl5zXoXemcvWBisselh+LDaTccm9Q3IeHsVH9qWHfOrV7GNYlzp2ZOhI1oOhalEzIxm FQ38pEJXorEZIfUyQJmBFQ7eBGdhU3kGlrCmBmdSrsXwJmRVdUFxY1e8w6BWvLbvQ3SZip6TI34YQA 743qUcrWXijdl+JN19YmwtPdHkYckOX7l6DVS9xao5FEkQAk1Ig2lKNDNuMLRbaDa3OP1kFcEAd+/7 Cm1jQ37tmyw8VT8okrsaLiHFvroO5P X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Use VM_HIGH_ARCH_5 for guarded control stack pages. Signed-off-by: Mark Brown --- Documentation/filesystems/proc.rst | 2 +- fs/proc/task_mmu.c | 3 +++ include/linux/mm.h | 12 +++++++++++- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesystems/proc.rst index 2b59cff8be17..9f61f34afc4c 100644 --- a/Documentation/filesystems/proc.rst +++ b/Documentation/filesystems/proc.rst @@ -570,7 +570,7 @@ encoded manner. The codes are the following: mt arm64 MTE allocation tags are enabled um userfaultfd missing tracking uw userfaultfd wr-protect tracking - ss shadow stack page + ss shadow/guarded control stack page == ======================================= Note that there is no guarantee that every flag and associated mnemonic will diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3dd5be96691b..c0fb4e8b3bbc 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -699,6 +699,9 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */ #ifdef CONFIG_X86_USER_SHADOW_STACK [ilog2(VM_SHADOW_STACK)] = "ss", +#endif +#ifdef CONFIG_ARM64_GCS + [ilog2(VM_SHADOW_STACK)] = "ss", #endif }; size_t i; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1f0d93151a36..9649a1942dda 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -352,7 +352,17 @@ extern unsigned int kobjsize(const void *objp); * for more details on the guard size. */ # define VM_SHADOW_STACK VM_HIGH_ARCH_5 -#else +#endif + +#if defined(CONFIG_ARM64_GCS) +/* + * arm64's Guarded Control Stack implements similar functionality and + * has similar constraints to shadow stacks. + */ +# define VM_SHADOW_STACK VM_HIGH_ARCH_5 +#endif + +#ifndef VM_SHADOW_STACK # define VM_SHADOW_STACK VM_NONE #endif From patchwork Mon Oct 9 12:08:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731244 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25BD9E95A97 for ; Mon, 9 Oct 2023 12:13:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346677AbjJIMNq (ORCPT ); Mon, 9 Oct 2023 08:13:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346631AbjJIMNP (ORCPT ); Mon, 9 Oct 2023 08:13:15 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C89B0199; Mon, 9 Oct 2023 05:12:17 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB78AC43397; Mon, 9 Oct 2023 12:12:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853537; bh=DQdjfUveIhNPPqVwZdWHihE2VRq12xchEABLOZK1HfI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=oLL83hbOmtscSO0PxQD/EOaGE8Qiz0tIqnzyl8bOMqgdll6vJb4H0So35c8Uqf5Zk vbhc4U3bNiYCFO/SgDmgoq8LM04Z4CqvaO4nef/FJsDYoEAK0CQ0glCjepZfOyAyzA /05mCxwa25QrZU9/38apir4UreXWuyiWYGvq2uSCKszXVR3ssIxUa007qSxBiHgqD5 VHPGYDdtxQgaIWydtQE/Xt9jx20CiKjWtvewAicjumFUedToaIJ4esyUrAGtBRLSfn eXYIJfaDRomsREUt1rkPGVONqoEyIC08t+Boq48Y18UYuJ5KRKzp3HnXLABo8J+sih hXrxCwOF3ErqA== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:48 +0100 Subject: [PATCH v6 14/38] KVM: arm64: Manage GCS registers for guests MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-14-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5343; i=broonie@kernel.org; h=from:subject:message-id; bh=DQdjfUveIhNPPqVwZdWHihE2VRq12xchEABLOZK1HfI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2cOqMxsIngsQ7FiuAEwC3vODto57yqr+sncOIg 3t7fey2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtnAAKCRAk1otyXVSH0GtQB/ 9fhqJQfXnPsmSl2P/gBGshXyv7rosoPImj6rUaan7wksa2dTf7gj3QOMeHTLXvDbMr1yi1Gi5Z+H3t //gR/KfyE59kAN0X/vKYxVGfrMbjsNlkaUIUjEffjJKIB/ZoSwP6lbeXYAyobxpYpx+ULJLO6nVVYp mrJCOqwHPRM2sqNmcz/xcsyv+Y1XssOmEAhCTLa1mOkYPEv5Foxot/R6kGBtAJ1SaIWg/pUWEHI97j JZ5rvo7MWdE8Oo2E+PgD3UvccRZnAa8vthv3p5pNg5f4u4XDt3a4ycdfxektfciAh9kSWzCSpwOxfA YDdSstN1QUiMhAGJ7SrcsiFSRIDKFH X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org GCS introduces a number of system registers for EL1 and EL0, on systems with GCS we need to context switch them and expose them to VMMs to allow guests to use GCS. Traps are already disabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++ arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 17 +++++++++++++++++ arch/arm64/kvm/sys_regs.c | 22 ++++++++++++++++++++++ 3 files changed, 51 insertions(+) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index af06ccb7ee34..7171d4c7e5ed 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -365,6 +365,12 @@ enum vcpu_sysreg { PIR_EL1, /* Permission Indirection Register 1 (EL1) */ PIRE0_EL1, /* Permission Indirection Register 0 (EL1) */ + /* Guarded Control Stack registers */ + GCSCRE0_EL1, /* Guarded Control Stack Control (EL0) */ + GCSCR_EL1, /* Guarded Control Stack Control (EL1) */ + GCSPR_EL0, /* Guarded Control Stack Pointer (EL0) */ + GCSPR_EL1, /* Guarded Control Stack Pointer (EL1) */ + /* 32bit specific registers. */ DACR32_EL2, /* Domain Access Control Register */ IFSR32_EL2, /* Instruction Fault Status Register */ @@ -1142,6 +1148,12 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); #define kvm_vm_has_ran_once(kvm) \ (test_bit(KVM_ARCH_FLAG_HAS_RAN_ONCE, &(kvm)->arch.flags)) +static inline bool has_gcs(void) +{ + return IS_ENABLED(CONFIG_ARM64_GCS) && + cpus_have_final_cap(ARM64_HAS_GCS); +} + int kvm_trng_call(struct kvm_vcpu *vcpu); #ifdef CONFIG_KVM extern phys_addr_t hyp_mem_base; diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index bb6b571ec627..ec34d4a90717 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -25,6 +25,8 @@ static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); + if (has_gcs()) + ctxt_sys_reg(ctxt, GCSPR_EL0) = read_sysreg_s(SYS_GCSPR_EL0); } static inline bool ctxt_has_mte(struct kvm_cpu_context *ctxt) @@ -62,6 +64,12 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) ctxt_sys_reg(ctxt, PAR_EL1) = read_sysreg_par(); ctxt_sys_reg(ctxt, TPIDR_EL1) = read_sysreg(tpidr_el1); + if (has_gcs()) { + ctxt_sys_reg(ctxt, GCSPR_EL1) = read_sysreg_el1(SYS_GCSPR); + ctxt_sys_reg(ctxt, GCSCR_EL1) = read_sysreg_el1(SYS_GCSCR); + ctxt_sys_reg(ctxt, GCSCRE0_EL1) = read_sysreg_s(SYS_GCSCRE0_EL1); + } + if (ctxt_has_mte(ctxt)) { ctxt_sys_reg(ctxt, TFSR_EL1) = read_sysreg_el1(SYS_TFSR); ctxt_sys_reg(ctxt, TFSRE0_EL1) = read_sysreg_s(SYS_TFSRE0_EL1); @@ -95,6 +103,8 @@ static inline void __sysreg_restore_user_state(struct kvm_cpu_context *ctxt) { write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL0), tpidr_el0); write_sysreg(ctxt_sys_reg(ctxt, TPIDRRO_EL0), tpidrro_el0); + if (has_gcs()) + write_sysreg_s(ctxt_sys_reg(ctxt, GCSPR_EL0), SYS_GCSPR_EL0); } static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) @@ -138,6 +148,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) write_sysreg(ctxt_sys_reg(ctxt, PAR_EL1), par_el1); write_sysreg(ctxt_sys_reg(ctxt, TPIDR_EL1), tpidr_el1); + if (has_gcs()) { + write_sysreg_el1(ctxt_sys_reg(ctxt, GCSPR_EL1), SYS_GCSPR); + write_sysreg_el1(ctxt_sys_reg(ctxt, GCSCR_EL1), SYS_GCSCR); + write_sysreg_s(ctxt_sys_reg(ctxt, GCSCRE0_EL1), + SYS_GCSCRE0_EL1); + } + if (ctxt_has_mte(ctxt)) { write_sysreg_el1(ctxt_sys_reg(ctxt, TFSR_EL1), SYS_TFSR); write_sysreg_s(ctxt_sys_reg(ctxt, TFSRE0_EL1), SYS_TFSRE0_EL1); diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index e92ec810d449..dd8966d52f3c 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1768,6 +1768,23 @@ static unsigned int mte_visibility(const struct kvm_vcpu *vcpu, .visibility = mte_visibility, \ } +static unsigned int gcs_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + if (has_gcs()) + return 0; + + return REG_HIDDEN; +} + +#define GCS_REG(name) { \ + SYS_DESC(SYS_##name), \ + .access = undef_access, \ + .reset = reset_unknown, \ + .reg = name, \ + .visibility = gcs_visibility, \ +} + static unsigned int el2_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { @@ -2080,6 +2097,10 @@ static const struct sys_reg_desc sys_reg_descs[] = { PTRAUTH_KEY(APDB), PTRAUTH_KEY(APGA), + GCS_REG(GCSCR_EL1), + GCS_REG(GCSPR_EL1), + GCS_REG(GCSCRE0_EL1), + { SYS_DESC(SYS_SPSR_EL1), access_spsr}, { SYS_DESC(SYS_ELR_EL1), access_elr}, @@ -2164,6 +2185,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_SMIDR_EL1), undef_access }, { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, { SYS_DESC(SYS_CTR_EL0), access_ctr }, + GCS_REG(GCSPR_EL0), { SYS_DESC(SYS_SVCR), undef_access }, { PMU_SYS_REG(PMCR_EL0), .access = access_pmcr, From patchwork Mon Oct 9 12:08:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA823E95A99 for ; Mon, 9 Oct 2023 12:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376328AbjJIMN7 (ORCPT ); Mon, 9 Oct 2023 08:13:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346510AbjJIMNR (ORCPT ); Mon, 9 Oct 2023 08:13:17 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4092F1A4; Mon, 9 Oct 2023 05:12:25 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BD389C433CC; Mon, 9 Oct 2023 12:12:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853544; bh=HYoopujMYYx9DIwWQ/56zZgOFvuBdtqWANEIYJVZJO8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=j9U89MV1GBU5jOVOLZpCSQry22wWl///VTwFucF5T+ppLeyNlY+ijNsKStJjX34zz TKtMy0coDvVyA+8ko0HqNYCUMkgHFACaF6LEWpLS1QxmhzeE9XnEjzyQ0M8mEULhOy LoDl1uMcTq+Z0BPqDawgEjWUIaaGtEiTfAuKnfR4txzV/sle6cv7axr7grDcYKQzB0 cMUl8mqk7ccL5ojvydYfH77A8erbD64hcfT3XJ4LIwDDSLm8GsFaXNsL/c0VGUO7LX elVXN6hoTFeVaqLLe/WXRoRAuUJPbwwSWMPqVxC1UU5DdJApxPBpYygj5t2E7gAAB0 PKRs52LSefP9A== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:49 +0100 Subject: [PATCH v6 15/38] arm64/gcs: Allow GCS usage at EL0 and EL1 MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-15-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2250; i=broonie@kernel.org; h=from:subject:message-id; bh=HYoopujMYYx9DIwWQ/56zZgOFvuBdtqWANEIYJVZJO8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2dfP1gHPRa8C0+dKhkGMXquypekN26lEUP9onB zh3nmgqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtnQAKCRAk1otyXVSH0GamB/ 4zkdD6yWJZ6LzOyEnr8Hw3FnF1NVfnuu8/QcW8ibj5rkATYPiavxMq3LPm07WlulO7ZtCP7Gdeg0GH kDP3wzqx62horUPdFah+l76lfzloOsBbU39pySCg9LZVRsnjG1FyFqDV/K7qE5GDRondjc529mjnmW /Zq7/3B279Cu8mPCyA322DF81o2QzFuyQqwCwR1f/nvr1kKFH/CAFqJR1R1zfkYTz948CaxtKKewpc 7k2FwkiXhUAo2OyVo5APZZQ4cGYkttf6x44lxUYW5FpwU9Okq5Y1aiKcdQ3ADCFCe2kgupisipbU4v oMCEU+UrDbK8wEiZ/6nPcObFK6gRMN X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org There is a control HCRX_EL2.GCSEn which must be set to allow GCS features to take effect at lower ELs and also fine grained traps for GCS usage at EL0 and EL1. Configure all these to allow GCS usage by EL0 and EL1. Signed-off-by: Mark Brown --- arch/arm64/include/asm/el2_setup.h | 17 +++++++++++++++++ arch/arm64/include/asm/kvm_arm.h | 4 ++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index b7afaa026842..17672563e333 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -27,6 +27,14 @@ ubfx x0, x0, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 cbz x0, .Lskip_hcrx_\@ mov_q x0, HCRX_HOST_FLAGS + + /* Enable GCS if supported */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_hcrx_\@ + orr x0, x0, #HCRX_EL2_GCSEn + +.Lset_hcrx_\@: msr_s SYS_HCRX_EL2, x0 .Lskip_hcrx_\@: .endm @@ -190,6 +198,15 @@ orr x0, x0, #HFGxTR_EL2_nPIR_EL1 orr x0, x0, #HFGxTR_EL2_nPIRE0_EL1 + /* GCS depends on PIE so we don't check it if PIE is absent */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_fgt_\@ + + /* Disable traps of access to GCS registers at EL0 and EL1 */ + orr x0, x0, #HFGxTR_EL2_nGCS_EL1_MASK + orr x0, x0, #HFGxTR_EL2_nGCS_EL0_MASK + .Lset_fgt_\@: msr_s SYS_HFGRTR_EL2, x0 msr_s SYS_HFGWTR_EL2, x0 diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 5882b2415596..d74b626b829a 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -102,8 +102,8 @@ #define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) -#define HCRX_GUEST_FLAGS (HCRX_EL2_SMPME | HCRX_EL2_TCR2En) -#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En) +#define HCRX_GUEST_FLAGS (HCRX_EL2_SMPME | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn) +#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn) /* TCR_EL2 Registers bits */ #define TCR_EL2_RES1 ((1U << 31) | (1 << 23)) From patchwork Mon Oct 9 12:08:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B17CE95A99 for ; Mon, 9 Oct 2023 12:14:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376285AbjJIMOe (ORCPT ); Mon, 9 Oct 2023 08:14:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376308AbjJIMNu (ORCPT ); Mon, 9 Oct 2023 08:13:50 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48BFE1988; Mon, 9 Oct 2023 05:12:47 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 793CAC433C7; Mon, 9 Oct 2023 12:12:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853566; bh=sIXiMHqGT53KOfswnAd1tnEcZ0j0aXrKTuf98iIbpAU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JV5GeEBlJhSJovOYlr9LVRaAEDTP0w+UILC7gn/T+CmREOnxo/15rVH2H5B29mcyy 7Me6PJ6ziPknbDa6vhOalgsQtJSh1R+Uevgl6EtmR0VXd2YnpTdHGf1gbu4aZfU2YJ mi9haE+Mjrd/z7jPoPRR0GuKrzD2GVM7uq9vs4IEBgvxmObSW+pPqTYJ+pL1Eg3I5V aF34C/Gwl2lS1YtI40+85xBGssAh4u3agUaBM3SnFVcwgTCtNSGoKOEioeEgFznaS4 xWabbOFnhdC+EgBBI9hJMz8d2fNQ95OxabxbNX4CqErqPLcso7muq66u+NG8ypuEzP m+436lfAVY4+Q== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:52 +0100 Subject: [PATCH v6 18/38] arm64/traps: Handle GCS exceptions MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-18-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=6026; i=broonie@kernel.org; h=from:subject:message-id; bh=sIXiMHqGT53KOfswnAd1tnEcZ0j0aXrKTuf98iIbpAU=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2fgmbR8ogvFOWuIBQXLZL/EI/xdEFTBW33fk9I jIPh42iJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtnwAKCRAk1otyXVSH0GCHB/ 9mrJqknBMqEyHavL9Xwwy1/do6TLd3nJsr2NHNNb75vImTX45rZzOAJGX1cuPwoH/mnx14L/0tTiJq Wy55ygcGfkLEn52PHv0jIL8GSBrqMJsWIzynC4I/pjtzoH5RGPjgKNiEtjBma6FnKxqo4bcAN5Vg4z 39hDxGjRNmRlugF518e8CxGly/U6GERIPZl5P5GVFIhW408whh6jikyaYxbkmHIHWs2SXLrTI8EDIi sf/MO9nsCrD8bQpXsRVYHBOhUI0s4crj0y8zJJy2LESmOrlCWvzJbHl9iaP+SfLRGVOcDJ7XSPXBYh l0/kx5/dtGQuSCmkX/GkJnk6slsSdI X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org A new exception code is defined for GCS specific faults other than standard load/store faults, for example GCS token validation failures, add handling for this. These faults are reported to userspace as segfaults with code SEGV_CPERR (protection error), mirroring the reporting for x86 shadow stack errors. GCS faults due to memory load/store operations generate data aborts with a flag set, these will be handled separately as part of the data abort handling. Since we do not currently enable GCS for EL1 we should not get any faults there but while we're at it we wire things up there, treating any GCS fault as fatal. Signed-off-by: Mark Brown --- arch/arm64/include/asm/esr.h | 28 +++++++++++++++++++++++++++- arch/arm64/include/asm/exception.h | 2 ++ arch/arm64/kernel/entry-common.c | 23 +++++++++++++++++++++++ arch/arm64/kernel/traps.c | 11 +++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index ae35939f395b..a87a8305051f 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -51,7 +51,8 @@ #define ESR_ELx_EC_FP_EXC32 (0x28) /* Unallocated EC: 0x29 - 0x2B */ #define ESR_ELx_EC_FP_EXC64 (0x2C) -/* Unallocated EC: 0x2D - 0x2E */ +#define ESR_ELx_EC_GCS (0x2D) +/* Unallocated EC: 0x2E */ #define ESR_ELx_EC_SERROR (0x2F) #define ESR_ELx_EC_BREAKPT_LOW (0x30) #define ESR_ELx_EC_BREAKPT_CUR (0x31) @@ -382,6 +383,31 @@ #define ESR_ELx_MOPS_ISS_SRCREG(esr) (((esr) & (UL(0x1f) << 5)) >> 5) #define ESR_ELx_MOPS_ISS_SIZEREG(esr) (((esr) & (UL(0x1f) << 0)) >> 0) +/* ISS field definitions for GCS */ +#define ESR_ELx_ExType_SHIFT (20) +#define ESR_ELx_ExType_MASK GENMASK(23, 20) +#define ESR_ELx_Raddr_SHIFT (10) +#define ESR_ELx_Raddr_MASK GENMASK(14, 10) +#define ESR_ELx_Rn_SHIFT (5) +#define ESR_ELx_Rn_MASK GENMASK(9, 5) +#define ESR_ELx_Rvalue_SHIFT 5 +#define ESR_ELx_Rvalue_MASK GENMASK(9, 5) +#define ESR_ELx_IT_SHIFT (0) +#define ESR_ELx_IT_MASK GENMASK(4, 0) + +#define ESR_ELx_ExType_DATA_CHECK 0 +#define ESR_ELx_ExType_EXLOCK 1 +#define ESR_ELx_ExType_STR 2 + +#define ESR_ELx_IT_RET 0 +#define ESR_ELx_IT_GCSPOPM 1 +#define ESR_ELx_IT_RET_KEYA 2 +#define ESR_ELx_IT_RET_KEYB 3 +#define ESR_ELx_IT_GCSSS1 4 +#define ESR_ELx_IT_GCSSS2 5 +#define ESR_ELx_IT_GCSPOPCX 6 +#define ESR_ELx_IT_GCSPOPX 7 + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index ad688e157c9b..99caff458e20 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -57,6 +57,8 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr); void do_el1_undef(struct pt_regs *regs, unsigned long esr); void do_el0_bti(struct pt_regs *regs); void do_el1_bti(struct pt_regs *regs, unsigned long esr); +void do_el0_gcs(struct pt_regs *regs, unsigned long esr); +void do_el1_gcs(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 0fc94207e69a..52d78ce63a4e 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -429,6 +429,15 @@ static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_el1_gcs(regs, esr); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -471,6 +480,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_BTI: el1_bti(regs, esr); break; + case ESR_ELx_EC_GCS: + el1_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: @@ -650,6 +662,14 @@ static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } +static void noinstr el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_user_mode(regs); + local_daif_restore(DAIF_PROCCTX); + do_el0_gcs(regs, esr); + exit_to_user_mode(regs); +} + static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); @@ -732,6 +752,9 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_MOPS: el0_mops(regs, esr); break; + case ESR_ELx_EC_GCS: + el0_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_LOW: case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 8b70759cdbb9..65dab959f620 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -500,6 +500,16 @@ void do_el1_bti(struct pt_regs *regs, unsigned long esr) die("Oops - BTI", regs, esr); } +void do_el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + force_signal_inject(SIGSEGV, SEGV_CPERR, regs->pc, 0); +} + +void do_el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + die("Oops - GCS", regs, esr); +} + void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); @@ -884,6 +894,7 @@ static const char *esr_class_str[] = { [ESR_ELx_EC_MOPS] = "MOPS", [ESR_ELx_EC_FP_EXC32] = "FP (AArch32)", [ESR_ELx_EC_FP_EXC64] = "FP (AArch64)", + [ESR_ELx_EC_GCS] = "Guarded Control Stack", [ESR_ELx_EC_SERROR] = "SError", [ESR_ELx_EC_BREAKPT_LOW] = "Breakpoint (lower EL)", [ESR_ELx_EC_BREAKPT_CUR] = "Breakpoint (current EL)", From patchwork Mon Oct 9 12:08:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2DFCE95A9C for ; Mon, 9 Oct 2023 12:15:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376301AbjJIMO6 (ORCPT ); Mon, 9 Oct 2023 08:14:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376343AbjJIMOB (ORCPT ); Mon, 9 Oct 2023 08:14:01 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14B3ED48; Mon, 9 Oct 2023 05:12:54 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B76BC433CA; Mon, 9 Oct 2023 12:12:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853573; bh=U1l0Oub8DGXeBnKIw3SOas1EfStDVYy3cBry+b4y2KY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nidKzb6KjaupAUNRQ7cqd5mxL//1PZ7Zr0fs9cUxP2Sqp3dUEAlrgp4LoAr30o55W U710+3rdLZ6ZzwuKUFNq/3ntxIVKLkn5Jp3FYJJm+pvRYVzp/jJUgl7V2Yz1NgIShJ PV9n9kGvXyWc6fCUyE/z8OmzA6pXnJ+ZP2i9eETmkNDUMu8lIrog4PNrjVYVbd1/tV o8w6eNqVhOf9GYYIjrWmGkApgNbk+6ArtRI/AwjFPMBBUz3+W2gE+vMZrCvgFrig8Z oM5dG9qkqa7qk4KCOugY5ojML/qLd+Hmu0zdL5OTwKi+A0NGUy01Ur8afAbwIxEHgX QxYQH47PfwqGA== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:53 +0100 Subject: [PATCH v6 19/38] arm64/mm: Handle GCS data aborts MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-19-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5679; i=broonie@kernel.org; h=from:subject:message-id; bh=U1l0Oub8DGXeBnKIw3SOas1EfStDVYy3cBry+b4y2KY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2gzvC6/uRznZqzKfe/TGEH5CxDCcE+V/5gJRNL +1aTQbKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtoAAKCRAk1otyXVSH0LU/B/ 0Ydu7yT5bbYSp6sxq/2tFGycuTvi9kpwlvDefRWYOXy0CmFYrhj7uu3+ALG2+fh7QzI3l5YiwjNw7t l1w03bBgoR9OW4hW4J94aGDQrIt9ZgpRP0eROtCgquKr4tFxdD+c/BVIumziARnNX8LU4HKUb1oe/5 WrND2lz3GpPjyo/ilirjaB/GhzLquoVwzOqINgs00dIkV9jp0cfBZqdSQVopTDVeNOB7lKhRJJIRg/ Bn9VJkcivmoT36Bxq6ij2Wdd2U0mhQNPQkxxpeFhUJvmxQ0Tgljc1tOOSl2W/HGw5D14eunK2/jhmh +SrtxgjjbNtrL/kg+/sLkqxk2hVxc1 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org All GCS operations at EL0 must happen on a page which is marked as having UnprivGCS access, including read operations. If a GCS operation attempts to access a page without this then it will generate a data abort with the GCS bit set in ESR_EL1.ISS2. EL0 may validly generate such faults, for example due to copy on write which will cause the GCS data to be stored in a read only page with no GCS permissions until the actual copy happens. Since UnprivGCS allows both reads and writes to the GCS (though only through GCS operations) we need to ensure that the memory management subsystem handles GCS accesses as writes at all times. Do this by adding FAULT_FLAG_WRITE to any GCS page faults, adding handling to ensure that invalid cases are identfied as such early so the memory management core does not think they will succeed. The core cannot distinguish between VMAs which are generally writeable and VMAs which are only writeable through GCS operations. EL1 may validly write to EL0 GCS for management purposes (eg, while initialising with cap tokens). We also report any GCS faults in VMAs not marked as part of a GCS as access violations, causing a fault to be delivered to userspace if it attempts to do GCS operations outside a GCS. Signed-off-by: Mark Brown --- arch/arm64/mm/fault.c | 79 +++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 2e5d1e238af9..9dd143df4483 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -494,13 +494,30 @@ static void do_bad_area(unsigned long far, unsigned long esr, } } +/* + * Note: not valid for EL1 DC IVAC, but we never use that such that it + * should fault. EL0 cannot issue DC IVAC (undef). + */ +static bool is_write_abort(unsigned long esr) +{ + return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); +} + +static bool is_gcs_fault(unsigned long esr) +{ + if (!esr_is_data_abort(esr)) + return false; + + return ESR_ELx_ISS2(esr) & ESR_ELx_GCS; +} + #define VM_FAULT_BADMAP ((__force vm_fault_t)0x010000) #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) static vm_fault_t __do_page_fault(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned int mm_flags, unsigned long vm_flags, - struct pt_regs *regs) + unsigned long esr, struct pt_regs *regs) { /* * Ok, we have a good vm_area for this memory access, so we can handle @@ -510,6 +527,26 @@ static vm_fault_t __do_page_fault(struct mm_struct *mm, */ if (!(vma->vm_flags & vm_flags)) return VM_FAULT_BADACCESS; + + if (vma->vm_flags & VM_SHADOW_STACK) { + /* + * Writes to a GCS must either be generated by a GCS + * operation or be from EL1. + */ + if (is_write_abort(esr) && + !(is_gcs_fault(esr) || is_el1_data_abort(esr))) + return VM_FAULT_BADACCESS; + } else { + /* + * GCS faults should never happen for pages that are + * not part of a GCS and the operation being attempted + * can never succeed. + */ + if (is_gcs_fault(esr)) + return VM_FAULT_BADACCESS; + } + + return handle_mm_fault(vma, addr, mm_flags, regs); } @@ -518,13 +555,18 @@ static bool is_el0_instruction_abort(unsigned long esr) return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_LOW; } -/* - * Note: not valid for EL1 DC IVAC, but we never use that such that it - * should fault. EL0 cannot issue DC IVAC (undef). - */ -static bool is_write_abort(unsigned long esr) +static bool is_invalid_el0_gcs_access(struct vm_area_struct *vma, u64 esr) { - return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); + if (!system_supports_gcs()) + return false; + if (likely(!(vma->vm_flags & VM_SHADOW_STACK))) { + if (is_gcs_fault(esr)) + return true; + return false; + } + if (is_gcs_fault(esr)) + return false; + return is_write_abort(esr); } static int __kprobes do_page_fault(unsigned long far, unsigned long esr, @@ -573,6 +615,13 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, /* If EPAN is absent then exec implies read */ if (!cpus_have_const_cap(ARM64_HAS_EPAN)) vm_flags |= VM_EXEC; + /* + * Upgrade read faults to write faults, GCS reads must + * occur on a page marked as GCS so we need to trigger + * copy on write always. + */ + if (is_gcs_fault(esr)) + mm_flags |= FAULT_FLAG_WRITE; } if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { @@ -594,6 +643,20 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, if (!vma) goto lock_mmap; + /* + * We get legitimate write faults for GCS pages from GCS + * operations, even when the initial operation was a read, as + * a result of upgrading GCS accesses to writes for CoW but + * GCS acceses outside of a GCS must fail. Specifically check + * for this since the mm core isn't able to distinguish + * invalid GCS access from valid ones and will try to resolve + * the fault. + */ + if (is_invalid_el0_gcs_access(vma, esr)) { + vma_end_read(vma); + goto lock_mmap; + } + if (!(vma->vm_flags & vm_flags)) { vma_end_read(vma); goto lock_mmap; @@ -623,7 +686,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto done; } - fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, regs); + fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, esr, regs); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { From patchwork Mon Oct 9 12:08:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FA11E95A8E for ; Mon, 9 Oct 2023 12:15:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346518AbjJIMPc (ORCPT ); Mon, 9 Oct 2023 08:15:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346530AbjJIMOu (ORCPT ); Mon, 9 Oct 2023 08:14:50 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DA4F1BFE; Mon, 9 Oct 2023 05:13:14 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6314FC433C9; Mon, 9 Oct 2023 12:13:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853593; bh=wOX/VCidy7u5iRp7jBzI1P7kDYiqBOUXs9IWPVUSObE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SvQGI1I6auJECsA5HNnZPnyfUEmJKB1kyHPmG4CVELIdmz23aGqtIEB5AHqLotzYw DxAAxMYBdToLGpdI2/CQy80sFIcV/M7qAWyaWrl/YQO6S6vMNpd38WALdW1/EGT8T3 F8B/Z7rtlUrTdyCLfcVSIolqwMwlzDUC38mIndIltNjwcRtyiXoay7wMMmiVyZf59L UWahtjJyBl8oYTy1+YMQ/XAUbdqrjnBRON3Dfak4y4Y7RvwJ/NXQOcUyT691wrVa9M U81w+7IDixDnVa1+qAkeE5guZkm1si0HgdS+qwia9QGC6W3BJvYspxbYMJMbXmjuCQ O5/inc6lIA/KQ== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:56 +0100 Subject: [PATCH v6 22/38] arm64/gcs: Implement shadow stack prctl() interface MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-22-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=5673; i=broonie@kernel.org; h=from:subject:message-id; bh=wOX/VCidy7u5iRp7jBzI1P7kDYiqBOUXs9IWPVUSObE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2iNjKPEwjokbez9Es4G4AUiv47sNRTtzlb93Ez Z1ACm7WJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtogAKCRAk1otyXVSH0OkuCA CClNtwZ/5fUorz1+MZqrsmriqbf343Fpy7PnkMNDhInuESIyi8lH+YEFCHTVTEReGs0XmsYL3fT/5D r6escX4sKstyFt7b7YXL7isYq2Q9gFysn1qt5ZJ7GMYZY42ZwzTiGaq6WNEjGIqpSu/7SueXHVU0Ff 1j8wvEXjR/e9gZIdBRAOC4XLa/SIXyGmzu99tgPMQcRB53xwIoJ7RRFWPo6Q89+1YSdzmvAO7oXn3o xUPcxMoAD9BxPRgyNM+kFpWmcBZhzvOE34UoGe5t9lyl/iH+2oYmvGAaZ5vP+z+qv4TL6KYCHC1Hv3 /gTZiJAz/IlR9MGyN4GKzPWQ3KGIl7 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Implement the architecture neutral prtctl() interface for setting the shadow stack status, this supports setting and reading the current GCS configuration for the current thread. Userspace can enable basic GCS functionality and additionally also support for GCS pushes and arbatrary GCS stores. It is expected that this prctl() will be called very early in application startup, for example by the dynamic linker, and not subsequently adjusted during normal operation. Users should carefully note that after enabling GCS for a thread GCS will become active with no call stack so it is not normally possible to return from the function that invoked the prctl(). State is stored per thread, enabling GCS for a thread causes a GCS to be allocated for that thread. Userspace may lock the current GCS configuration by specifying PR_SHADOW_STACK_ENABLE_LOCK, this prevents any further changes to the GCS configuration via any means. If GCS is not being enabled then all flags other than _LOCK are ignored, it is not possible to enable stores or pops without enabling GCS. When disabling the GCS we do not free the allocated stack, this allows for inspection of the GCS after disabling as part of fault reporting. Since it is not an expected use case and since it presents some complications in determining what to do with previously initialsed data on the GCS attempts to reenable GCS after this are rejected. This can be revisted if a use case arises. Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 22 ++++++++++ arch/arm64/include/asm/processor.h | 1 + arch/arm64/mm/gcs.c | 82 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 105 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 4371a2f99b4a..c150e76869a1 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -48,6 +48,9 @@ static inline u64 gcsss2(void) return Xt; } +#define PR_SHADOW_STACK_SUPPORTED_STATUS_MASK \ + (PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | PR_SHADOW_STACK_PUSH) + #ifdef CONFIG_ARM64_GCS static inline bool task_gcs_el0_enabled(struct task_struct *task) @@ -61,6 +64,20 @@ void gcs_preserve_current_state(void); unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, size_t size); +static inline int gcs_check_locked(struct task_struct *task, + unsigned long new_val) +{ + unsigned long cur_val = task->thread.gcs_el0_mode; + + cur_val &= task->thread.gcs_el0_locked; + new_val &= task->thread.gcs_el0_locked; + + if (cur_val != new_val) + return -EBUSY; + + return 0; +} + #else static inline bool task_gcs_el0_enabled(struct task_struct *task) @@ -76,6 +93,11 @@ static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, { return -ENOTSUPP; } +static inline int gcs_check_locked(struct task_struct *task, + unsigned long new_val) +{ + return 0; +} #endif diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index c28681cf9721..029d20ea7878 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -181,6 +181,7 @@ struct thread_struct { u64 tpidr2_el0; #ifdef CONFIG_ARM64_GCS unsigned int gcs_el0_mode; + unsigned int gcs_el0_locked; u64 gcspr_el0; u64 gcs_base; u64 gcs_size; diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index cb0a64bf90af..2b2223b13fc3 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -84,3 +84,85 @@ void gcs_free(struct task_struct *task) task->thread.gcs_base = 0; task->thread.gcs_size = 0; } + +int arch_set_shadow_stack_status(struct task_struct *task, unsigned long arg) +{ + unsigned long gcs, size; + int ret; + + if (!system_supports_gcs()) + return -EINVAL; + + if (is_compat_thread(task_thread_info(task))) + return -EINVAL; + + /* Reject unknown flags */ + if (arg & ~PR_SHADOW_STACK_SUPPORTED_STATUS_MASK) + return -EINVAL; + + ret = gcs_check_locked(task, arg); + if (ret != 0) + return ret; + + /* If we are enabling GCS then make sure we have a stack */ + if (arg & PR_SHADOW_STACK_ENABLE) { + if (!task_gcs_el0_enabled(task)) { + /* Do not allow GCS to be reenabled */ + if (task->thread.gcs_base) + return -EINVAL; + + if (task != current) + return -EBUSY; + + size = gcs_size(0); + gcs = alloc_gcs(task->thread.gcspr_el0, size, + 0, 0); + if (!gcs) + return -ENOMEM; + + task->thread.gcspr_el0 = gcs + size - sizeof(u64); + task->thread.gcs_base = gcs; + task->thread.gcs_size = size; + if (task == current) + write_sysreg_s(task->thread.gcspr_el0, + SYS_GCSPR_EL0); + + } + } + + task->thread.gcs_el0_mode = arg; + if (task == current) + gcs_set_el0_mode(task); + + return 0; +} + +int arch_get_shadow_stack_status(struct task_struct *task, + unsigned long __user *arg) +{ + if (!system_supports_gcs()) + return -EINVAL; + + if (is_compat_thread(task_thread_info(task))) + return -EINVAL; + + return put_user(task->thread.gcs_el0_mode, arg); +} + +int arch_lock_shadow_stack_status(struct task_struct *task, + unsigned long arg) +{ + if (!system_supports_gcs()) + return -EINVAL; + + if (is_compat_thread(task_thread_info(task))) + return -EINVAL; + + /* + * We support locking unknown bits so applications can prevent + * any changes in a future proof manner. + */ + task->thread.gcs_el0_locked |= arg; + + return 0; +} From patchwork Mon Oct 9 12:08:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13939E95A9C for ; Mon, 9 Oct 2023 12:16:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346550AbjJIMQO (ORCPT ); Mon, 9 Oct 2023 08:16:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346459AbjJIMPV (ORCPT ); Mon, 9 Oct 2023 08:15:21 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C8081FCE; Mon, 9 Oct 2023 05:13:27 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93E3DC4167D; Mon, 9 Oct 2023 12:13:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853606; bh=ZneLmCePUg63/qsSwQjRgo0lJ2wvCQfwcnIhBN+HCvU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pMylGXyro5zpDCWMajUKC7BW/REgEuYKhID+LyYyH5Snn88sviH2mVDG1LqC8DhHu /rb3/WdsVXascc+TWcbAaQm0weKwc4RHZ70ncA8M5m74+GC0jm2FIhgPg6BqEGj35q kuTxVoxUHtlFJ9QB3Dl73QTk+6rSJuXH0kFYYc1iRpFuROKh1KF02HhMwvl/Nb/E8r 3FtkRwb0ogEexxNWpKAKd91RoNh4ExK9KLLNAqVDE7Va4WbOOIh3BJPaA2pvYJ7dwB KE7Jq+y1W0t2C49fls2u2nbbaiXVCOSlnCMtM5nhqTolwzAwx73/nnnUgm1BR+raFh aeDk1sl/8cbfg== From: Mark Brown Date: Mon, 09 Oct 2023 13:08:58 +0100 Subject: [PATCH v6 24/38] arm64/signal: Set up and restore the GCS context for signal handlers MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-24-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=7272; i=broonie@kernel.org; h=from:subject:message-id; bh=ZneLmCePUg63/qsSwQjRgo0lJ2wvCQfwcnIhBN+HCvU=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2k+Yu5E/WU+7sfmQCBl2rh45oRS4s0qpeS50/J 6wxbOqSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtpAAKCRAk1otyXVSH0BTJB/ 9XHnWgdli5cKd6AxjFaDID/jh9Wyh5sSGi9fWVij/UGcOE2B6pBw3mi/DxCe01u2p6yJx5TUMuwbxP aeqLutwc9FpixTiJHU6JDxzlMDYSbhphgMNiY7AY4jBfimBy56c1aLQuKs8dH+EtFNFdKlRPLHNedN nmk51x/P4LGZ/rQrEWMDAMeglVnR2GZFmCmKGAFN71Y6lWJjl/kRrurQwp0tl3KmUyZq2ByedwGTm6 ZmKQCncBQ7OnyZa35wzxpKByWTj6Ponkov5hEtG6aOL9Ksc7j048tu+35DruNM7tJ13846OK+RUJcy DXJ23fGcRtQVlyTpYlktLHU8KcpVIW X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org When invoking a signal handler we use the GCS configuration and stack for the current thread. Since we implement signal return by calling the signal handler with a return address set up pointing to a trampoline in the vDSO we need to also configure any active GCS for this by pushing a frame for the trampoline onto the GCS. If we do not do this then signal return will generate a GCS protection fault. In order to guard against attempts to bypass GCS protections via signal return we only allow returning with GCSPR_EL0 pointing to an address where it was previously preempted by a signal. We do this by pushing a cap onto the GCS, this takes the form of an architectural GCS cap token with the top bit set which we add on signal entry and validate and pop off on signal return. Since the top bit is set address validation for the token will fail if an attempt is made to use it with the stack switch instructions. Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 2 + arch/arm64/kernel/signal.c | 130 +++++++++++++++++++++++++++++++++++++++++-- arch/arm64/mm/gcs.c | 1 + 3 files changed, 128 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index c150e76869a1..65496103d462 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct ksignal; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 0e8beb3349ea..97004c52aea3 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -34,6 +35,36 @@ #include #include +#ifdef CONFIG_ARM64_GCS +/* Extra bit set in the address distinguishing a signal cap token. */ +#define GCS_SIGNAL_CAP_FLAG BIT(63) + +#define GCS_SIGNAL_CAP(addr) (GCS_CAP(addr) | GCS_SIGNAL_CAP_FLAG) + +static bool gcs_signal_cap_valid(u64 addr, u64 val) +{ + /* + * The top bit should be set, this is an invalid address for + * EL0 and will only be set for caps created by signals. + */ + if (!(val & GCS_SIGNAL_CAP_FLAG)) + return false; + + /* The rest should be a standard architectural cap token. */ + val &= ~GCS_SIGNAL_CAP_FLAG; + + /* The cap must have the low bits set to a token value */ + if (GCS_CAP_TOKEN(val) != GCS_CAP_VALID_TOKEN) + return false; + + /* The cap must store the VA the cap was stored at */ + if (GCS_CAP_ADDR(addr) != GCS_CAP_ADDR(val)) + return false; + + return true; +} +#endif + /* * Do a signal return; undo the signal stack. These are aligned to 128-bit. */ @@ -815,6 +846,45 @@ static int restore_sigframe(struct pt_regs *regs, return err; } +#ifdef CONFIG_ARM64_GCS +static int gcs_restore_signal(void) +{ + u64 gcspr_el0, cap; + int ret; + + if (!system_supports_gcs()) + return 0; + + if (!(current->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE)) + return 0; + + gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + + /* + * GCSPR_EL0 should be pointing at a capped GCS, read the cap... + */ + gcsb_dsync(); + ret = copy_from_user(&cap, (__user void*)gcspr_el0, sizeof(cap)); + if (ret) + return -EFAULT; + + /* + * ...then check that the cap is the actual GCS before + * restoring it. + */ + if (!gcs_signal_cap_valid(gcspr_el0, cap)) + return -EINVAL; + + current->thread.gcspr_el0 = gcspr_el0 + sizeof(cap); + write_sysreg_s(current->thread.gcspr_el0, SYS_GCSPR_EL0); + + return 0; +} + +#else +static int gcs_restore_signal(void) { return 0; } +#endif + SYSCALL_DEFINE0(rt_sigreturn) { struct pt_regs *regs = current_pt_regs(); @@ -841,6 +911,9 @@ SYSCALL_DEFINE0(rt_sigreturn) if (restore_altstack(&frame->uc.uc_stack)) goto badframe; + if (gcs_restore_signal()) + goto badframe; + return regs->regs[0]; badframe: @@ -1071,7 +1144,52 @@ static int get_sigframe(struct rt_sigframe_user_layout *user, return 0; } -static void setup_return(struct pt_regs *regs, struct k_sigaction *ka, +#ifdef CONFIG_ARM64_GCS + +static int gcs_signal_entry(__sigrestore_t sigtramp, struct ksignal *ksig) +{ + unsigned long __user *gcspr_el0; + unsigned long cap[2]; + int ret; + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(current)) + return 0; + + /* + * We are entering a signal handler, current register state is + * active. + */ + gcspr_el0 = (unsigned long __user *)read_sysreg_s(SYS_GCSPR_EL0); + + /* + * Push a cap and the GCS entry for the trampoline onto the GCS. + */ + cap[1] = GCS_SIGNAL_CAP(gcspr_el0 - 1); + cap[0] = (unsigned long)sigtramp; + ret = copy_to_user_gcs(gcspr_el0 - 2, cap, ARRAY_SIZE(cap)); + if (ret != 0) + return ret; + + gcsb_dsync(); + + gcspr_el0 -= 2; + write_sysreg_s((unsigned long)gcspr_el0, SYS_GCSPR_EL0); + + return 0; +} +#else + +static int gcs_signal_entry(__sigrestore_t sigtramp, struct ksignal *ksig) +{ + return 0; +} + +#endif + +static int setup_return(struct pt_regs *regs, struct ksignal *ksig, struct rt_sigframe_user_layout *user, int usig) { __sigrestore_t sigtramp; @@ -1079,7 +1197,7 @@ static void setup_return(struct pt_regs *regs, struct k_sigaction *ka, regs->regs[0] = usig; regs->sp = (unsigned long)user->sigframe; regs->regs[29] = (unsigned long)&user->next_frame->fp; - regs->pc = (unsigned long)ka->sa.sa_handler; + regs->pc = (unsigned long)ksig->ka.sa.sa_handler; /* * Signal delivery is a (wacky) indirect function call in @@ -1119,12 +1237,14 @@ static void setup_return(struct pt_regs *regs, struct k_sigaction *ka, sme_smstop(); } - if (ka->sa.sa_flags & SA_RESTORER) - sigtramp = ka->sa.sa_restorer; + if (ksig->ka.sa.sa_flags & SA_RESTORER) + sigtramp = ksig->ka.sa.sa_restorer; else sigtramp = VDSO_SYMBOL(current->mm->context.vdso, sigtramp); regs->regs[30] = (unsigned long)sigtramp; + + return gcs_signal_entry(sigtramp, ksig); } static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set, @@ -1147,7 +1267,7 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set, err |= __save_altstack(&frame->uc.uc_stack, regs->sp); err |= setup_sigframe(&user, regs, set); if (err == 0) { - setup_return(regs, &ksig->ka, &user, usig); + err = setup_return(regs, ksig, &user, usig); if (ksig->ka.sa.sa_flags & SA_SIGINFO) { err |= copy_siginfo_to_user(&frame->info, &ksig->info); regs->regs[1] = (unsigned long)&frame->info; diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index c718ac4325bb..32e22bff9bce 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -6,6 +6,7 @@ #include #include +#include #include static unsigned long alloc_gcs(unsigned long addr, unsigned long size, From patchwork Mon Oct 9 12:09:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CCDCE95A8E for ; Mon, 9 Oct 2023 12:16:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346654AbjJIMQi (ORCPT ); Mon, 9 Oct 2023 08:16:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346670AbjJIMPd (ORCPT ); Mon, 9 Oct 2023 08:15:33 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B40E21FE3; Mon, 9 Oct 2023 05:13:40 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DC4BAC433C7; Mon, 9 Oct 2023 12:13:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853620; bh=1i+GSTpTbO5P/qld4yaRpURtIaffrsBagKFpXCXGmzg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Dn+JTB1Q6MBUWHdRdmX1zVJKlbRP82QRAInihBBBPCSvLKASb+DMl1YxFOajCSY/E tkJxqrl67Ke/aipeuAJYv1qEFEMNYX7QYKbLKpb0ekjLWD0oOMx6jarm8pyL68fEUN p6daZNVLAQmNitNSRtPqqKZBwI3Cx32AKFrcA066xT5vh6vQbSnKH9mHyKO961wa2Y f/ZBWZ/PuexVcNk05aIiD+VBQU23+uVHhBiPf+qdhMkqcU2njj48ROrM9XrLpKlskH UMCymKGUkrccZ6jkiKcq3ikK9H0NGmKi0heiEtZJWUtToYJdcn0frKtXAfF9drE77V QRp65+26zQUwg== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:00 +0100 Subject: [PATCH v6 26/38] arm64/ptrace: Expose GCS via ptrace and core files MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-26-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=4259; i=broonie@kernel.org; h=from:subject:message-id; bh=1i+GSTpTbO5P/qld4yaRpURtIaffrsBagKFpXCXGmzg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2l6LKdpST7ySLYv5LWeHkMNkrcQaOJdWoxoBqf QQUckdaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtpQAKCRAk1otyXVSH0JQsB/ 9QjqXMz883xP7630YeP/WsHDXXMqbUL1LZMm+9OQba6q0kMurV9IiHrfNL+X3/jURiix8fFKkuQG84 ZA7LYts4GHIaQ8SMmj5KuMNKNxXZyoz5b316NaTGLgUedhn8Q70Z0H7fLxagK1H7+ccLrJaKkbiETa yToRZqV1yatPowYRk614+CUA2/jURY6M5bWe3e8dfKBBrcnYqNPHInwX5Kx16mnEYJT+XPvXjUmUx+ kgvsAPwoaaZzoSfBLT+LuPy+oago2pJK7XJbrRfHheuH3Gzl0uEtdO4TOtiyc9Sb0096fLm/blVzBJ hSKrRrdwGBwx4X5JPWvIF5vyVrxGgN X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Provide a new register type NT_ARM_GCS reporting the current GCS mode and pointer for EL0. Due to the interactions with allocation and deallocation of Guarded Control Stacks we do not permit any changes to the GCS mode via ptrace, only GCSPR_EL0 may be changed. Signed-off-by: Mark Brown --- arch/arm64/include/uapi/asm/ptrace.h | 8 +++++ arch/arm64/kernel/ptrace.c | 59 ++++++++++++++++++++++++++++++++++++ include/uapi/linux/elf.h | 1 + 3 files changed, 68 insertions(+) diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index 7fa2f7036aa7..0f39ba4f3efd 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -324,6 +324,14 @@ struct user_za_header { #define ZA_PT_SIZE(vq) \ (ZA_PT_ZA_OFFSET + ZA_PT_ZA_SIZE(vq)) +/* GCS state (NT_ARM_GCS) */ + +struct user_gcs { + __u64 features_enabled; + __u64 features_locked; + __u64 gcspr_el0; +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI__ASM_PTRACE_H */ diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 20d7ef82de90..f15b8e33561e 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -1409,6 +1410,51 @@ static int tagged_addr_ctrl_set(struct task_struct *target, const struct } #endif +#ifdef CONFIG_ARM64_GCS +static int gcs_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct user_gcs user_gcs; + + if (target == current) + gcs_preserve_current_state(); + + user_gcs.features_enabled = target->thread.gcs_el0_mode; + user_gcs.features_locked = target->thread.gcs_el0_locked; + user_gcs.gcspr_el0 = target->thread.gcspr_el0; + + return membuf_write(&to, &user_gcs, sizeof(user_gcs)); +} + +static int gcs_set(struct task_struct *target, const struct + user_regset *regset, unsigned int pos, + unsigned int count, const void *kbuf, const + void __user *ubuf) +{ + int ret; + struct user_gcs user_gcs; + + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &user_gcs, 0, -1); + if (ret) + return ret; + + if (user_gcs.features_enabled & ~PR_SHADOW_STACK_SUPPORTED_STATUS_MASK) + return -EINVAL; + + /* Do not allow enable via ptrace */ + if ((user_gcs.features_enabled & PR_SHADOW_STACK_ENABLE) && + !!(target->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE)) + return -EBUSY; + + target->thread.gcs_el0_mode = user_gcs.features_enabled; + target->thread.gcs_el0_locked = user_gcs.features_locked; + target->thread.gcspr_el0 = user_gcs.gcspr_el0; + + return 0; +} +#endif + enum aarch64_regset { REGSET_GPR, REGSET_FPR, @@ -1437,6 +1483,9 @@ enum aarch64_regset { #ifdef CONFIG_ARM64_TAGGED_ADDR_ABI REGSET_TAGGED_ADDR_CTRL, #endif +#ifdef CONFIG_ARM64_GCS + REGSET_GCS, +#endif }; static const struct user_regset aarch64_regsets[] = { @@ -1587,6 +1636,16 @@ static const struct user_regset aarch64_regsets[] = { .set = tagged_addr_ctrl_set, }, #endif +#ifdef CONFIG_ARM64_GCS + [REGSET_GCS] = { + .core_note_type = NT_ARM_GCS, + .n = sizeof(struct user_gcs) / sizeof(u64), + .size = sizeof(u64), + .align = sizeof(u64), + .regset_get = gcs_get, + .set = gcs_set, + }, +#endif }; static const struct user_regset_view user_aarch64_view = { diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h index 9b731976ce2f..fe854a53099e 100644 --- a/include/uapi/linux/elf.h +++ b/include/uapi/linux/elf.h @@ -440,6 +440,7 @@ typedef struct elf64_shdr { #define NT_ARM_SSVE 0x40b /* ARM Streaming SVE registers */ #define NT_ARM_ZA 0x40c /* ARM SME ZA registers */ #define NT_ARM_ZT 0x40d /* ARM SME ZT registers */ +#define NT_ARM_GCS 0x40e /* ARM GCS state */ #define NT_ARC_V2 0x600 /* ARCv2 accumulator/extra registers */ #define NT_VMCOREDD 0x700 /* Vmcore Device Dump Note */ #define NT_MIPS_DSP 0x800 /* MIPS DSP ASE registers */ From patchwork Mon Oct 9 12:09:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D9DCE95A99 for ; Mon, 9 Oct 2023 12:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346763AbjJIMV0 (ORCPT ); Mon, 9 Oct 2023 08:21:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346597AbjJIMVR (ORCPT ); Mon, 9 Oct 2023 08:21:17 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1091310E6; Mon, 9 Oct 2023 05:14:07 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70D5FC4339A; Mon, 9 Oct 2023 12:14:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853646; bh=DtrqsekNxnVEUxJ8rAESK/JTguJ8OvysbbylOwjoIT0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QDoT3Rs0dy70eTof0MpL8Clmp5D5N4G+bALCxg28WbX45WAE5V6sk/vUBxZzkCB/w +FzuuGTRC+vbQZTP/LJpXLFQoJoa/dFN8A2DtrbsMC44VXAJojUm/L6KjiZYupFDCz OMIxqgNnfIkH9qe9SiQAivSWq/uD540hHtpMeukDnCv+Vc3dGA9ZE1l8BLDAHcW1Dy 8gni6ij4/iID3PCNkQjgI1ZeN1tRleWQo/QbuHLXszaDuCR53Gq8KF+8bPUzhuXaC9 +/wUvXP2RYyXaJVthDj1tU9T5Ul+HV6UcQvMFV2HaaFGAZXQV0aJ5NpUZoW+zq7wOT 94CHAAamp91hg== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:04 +0100 Subject: [PATCH v6 30/38] kselftest/arm64: Add framework support for GCS to signal handling tests MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-30-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=1590; i=broonie@kernel.org; h=from:subject:message-id; bh=DtrqsekNxnVEUxJ8rAESK/JTguJ8OvysbbylOwjoIT0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2pq7Qa4GULwtgg75uflK1VZRll9L2EqVA5sWym iRyIswiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtqQAKCRAk1otyXVSH0ANuB/ 9EYHlc6mJYvvbF83tv2urYj/dsRi3MZGM9bL42WvASqT9Fe6RxhqSl8+yBkjK9zlnt99TI4RlGWH2H 7TfVXMoQwlEhSvycSBbhwJp3KYFlja1mU/BKWyUAB524VVNqk0FaXeYcYsePJoMup3ysfKgET+5Ow7 BudxsoJJPf8SPjz7bcGQshCK0bskVX6ZG6P5z72NmpAo78PQjf/fdgwdqG7MgXhLVpnJcrp6LUXsKI py/v2S1R5Fz2sZW8EPhj+QlJjC+XxuA+xBNiUeTNFfblGGaQHnVhVpXKDfA9t69zSl3KsWhTZ14286 LPcR/NM0Ve/MSPwybGmUQHozt93Wpz X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Teach the framework about the GCS signal context, avoiding warnings on the unknown context. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/signal/testcases/testcases.c | 7 +++++++ tools/testing/selftests/arm64/signal/testcases/testcases.h | 1 + 2 files changed, 8 insertions(+) diff --git a/tools/testing/selftests/arm64/signal/testcases/testcases.c b/tools/testing/selftests/arm64/signal/testcases/testcases.c index 9f580b55b388..1cd124732be4 100644 --- a/tools/testing/selftests/arm64/signal/testcases/testcases.c +++ b/tools/testing/selftests/arm64/signal/testcases/testcases.c @@ -209,6 +209,13 @@ bool validate_reserved(ucontext_t *uc, size_t resv_sz, char **err) zt = (struct zt_context *)head; new_flags |= ZT_CTX; break; + case GCS_MAGIC: + if (flags & GCS_CTX) + *err = "Multiple GCS_MAGIC"; + if (head->size != sizeof(struct gcs_context)) + *err = "Bad size for gcs_context"; + new_flags |= GCS_CTX; + break; case EXTRA_MAGIC: if (flags & EXTRA_CTX) *err = "Multiple EXTRA_MAGIC"; diff --git a/tools/testing/selftests/arm64/signal/testcases/testcases.h b/tools/testing/selftests/arm64/signal/testcases/testcases.h index a08ab0d6207a..9b2599745c29 100644 --- a/tools/testing/selftests/arm64/signal/testcases/testcases.h +++ b/tools/testing/selftests/arm64/signal/testcases/testcases.h @@ -19,6 +19,7 @@ #define ZA_CTX (1 << 2) #define EXTRA_CTX (1 << 3) #define ZT_CTX (1 << 4) +#define GCS_CTX (1 << 5) #define KSFT_BAD_MAGIC 0xdeadbeef From patchwork Mon Oct 9 12:09:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29D7AE95A8E for ; Mon, 9 Oct 2023 12:21:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346556AbjJIMVr (ORCPT ); Mon, 9 Oct 2023 08:21:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346733AbjJIMVR (ORCPT ); Mon, 9 Oct 2023 08:21:17 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18A821700; Mon, 9 Oct 2023 05:14:14 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2050BC433CA; Mon, 9 Oct 2023 12:14:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853653; bh=JCh6BoCIOxePO8zZIr/FO/CERnc7ZRYHEz9dpM/Qpg4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QqdXYEHvz3FTUT3Np9Qa0P6bHjgBrWxF6k+IAbQRqvtWpL8f6b+VeSy4m8x2oqy71 ADwkVHK0JFtJgVJ5MDSiHXkPXaq+hz4ZoRNtnQckXdlxuenDWIAYNqzUj2udq8xmWv oqmTU5Xq0li9wHBGqSBQ9sbmY68Wy9ELc94DNATmv5qg1tuxaqe14CnmERaNXDshN3 YGMpDNvjMxKOu7vCPbagRBZXjN7AxL5vq6Dr75uspEg6TvefipEPwHbAiexRXLz0us 4piL/W6e/otK64TWJnoDDkE96YS9XRVE2WxPCrqYCLcrgnmfnfelhep/y3yXK/EPoy GOyBlVw4eSQGw== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:05 +0100 Subject: [PATCH v6 31/38] kselftest/arm64: Allow signals tests to specify an expected si_code MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-31-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=2627; i=broonie@kernel.org; h=from:subject:message-id; bh=JCh6BoCIOxePO8zZIr/FO/CERnc7ZRYHEz9dpM/Qpg4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2pduXLA8Gjeg0HRP3GD5uX7WzcCSjwA/7BD3Ya AfDkNXOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtqQAKCRAk1otyXVSH0A4rB/ 9GhG9VGix0tl/HuVXdlcFfNVXuRxD5QJ/rim/BQuE9TdegqTB4Wa2IPeR/UkMT+d56/JEjvZOzEQ5M mvpXoA6JburaUxhI8T0E4Yc+9WN6ZN72KU53GP7ixO51ZHtnuzYJ+axiuZMHjHJmnOvyTrHsuaFlRa pKqWLEU8GFt7IoIyqmpj21zg4HhmbtMQ7IWGy+3YdPAj5r4hLtJBHVPBhZkLoMRt+ZoNg2YUxhLDvI KEnc7LR1GkFwfDUfbZBnLtErAFDiafdmn1l/24451Gi4FrVLEenZJhWKLdQUG86uJWwoG4Ni//UTnF eICANpkRccnJT0BR2Nuleg4KrbUqsy X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Currently we ignore si_code unless the expected signal is a SIGSEGV, in which case we enforce it being SEGV_ACCERR. Allow test cases to specify exactly which si_code should be generated so we can validate this, and test for other segfault codes. Signed-off-by: Mark Brown --- .../testing/selftests/arm64/signal/test_signals.h | 4 +++ .../selftests/arm64/signal/test_signals_utils.c | 29 ++++++++++++++-------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 7ada43688c02..ee75a2c25ce7 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -71,6 +71,10 @@ struct tdescr { * Zero when no signal is expected on success */ int sig_ok; + /* + * expected si_code for sig_ok, or 0 to not check + */ + int sig_ok_code; /* signum expected on unsupported CPU features. */ int sig_unsupp; /* a timeout in second for test completion */ diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 89ef95c1af0e..63deca32b0df 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -143,16 +143,25 @@ static bool handle_signal_ok(struct tdescr *td, "current->token ZEROED...test is probably broken!\n"); abort(); } - /* - * Trying to narrow down the SEGV to the ones generated by Kernel itself - * via arm64_notify_segfault(). This is a best-effort check anyway, and - * the si_code check may need to change if this aspect of the kernel - * ABI changes. - */ - if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { - fprintf(stdout, - "si_code != SEGV_ACCERR...test is probably broken!\n"); - abort(); + if (td->sig_ok_code) { + if (si->si_code != td->sig_ok_code) { + fprintf(stdout, "si_code is %d not %d\n", + si->si_code, td->sig_ok_code); + abort(); + } + } else { + /* + * Trying to narrow down the SEGV to the ones + * generated by Kernel itself via + * arm64_notify_segfault(). This is a best-effort + * check anyway, and the si_code check may need to + * change if this aspect of the kernel ABI changes. + */ + if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { + fprintf(stdout, + "si_code != SEGV_ACCERR...test is probably broken!\n"); + abort(); + } } td->pass = 1; /* From patchwork Mon Oct 9 12:09:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71A1AE95A91 for ; Mon, 9 Oct 2023 12:21:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376323AbjJIMVw (ORCPT ); Mon, 9 Oct 2023 08:21:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346444AbjJIMVU (ORCPT ); Mon, 9 Oct 2023 08:21:20 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FA011FE8; Mon, 9 Oct 2023 05:14:20 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3630C433CD; Mon, 9 Oct 2023 12:14:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853660; bh=OYaAyRMLSn6kTs6FMYnMtTS8W24+WIO4Ly8HwzP8d7o=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=aRNZd3pInrVeMdKBw7H6QrC9eqFo/kLOrdLcZwp6V+ul7HAC9gLTUyM1GwvLQB1xc 63PWfOonUjUAaZfItFjlrBSYj0C1UcPcMcnTdWr5qP0HwSHUCIy9tFx5enWS0I8iGd wjWnfqGYjjOFM/9uuXGnODB0Zd9KJOmfqfQ/gBUBt2IPK/IELuF8BE8/T0iAWJl0RN mvrxsrT3AkWz/G0c38Ss4zwSuGHoGW3LdHsPvzWcdBrnOND7PG6AsRZFb6C77RTVUz GIOt7HzNDVv+43GI4Uj9im0YHhm9U7UGFLKfwCWJ1Im7sMe7ZhKQk6x5XnqFgvUaas Y9JWtr+JTJzBw== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:06 +0100 Subject: [PATCH v6 32/38] kselftest/arm64: Always run signals tests with GCS enabled MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-32-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=3590; i=broonie@kernel.org; h=from:subject:message-id; bh=OYaAyRMLSn6kTs6FMYnMtTS8W24+WIO4Ly8HwzP8d7o=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2q4BjYkOhejyoKKvxOrxbGdTheXJOcb2eROkLO TpWR1lWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtqgAKCRAk1otyXVSH0JmAB/ sF/bW4NV1xIXh7KwGzExoXmOWHnZry7NDDHROu0qS/35UQvrwfrBi6tTQNyeIn3v23ghEC2izYQe5/ xF4AI4iosNEQXMNfGRX3CemNMDMILSqO4k6FKtNbqu+VFvTfUZwG8xYNORtZ45iW3tBPk3TCpX09M4 XfPi8G9lx+Y0oWjPDQNtH0cOJA2ummMCDEBK2GKpN6raQ0++tWx80btDNgehnvMePinSWSnYZcbrpO 4YNGY4kE3Tbh20tWr65zqBWE7rqLO6TTsxbHF4VghVQs3K5htXUqYipLcCJk3YKHM8IM9rPSG4lWz8 x7N6aOifCEoSoXFiW/0gq1RcqyV0xO X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Since it is not possible to return from the function that enabled GCS without disabling GCS it is very inconvenient to use the signal handling tests to cover GCS when GCS is not enabled by the toolchain and runtime, something that no current distribution does. Since none of the testcases do anything with stacks that would cause problems with GCS we can sidestep this issue by unconditionally enabling GCS on startup and exiting with a call to exit() rather than a return from main(). Signed-off-by: Mark Brown --- .../testing/selftests/arm64/signal/test_signals.c | 17 ++++++++++++- .../selftests/arm64/signal/test_signals_utils.h | 29 ++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals.c b/tools/testing/selftests/arm64/signal/test_signals.c index 00051b40d71e..30e95f50db19 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.c +++ b/tools/testing/selftests/arm64/signal/test_signals.c @@ -7,6 +7,10 @@ * Each test provides its own tde struct tdescr descriptor to link with * this wrapper. Framework provides common helpers. */ + +#include +#include + #include #include "test_signals.h" @@ -16,6 +20,16 @@ struct tdescr *current = &tde; int main(int argc, char *argv[]) { + /* + * Ensure GCS is at least enabled throughout the tests if + * supported, otherwise the inability to return from the + * function that enabled GCS makes it very inconvenient to set + * up test cases. The prctl() may fail if GCS was locked by + * libc setup code. + */ + if (getauxval(AT_HWCAP2) & HWCAP2_GCS) + gcs_set_state(PR_SHADOW_STACK_ENABLE); + ksft_print_msg("%s :: %s\n", current->name, current->descr); if (test_setup(current) && test_init(current)) { test_run(current); @@ -23,5 +37,6 @@ int main(int argc, char *argv[]) } test_result(current); - return current->result; + /* Do not return in case GCS was enabled */ + exit(current->result); } diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.h b/tools/testing/selftests/arm64/signal/test_signals_utils.h index 762c8fe9c54a..1e80808ee105 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.h +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.h @@ -18,6 +18,35 @@ void test_cleanup(struct tdescr *td); int test_run(struct tdescr *td); void test_result(struct tdescr *td); +#ifndef __NR_prctl +#define __NR_prctl 167 +#endif + +/* + * The prctl takes 1 argument but we need to ensure that the other + * values passed in registers to the syscall are zero since the kernel + * validates them. + */ +#define gcs_set_state(state) \ + ({ \ + register long _num __asm__ ("x8") = __NR_prctl; \ + register long _arg1 __asm__ ("x0") = PR_SET_SHADOW_STACK_STATUS; \ + register long _arg2 __asm__ ("x1") = (long)(state); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ + }) + static inline bool feats_ok(struct tdescr *td) { if (td->feats_incompatible & td->feats_supported) From patchwork Mon Oct 9 12:09:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 443FFE95A97 for ; Mon, 9 Oct 2023 12:17:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346612AbjJIMRL (ORCPT ); Mon, 9 Oct 2023 08:17:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346564AbjJIMQ4 (ORCPT ); Mon, 9 Oct 2023 08:16:56 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C7318F; Mon, 9 Oct 2023 05:14:34 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 299B9C433AB; Mon, 9 Oct 2023 12:14:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853673; bh=GoX5+5TPWKMBeVX5ln3BsEwrTT7WlIZGkNrZpsUPmck=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=W0FSwNyqYZWNsOY3/ccw25GF3rzgDwnMk40GARdg5nA7BmNVFjlfiXp5Ppxrd+fL9 sKEF2cdqC8DpxBGQBRLxAAroWLmgJfjBDU8b3m1eWfhjOE3b/VAAbMW1jLl3d89wfi 2d/8HJPzISmQNXLyW2ZcbMYKqPGzq/OFlcI2XtNilb1m3bCCUCb6OA8G30F/sgKiU2 bpexuyb6Rt/imFxXl3SdUbvph5Rytpw7AuQV1CcftoZTiQD/tO8j62Y7kni0ad6vXI wCHeXD4dkJA/JXWHWgCa2QKEQepJGl2ED/cEJFNDvW7MDps9OI016UESOJf+gNUkld o2uMVrd6B3FnQ== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:08 +0100 Subject: [PATCH v6 34/38] kselftest/arm64: Add a GCS test program built with the system libc MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-34-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=21534; i=broonie@kernel.org; h=from:subject:message-id; bh=GoX5+5TPWKMBeVX5ln3BsEwrTT7WlIZGkNrZpsUPmck=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2s3ZxHwCqqzhSo1a3a58tdnP9PUmH0mPnDeUuv PgKtKHCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtrAAKCRAk1otyXVSH0CIwB/ 9tWDmxl4OO+XrU+I3nLBXtcmPzmTHXbJyfrSkHIqAznge5tlKUC2VRqVbHtWhV4pvFmBpFddBOpTa4 Go8XyJylvmCfywK+UqxvkBJIYZfhy+FAocIB3jdamcPfsxXcrtYdsCEomj34AFBnhTJTp+kKUsqinE ptfcFzJyUF/omVPeFj4qlELtWgDteGIsd1c4TBkASJvrtEm8PrwEewthHZMhW1+E7WWYAzQzUk6wWl Ca4KmdvuA11aUmZSiMLP5CHvxc5I07QoRv7fwB50LpgIM3vM6wwElmbcDvTBKidjoOAk0cQbVWYoia q9fIBxsU82/nXfsY5mCf+/iQ6e8dM+ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org There are things like threads which nolibc struggles with which we want to add coverage for, and the ABI allows us to test most of these even if libc itself does not understand GCS so add a test application built using the system libc. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 4 +- tools/testing/selftests/arm64/gcs/gcs-util.h | 10 + tools/testing/selftests/arm64/gcs/libc-gcs.c | 742 +++++++++++++++++++++++++++ 4 files changed, 756 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 0e5e695ecba5..5810c4a163d4 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1 +1,2 @@ basic-gcs +libc-gcs diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index 61a30f483429..a8fdf21e9a47 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,9 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs +TEST_GEN_PROGS := basic-gcs libc-gcs + +LDLIBS+=-lpthread include ../../lib.mk diff --git a/tools/testing/selftests/arm64/gcs/gcs-util.h b/tools/testing/selftests/arm64/gcs/gcs-util.h index 65e6de583506..fd50d0523558 100644 --- a/tools/testing/selftests/arm64/gcs/gcs-util.h +++ b/tools/testing/selftests/arm64/gcs/gcs-util.h @@ -16,6 +16,16 @@ #define __NR_prctl 167 #endif +#ifndef NT_ARM_GCS +#define NT_ARM_GCS 0x40e + +struct user_gcs { + __u64 features_enabled; + __u64 features_locked; + __u64 gcspr_el0; +}; +#endif + /* Shadow Stack/Guarded Control Stack interface */ #define PR_GET_SHADOW_STACK_STATUS 71 #define PR_SET_SHADOW_STACK_STATUS 72 diff --git a/tools/testing/selftests/arm64/gcs/libc-gcs.c b/tools/testing/selftests/arm64/gcs/libc-gcs.c new file mode 100644 index 000000000000..58141d46f861 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/libc-gcs.c @@ -0,0 +1,742 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + */ + +#define _GNU_SOURCE + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include "kselftest_harness.h" + +#include "gcs-util.h" + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +static noinline void gcs_recurse(int depth) +{ + if (depth) + gcs_recurse(depth - 1); + + /* Prevent tail call optimization so we actually recurse */ + asm volatile("dsb sy" : : : "memory"); +} + +/* Smoke test that a function call and return works*/ +TEST(can_call_function) +{ + gcs_recurse(0); +} + +static void *gcs_test_thread(void *arg) +{ + int ret; + unsigned long mode; + + /* + * Some libcs don't seem to fill unused arguments with 0 but + * the kernel validates this so we supply all 5 arguments. + */ + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("PR_GET_SHADOW_STACK_STATUS failed: %d\n", ret); + return NULL; + } + + if (!(mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in thread, mode is %u\n", + mode); + return NULL; + } + + /* Just in case... */ + gcs_recurse(0); + + /* Use a non-NULL value to indicate a pass */ + return &gcs_test_thread; +} + +/* Verify that if we start a new thread it has GCS enabled */ +TEST(gcs_enabled_thread) +{ + pthread_t thread; + void *thread_ret; + int ret; + + ret = pthread_create(&thread, NULL, gcs_test_thread, NULL); + ASSERT_TRUE(ret == 0); + if (ret != 0) + return; + + ret = pthread_join(thread, &thread_ret); + ASSERT_TRUE(ret == 0); + if (ret != 0) + return; + + ASSERT_TRUE(thread_ret != NULL); +} + +/* Read the GCS until we find the terminator */ +TEST(gcs_find_terminator) +{ + unsigned long *gcs, *cur; + + gcs = get_gcspr(); + cur = gcs; + while (*cur) + cur++; + + ksft_print_msg("GCS in use from %p-%p\n", gcs, cur); + + /* + * We should have at least whatever called into this test so + * the two pointer should differ. + */ + ASSERT_TRUE(gcs != cur); +} + +/* + * We can access a GCS via ptrace + * + * This could usefully have a fixture but note that each test is + * fork()ed into a new child whcih causes issues. Might be better to + * lift at least some of this out into a separate, non-harness, test + * program. + */ +TEST(ptrace_read_write) +{ + pid_t child, pid; + int ret, status; + siginfo_t si; + uint64_t val, rval, gcspr; + struct user_gcs child_gcs; + struct iovec iov, local_iov, remote_iov; + + child = fork(); + if (child == -1) { + ksft_print_msg("fork() failed: %d (%s)\n", + errno, strerror(errno)); + ASSERT_NE(child, -1); + } + + if (child == 0) { + /* + * In child, make sure there's something on the stack and + * ask to be traced. + */ + gcs_recurse(0); + if (ptrace(PTRACE_TRACEME, -1, NULL, NULL)) + ksft_exit_fail_msg("PTRACE_TRACEME", strerror(errno)); + + if (raise(SIGSTOP)) + ksft_exit_fail_msg("raise(SIGSTOP)", strerror(errno)); + + return; + } + + ksft_print_msg("Child: %d\n", child); + + /* Attach to the child */ + while (1) { + int sig; + + pid = wait(&status); + if (pid == -1) { + ksft_print_msg("wait() failed: %s", + strerror(errno)); + goto error; + } + + /* + * This should never happen but it's hard to flag in + * the framework. + */ + if (pid != child) + continue; + + if (WIFEXITED(status) || WIFSIGNALED(status)) + ksft_exit_fail_msg("Child died unexpectedly\n"); + + if (!WIFSTOPPED(status)) + goto error; + + sig = WSTOPSIG(status); + + if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &si)) { + if (errno == ESRCH) { + ASSERT_NE(errno, ESRCH); + return; + } + + if (errno == EINVAL) { + sig = 0; /* bust group-stop */ + goto cont; + } + + ksft_print_msg("PTRACE_GETSIGINFO: %s\n", + strerror(errno)); + goto error; + } + + if (sig == SIGSTOP && si.si_code == SI_TKILL && + si.si_pid == pid) + break; + + cont: + if (ptrace(PTRACE_CONT, pid, NULL, sig)) { + if (errno == ESRCH) { + ASSERT_NE(errno, ESRCH); + return; + } + + ksft_print_msg("PTRACE_CONT: %s\n", strerror(errno)); + goto error; + } + } + + /* Where is the child GCS? */ + iov.iov_base = &child_gcs; + iov.iov_len = sizeof(child_gcs); + ret = ptrace(PTRACE_GETREGSET, child, NT_ARM_GCS, &iov); + if (ret != 0) { + ksft_print_msg("Failed to read child GCS state: %s (%d)\n", + strerror(errno), errno); + goto error; + } + + /* We should have inherited GCS over fork(), confirm */ + if (!(child_gcs.features_enabled & PR_SHADOW_STACK_ENABLE)) { + ASSERT_TRUE(child_gcs.features_enabled & + PR_SHADOW_STACK_ENABLE); + goto error; + } + + gcspr = child_gcs.gcspr_el0; + ksft_print_msg("Child GCSPR 0x%lx, flags %x, locked %x\n", + gcspr, child_gcs.features_enabled, + child_gcs.features_locked); + + /* Ideally we'd cross check with the child memory map */ + + errno = 0; + val = ptrace(PTRACE_PEEKDATA, child, (void *)gcspr, NULL); + ret = errno; + if (ret != 0) + ksft_print_msg("PTRACE_PEEKDATA failed: %s (%d)\n", + strerror(ret), ret); + EXPECT_EQ(ret, 0); + + /* The child should be in a function, the GCSPR shouldn't be 0 */ + EXPECT_NE(val, 0); + + /* Same thing via process_vm_readv() */ + local_iov.iov_base = &rval; + local_iov.iov_len = sizeof(rval); + remote_iov.iov_base = (void *)gcspr; + remote_iov.iov_len = sizeof(rval); + ret = process_vm_writev(child, &local_iov, 1, &remote_iov, 1, 0); + if (ret == -1) + ksft_print_msg("process_vm_readv() failed: %s (%d)\n", + strerror(errno), errno); + EXPECT_EQ(ret, sizeof(rval)); + EXPECT_EQ(val, rval); + + /* Write data via a peek */ + ret = ptrace(PTRACE_POKEDATA, child, (void *)gcspr, NULL); + if (ret == -1) + ksft_print_msg("PTRACE_POKEDATA failed: %s (%d)\n", + strerror(errno), errno); + EXPECT_EQ(ret, 0); + EXPECT_EQ(0, ptrace(PTRACE_PEEKDATA, child, (void *)gcspr, NULL)); + + /* Restore what we had before, this time with process_vm_writev() */ + local_iov.iov_base = &rval; + local_iov.iov_len = sizeof(val); + remote_iov.iov_base = (void *)gcspr; + remote_iov.iov_len = sizeof(rval); + ret = process_vm_readv(child, &local_iov, 1, &remote_iov, 1, 0); + if (ret == -1) + ksft_print_msg("process_vm_writev() failed: %s (%d)\n", + strerror(errno), errno); + EXPECT_EQ(ret, sizeof(rval)); + + EXPECT_EQ(val, ptrace(PTRACE_PEEKDATA, child, (void *)gcspr, NULL)); + + /* That's all, folks */ + kill(child, SIGKILL); + return; + +error: + kill(child, SIGKILL); + ASSERT_FALSE(true); +} + +FIXTURE(map_gcs) +{ + unsigned long *stack; +}; + +FIXTURE_VARIANT(map_gcs) +{ + size_t stack_size; + unsigned long flags; +}; + +FIXTURE_VARIANT_ADD(map_gcs, s2k_cap_marker) +{ + .stack_size = 2 * 1024, + .flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s2k_cap) +{ + .stack_size = 2 * 1024, + .flags = SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s2k_marker) +{ + .stack_size = 2 * 1024, + .flags = SHADOW_STACK_SET_MARKER, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s2k) +{ + .stack_size = 2 * 1024, + .flags = 0, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s4k_cap_marker) +{ + .stack_size = 4 * 1024, + .flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s4k_cap) +{ + .stack_size = 4 * 1024, + .flags = SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s3k_marker) +{ + .stack_size = 4 * 1024, + .flags = SHADOW_STACK_SET_MARKER, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s4k) +{ + .stack_size = 4 * 1024, + .flags = 0, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s16k_cap_marker) +{ + .stack_size = 16 * 1024, + .flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s16k_cap) +{ + .stack_size = 16 * 1024, + .flags = SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s16k_marker) +{ + .stack_size = 16 * 1024, + .flags = SHADOW_STACK_SET_MARKER, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s16k) +{ + .stack_size = 16 * 1024, + .flags = 0, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s64k_cap_marker) +{ + .stack_size = 64 * 1024, + .flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s64k_cap) +{ + .stack_size = 64 * 1024, + .flags = SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s64k_marker) +{ + .stack_size = 64 * 1024, + .flags = SHADOW_STACK_SET_MARKER, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s64k) +{ + .stack_size = 64 * 1024, + .flags = 0, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s128k_cap_marker) +{ + .stack_size = 128 * 1024, + .flags = SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s128k_cap) +{ + .stack_size = 128 * 1024, + .flags = SHADOW_STACK_SET_TOKEN, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s128k_marker) +{ + .stack_size = 128 * 1024, + .flags = SHADOW_STACK_SET_MARKER, +}; + +FIXTURE_VARIANT_ADD(map_gcs, s128k) +{ + .stack_size = 128 * 1024, + .flags = 0, +}; + +FIXTURE_SETUP(map_gcs) +{ + self->stack = (void *)syscall(__NR_map_shadow_stack, 0, + variant->stack_size, + variant->flags); + ASSERT_FALSE(self->stack == MAP_FAILED); + ksft_print_msg("Allocated stack from %p-%p\n", self->stack, + (unsigned long)self->stack + variant->stack_size); +} + +FIXTURE_TEARDOWN(map_gcs) +{ + int ret; + + if (self->stack != MAP_FAILED) { + ret = munmap(self->stack, variant->stack_size); + ASSERT_EQ(ret, 0); + } +} + +/* The stack has a cap token */ +TEST_F(map_gcs, stack_capped) +{ + unsigned long *stack = self->stack; + size_t cap_index; + + cap_index = (variant->stack_size / sizeof(unsigned long)); + + switch (variant->flags & (SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN)) { + case SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN: + cap_index -= 2; + break; + case SHADOW_STACK_SET_TOKEN: + cap_index -= 1; + break; + case SHADOW_STACK_SET_MARKER: + case 0: + /* No cap, no test */ + return; + } + + ASSERT_EQ(stack[cap_index], GCS_CAP(&stack[cap_index])); +} + +/* The top of the stack is 0 */ +TEST_F(map_gcs, stack_terminated) +{ + unsigned long *stack = self->stack; + size_t term_index; + + if (!(variant->flags & SHADOW_STACK_SET_MARKER)) + return; + + term_index = (variant->stack_size / sizeof(unsigned long)) - 1; + + ASSERT_EQ(stack[term_index], 0); +} + +/* Writes should fault */ +TEST_F_SIGNAL(map_gcs, not_writeable, SIGSEGV) +{ + self->stack[0] = 0; +} + +/* Put it all together, we can safely switch to and from the stack */ +TEST_F(map_gcs, stack_switch) +{ + size_t cap_index; + cap_index = (variant->stack_size / sizeof(unsigned long)); + unsigned long *orig_gcspr_el0, *pivot_gcspr_el0; + + /* Skip over the stack terminator and point at the cap */ + switch (variant->flags & (SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN)) { + case SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN: + cap_index -= 2; + break; + case SHADOW_STACK_SET_TOKEN: + cap_index -= 1; + break; + case SHADOW_STACK_SET_MARKER: + case 0: + /* No cap, no test */ + return; + } + pivot_gcspr_el0 = &self->stack[cap_index]; + + /* Pivot to the new GCS */ + ksft_print_msg("Pivoting to %p from %p, target has value 0x%lx\n", + pivot_gcspr_el0, get_gcspr(), + *pivot_gcspr_el0); + gcsss1(pivot_gcspr_el0); + orig_gcspr_el0 = gcsss2(); + ksft_print_msg("Pivoted to %p from %p, target has value 0x%lx\n", + pivot_gcspr_el0, get_gcspr(), + *pivot_gcspr_el0); + + ksft_print_msg("Pivoted, GCSPR_EL0 now %p\n", get_gcspr()); + + /* New GCS must be in the new buffer */ + ASSERT_TRUE((unsigned long)get_gcspr() > (unsigned long)self->stack); + ASSERT_TRUE((unsigned long)get_gcspr() <= + (unsigned long)self->stack + variant->stack_size); + + /* We should be able to use all but 2 slots of the new stack */ + ksft_print_msg("Recursing %d levels\n", cap_index - 1); + gcs_recurse(cap_index - 1); + + /* Pivot back to the original GCS */ + gcsss1(orig_gcspr_el0); + pivot_gcspr_el0 = gcsss2(); + + gcs_recurse(0); + ksft_print_msg("Pivoted back to GCSPR_EL0 0x%lx\n", get_gcspr()); +} + +/* We fault if we try to go beyond the end of the stack */ +TEST_F_SIGNAL(map_gcs, stack_overflow, SIGSEGV) +{ + size_t cap_index; + cap_index = (variant->stack_size / sizeof(unsigned long)); + unsigned long *orig_gcspr_el0, *pivot_gcspr_el0; + + /* Skip over the stack terminator and point at the cap */ + switch (variant->flags & (SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN)) { + case SHADOW_STACK_SET_MARKER | SHADOW_STACK_SET_TOKEN: + cap_index -= 2; + break; + case SHADOW_STACK_SET_TOKEN: + cap_index -= 1; + break; + case SHADOW_STACK_SET_MARKER: + case 0: + /* No cap, no test but we need to SEGV to avoid a false fail */ + orig_gcspr_el0 = get_gcspr(); + *orig_gcspr_el0 = 0; + return; + } + pivot_gcspr_el0 = &self->stack[cap_index]; + + /* Pivot to the new GCS */ + ksft_print_msg("Pivoting to %p from %p, target has value 0x%lx\n", + pivot_gcspr_el0, get_gcspr(), + *pivot_gcspr_el0); + gcsss1(pivot_gcspr_el0); + orig_gcspr_el0 = gcsss2(); + ksft_print_msg("Pivoted to %p from %p, target has value 0x%lx\n", + pivot_gcspr_el0, get_gcspr(), + *pivot_gcspr_el0); + + ksft_print_msg("Pivoted, GCSPR_EL0 now %p\n", get_gcspr()); + + /* New GCS must be in the new buffer */ + ASSERT_TRUE((unsigned long)get_gcspr() > (unsigned long)self->stack); + ASSERT_TRUE((unsigned long)get_gcspr() <= + (unsigned long)self->stack + variant->stack_size); + + /* Now try to recurse, we should fault doing this. */ + ksft_print_msg("Recursing %d levels...\n", cap_index + 1); + gcs_recurse(cap_index + 1); + ksft_print_msg("...done\n"); + + /* Clean up properly to try to guard against spurious passes. */ + gcsss1(orig_gcspr_el0); + pivot_gcspr_el0 = gcsss2(); + ksft_print_msg("Pivoted back to GCSPR_EL0 0x%lx\n", get_gcspr()); +} + +FIXTURE(map_invalid_gcs) +{ +}; + +FIXTURE_VARIANT(map_invalid_gcs) +{ + size_t stack_size; +}; + +FIXTURE_SETUP(map_invalid_gcs) +{ +} + +FIXTURE_TEARDOWN(map_invalid_gcs) +{ +} + +/* GCS must be larger than 16 bytes */ +FIXTURE_VARIANT_ADD(map_invalid_gcs, too_small) +{ + .stack_size = 8, +}; + +/* GCS size must be 16 byte aligned */ +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_1) { .stack_size = 1024 + 1 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_2) { .stack_size = 1024 + 2 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_3) { .stack_size = 1024 + 3 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_4) { .stack_size = 1024 + 4 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_5) { .stack_size = 1024 + 5 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_6) { .stack_size = 1024 + 6 }; +FIXTURE_VARIANT_ADD(map_invalid_gcs, unligned_7) { .stack_size = 1024 + 7 }; + +TEST_F(map_invalid_gcs, do_map) +{ + void *stack; + + stack = (void *)syscall(__NR_map_shadow_stack, 0, + variant->stack_size, 0); + ASSERT_TRUE(stack == MAP_FAILED); + if (stack != MAP_FAILED) + munmap(stack, variant->stack_size); +} + +FIXTURE(invalid_mprotect) +{ + unsigned long *stack; + size_t stack_size; +}; + +FIXTURE_VARIANT(invalid_mprotect) +{ + unsigned long flags; +}; + +FIXTURE_SETUP(invalid_mprotect) +{ + self->stack_size = sysconf(_SC_PAGE_SIZE); + self->stack = (void *)syscall(__NR_map_shadow_stack, 0, + self->stack_size, 0); + ASSERT_FALSE(self->stack == MAP_FAILED); + ksft_print_msg("Allocated stack from %p-%p\n", self->stack, + (unsigned long)self->stack + self->stack_size); +} + +FIXTURE_TEARDOWN(invalid_mprotect) +{ + int ret; + + if (self->stack != MAP_FAILED) { + ret = munmap(self->stack, self->stack_size); + ASSERT_EQ(ret, 0); + } +} + +FIXTURE_VARIANT_ADD(invalid_mprotect, exec) +{ + .flags = PROT_EXEC, +}; + +FIXTURE_VARIANT_ADD(invalid_mprotect, bti) +{ + .flags = PROT_BTI, +}; + +FIXTURE_VARIANT_ADD(invalid_mprotect, exec_bti) +{ + .flags = PROT_EXEC | PROT_BTI, +}; + +TEST_F(invalid_mprotect, do_map) +{ + int ret; + + ret = mprotect(self->stack, self->stack_size, variant->flags); + ASSERT_EQ(ret, -1); +} + +TEST_F(invalid_mprotect, do_map_read) +{ + int ret; + + ret = mprotect(self->stack, self->stack_size, + variant->flags | PROT_READ); + ASSERT_EQ(ret, -1); +} + +int main(int argc, char **argv) +{ + unsigned long gcs_mode; + int ret; + + if (!(getauxval(AT_HWCAP2) & HWCAP2_GCS)) + ksft_exit_skip("SKIP GCS not supported\n"); + + /* + * Force shadow stacks on, our tests *should* be fine with or + * without libc support and with or without this having ended + * up tagged for GCS and enabled by the dynamic linker. We + * can't use the libc prctl() function since we can't return + * from enabling the stack. Also lock GCS if not already + * locked so we can test behaviour when it's locked. + */ + ret = my_syscall2(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, &gcs_mode); + if (ret) { + ksft_print_msg("Failed to read GCS state: %d\n", ret); + return EXIT_FAILURE; + } + + if (!(gcs_mode & PR_SHADOW_STACK_ENABLE)) { + gcs_mode = PR_SHADOW_STACK_ENABLE; + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + gcs_mode); + if (ret) { + ksft_print_msg("Failed to configure GCS: %d\n", ret); + return EXIT_FAILURE; + } + } + + /* Avoid returning in case libc doesn't understand GCS */ + exit(test_harness_run(argc, argv)); +} From patchwork Mon Oct 9 12:09:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41E37E95A91 for ; Mon, 9 Oct 2023 12:17:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346673AbjJIMRu (ORCPT ); Mon, 9 Oct 2023 08:17:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346678AbjJIMR3 (ORCPT ); Mon, 9 Oct 2023 08:17:29 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 545021996; Mon, 9 Oct 2023 05:14:47 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87C14C433C8; Mon, 9 Oct 2023 12:14:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853686; bh=n0brHaZlsdiu+vZdVacJ8xeue5gjh150PYMG3rRf1kA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=n5UQQ1AT5I82mU7Nx0+Y9k7xNEgSigXUYhr5lH2zAhdzjpcwZ8tovFxITNmUjsyzB lgWEwU/CIX2nK95j1QKfUTqy0dVyqg7ab/35dbMkWUEBFAzXyYkdzNxLeOEEO+Fbso nLY+fU7CxvXItc8JKRQgE6SH+um7qVxQZuEoMM8oELlDrauEECyhoRvdyo0SiroXCa EmYSuyUki5zAvADxN6aIvmHDtXnQAyZjEE/apK221patQzIMUW6FRzPZJr+gCFOTv1 pQAtS4zl+B8/9rPsm3VpVMIpN5cBdf1Soigrmu4XTQeb5edI/V7i8RapIBuF8JFcA7 6a3j+oCTK34wA== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:10 +0100 Subject: [PATCH v6 36/38] selftests/arm64: Add GCS signal tests MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-36-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=7315; i=broonie@kernel.org; h=from:subject:message-id; bh=n0brHaZlsdiu+vZdVacJ8xeue5gjh150PYMG3rRf1kA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2tveb6LcwYU7gt5D9M1Yzq33Su1fslnEnmUQoy SjnyNBGJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtrQAKCRAk1otyXVSH0EV1B/ 9focZ0RCun/nSyD/YCRi05xh3aUeAY3eqz5BL3aQGBPQgmMG3yHfYyM6vm9FrA2pXjlP99T2v3wy+1 kVSVboJJMmImQFHJJn16MuZQzmFX8w4XVHQurQLPuKpWh3PjT7OjDwcblT15zbd87OB//9A9aLUs/v uEJFElHcaq+Ug126uPaYzJQ+fc1ISuIxC3t1jZP/6LjjxGJgeSz0nu4MK5g6tx7Cw+zY7NHlPeHGkl ZQH+jGU/3G9WZtRyDcA3garOiz/TAtphRjFPKloSfcvdRvlYt/ecDyi+IkkNXH3wD46lJnZWzOJ2L4 AhEUBi+nK7qZI8QOb8EVMOXU6wg/2M X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Do some testing of the signal handling for GCS, checking that a GCS frame has the expected information in it and that the expected signals are delivered with invalid operations. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/signal/.gitignore | 1 + .../selftests/arm64/signal/test_signals_utils.h | 10 +++ .../arm64/signal/testcases/gcs_exception_fault.c | 59 ++++++++++++++++ .../selftests/arm64/signal/testcases/gcs_frame.c | 78 ++++++++++++++++++++++ .../arm64/signal/testcases/gcs_write_fault.c | 67 +++++++++++++++++++ 5 files changed, 215 insertions(+) diff --git a/tools/testing/selftests/arm64/signal/.gitignore b/tools/testing/selftests/arm64/signal/.gitignore index 839e3a252629..26de12918890 100644 --- a/tools/testing/selftests/arm64/signal/.gitignore +++ b/tools/testing/selftests/arm64/signal/.gitignore @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only mangle_* fake_sigreturn_* +gcs_* sme_* ssve_* sve_* diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.h b/tools/testing/selftests/arm64/signal/test_signals_utils.h index 1e80808ee105..36fc12b3cd60 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.h +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.h @@ -6,6 +6,7 @@ #include #include +#include #include #include @@ -47,6 +48,15 @@ void test_result(struct tdescr *td); _arg1; \ }) +static inline __attribute__((always_inline)) uint64_t get_gcspr_el0(void) +{ + uint64_t val; + + asm volatile("mrs %0, S3_3_C2_C5_1" : "=r" (val)); + + return val; +} + static inline bool feats_ok(struct tdescr *td) { if (td->feats_incompatible & td->feats_supported) diff --git a/tools/testing/selftests/arm64/signal/testcases/gcs_exception_fault.c b/tools/testing/selftests/arm64/signal/testcases/gcs_exception_fault.c new file mode 100644 index 000000000000..532d533592a1 --- /dev/null +++ b/tools/testing/selftests/arm64/signal/testcases/gcs_exception_fault.c @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2023 ARM Limited + */ + +#include +#include +#include + +#include +#include + +#include "test_signals_utils.h" +#include "testcases.h" + +/* This should be includable from some standard header, but which? */ +#ifndef SEGV_CPERR +#define SEGV_CPERR 10 +#endif + +static inline void gcsss1(uint64_t Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static int gcs_op_fault_trigger(struct tdescr *td) +{ + /* + * The slot below our current GCS should be in a valid GCS but + * must not have a valid cap in it. + */ + gcsss1(get_gcspr_el0() - 8); + + return 0; +} + +static int gcs_op_fault_signal(struct tdescr *td, siginfo_t *si, + ucontext_t *uc) +{ + ASSERT_GOOD_CONTEXT(uc); + + return 1; +} + +struct tdescr tde = { + .name = "Invalid GCS operation", + .descr = "An invalid GCS operation generates the expected signal", + .feats_required = FEAT_GCS, + .timeout = 3, + .sig_ok = SIGSEGV, + .sig_ok_code = SEGV_CPERR, + .sanity_disabled = true, + .trigger = gcs_op_fault_trigger, + .run = gcs_op_fault_signal, +}; diff --git a/tools/testing/selftests/arm64/signal/testcases/gcs_frame.c b/tools/testing/selftests/arm64/signal/testcases/gcs_frame.c new file mode 100644 index 000000000000..d67cb26195a6 --- /dev/null +++ b/tools/testing/selftests/arm64/signal/testcases/gcs_frame.c @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2023 ARM Limited + */ + +#include +#include +#include + +#include "test_signals_utils.h" +#include "testcases.h" + +static union { + ucontext_t uc; + char buf[1024 * 64]; +} context; + +static int gcs_regs(struct tdescr *td, siginfo_t *si, ucontext_t *uc) +{ + size_t offset; + struct _aarch64_ctx *head = GET_BUF_RESV_HEAD(context); + struct gcs_context *gcs; + unsigned long expected, gcspr; + int ret; + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &expected, 0, 0, 0); + if (ret != 0) { + fprintf(stderr, "Unable to query GCS status\n"); + return 1; + } + + /* We expect a cap to be added to the GCS in the signal frame */ + gcspr = get_gcspr_el0(); + gcspr -= 8; + fprintf(stderr, "Expecting GCSPR_EL0 %lx\n", gcspr); + + if (!get_current_context(td, &context.uc, sizeof(context))) { + fprintf(stderr, "Failed getting context\n"); + return 1; + } + fprintf(stderr, "Got context\n"); + + head = get_header(head, GCS_MAGIC, GET_BUF_RESV_SIZE(context), + &offset); + if (!head) { + fprintf(stderr, "No GCS context\n"); + return 1; + } + + gcs = (struct gcs_context *)head; + + /* Basic size validation is done in get_current_context() */ + + if (gcs->features_enabled != expected) { + fprintf(stderr, "Features enabled %llx but expected %lx\n", + gcs->features_enabled, expected); + return 1; + } + + if (gcs->gcspr != gcspr) { + fprintf(stderr, "Got GCSPR %llx but expected %lx\n", + gcs->gcspr, gcspr); + return 1; + } + + fprintf(stderr, "GCS context validated\n"); + td->pass = 1; + + return 0; +} + +struct tdescr tde = { + .name = "GCS basics", + .descr = "Validate a GCS signal context", + .feats_required = FEAT_GCS, + .timeout = 3, + .run = gcs_regs, +}; diff --git a/tools/testing/selftests/arm64/signal/testcases/gcs_write_fault.c b/tools/testing/selftests/arm64/signal/testcases/gcs_write_fault.c new file mode 100644 index 000000000000..126b1a294a29 --- /dev/null +++ b/tools/testing/selftests/arm64/signal/testcases/gcs_write_fault.c @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2023 ARM Limited + */ + +#include +#include +#include + +#include +#include + +#include "test_signals_utils.h" +#include "testcases.h" + +static uint64_t *gcs_page; + +#ifndef __NR_map_shadow_stack +#define __NR_map_shadow_stack 452 +#endif + +static bool alloc_gcs(struct tdescr *td) +{ + long page_size = sysconf(_SC_PAGE_SIZE); + + gcs_page = (void *)syscall(__NR_map_shadow_stack, 0, + page_size, 0); + if (gcs_page == MAP_FAILED) { + fprintf(stderr, "Failed to map %ld byte GCS: %d\n", + page_size, errno); + return false; + } + + return true; +} + +static int gcs_write_fault_trigger(struct tdescr *td) +{ + /* Verify that the page is readable (ie, not completely unmapped) */ + fprintf(stderr, "Read value 0x%lx\n", gcs_page[0]); + + /* A regular write should trigger a fault */ + gcs_page[0] = EINVAL; + + return 0; +} + +static int gcs_write_fault_signal(struct tdescr *td, siginfo_t *si, + ucontext_t *uc) +{ + ASSERT_GOOD_CONTEXT(uc); + + return 1; +} + + +struct tdescr tde = { + .name = "GCS write fault", + .descr = "Normal writes to a GCS segfault", + .feats_required = FEAT_GCS, + .timeout = 3, + .sig_ok = SIGSEGV, + .sanity_disabled = true, + .init = alloc_gcs, + .trigger = gcs_write_fault_trigger, + .run = gcs_write_fault_signal, +}; From patchwork Mon Oct 9 12:09:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 731235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A65EBE95A9B for ; Mon, 9 Oct 2023 12:18:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346500AbjJIMSJ (ORCPT ); Mon, 9 Oct 2023 08:18:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346490AbjJIMRu (ORCPT ); Mon, 9 Oct 2023 08:17:50 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44B412D79; Mon, 9 Oct 2023 05:15:00 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6D4BC433B6; Mon, 9 Oct 2023 12:14:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696853700; bh=o3F0gqwirOzTh8gwGkZMmsXM7Yu/yi+gPnGo/AGnFQM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=H6L2zm3OMsRyXQZfw5aCB2cQkw9G1wWr2dacwbrbjZXZPIvYK/weMVGb/cCzZKiC7 ZWZE+3KXbORqpB/LvrPa2XKaZcYAjAFSMj4QWQFzw3QNTXKDYCgERhCXIRuk6Sjumc zB5fHswdhfFPA4MwUbxsr8ag7wUXWXMvOwCfogAugBxO+lLIGF8fBuDPcfcrWt9bL/ UfwoJNGMVjZIYaA2ZzmhBY6HI7q3jqJtDEajWrgfdPB+xTlJBBPSLdojTFkIx10j12 ZNBSMSXaS2qYh9yW/wlBi/EargZA/4mUtrGrQO91UPNkTiaKE8IffZTX8D3vhoHTR8 OG9JYN3wJLxnw== From: Mark Brown Date: Mon, 09 Oct 2023 13:09:12 +0100 Subject: [PATCH v6 38/38] kselftest/arm64: Enable GCS for the FP stress tests MIME-Version: 1.0 Message-Id: <20231009-arm64-gcs-v6-38-78e55deaa4dd@kernel.org> References: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> In-Reply-To: <20231009-arm64-gcs-v6-0-78e55deaa4dd@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-0438c X-Developer-Signature: v=1; a=openpgp-sha256; l=3085; i=broonie@kernel.org; h=from:subject:message-id; bh=o3F0gqwirOzTh8gwGkZMmsXM7Yu/yi+gPnGo/AGnFQM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlI+2vG7XbuVEpS5pxDaAIrEX013t37tEVAfRK24qD lhEdNXWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZSPtrwAKCRAk1otyXVSH0O9gB/ 9FoGr0eST8WBd+FIsUjhsE3d9wYovTCREYw/+Ayy30NkoVgB2HmjZh7K+hb1YN/LMp4ABI6gQv9m7U CD4WfBwvZ66QhtkrTfCIAGZpIQukSOsbVk6gdZYFnG7YW+e8eW3WDSRw9MC8WvY4S8Se1kCKCSUCeb Qz3siqGYszHBGhCUXtGLknYBqrkRfkRcrLwD25GetvYNDznn9zMYXqk6yRPx0AfSoebviB+WXqPi2L dbp5/fysyysuQlZh6LF/DXv7rc3dApbApo0DQVkQsmIg1LX6vsrhlBVTV2UmFdpn3sqVmIl8CtGlBA iYNaW3vGcLHl9DSvc6WgiC/Kg5uRyg X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org While it's a bit off topic for them the floating point stress tests do give us some coverage of context thrashing cases, and also of active signal delivery separate to the relatively complicated framework in the actual signals tests. Have the tests enable GCS on startup, ignoring failures so they continue to work as before on systems without GCS. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/fp/assembler.h | 15 +++++++++++++++ tools/testing/selftests/arm64/fp/fpsimd-test.S | 2 ++ tools/testing/selftests/arm64/fp/sve-test.S | 2 ++ tools/testing/selftests/arm64/fp/za-test.S | 2 ++ tools/testing/selftests/arm64/fp/zt-test.S | 2 ++ 5 files changed, 23 insertions(+) diff --git a/tools/testing/selftests/arm64/fp/assembler.h b/tools/testing/selftests/arm64/fp/assembler.h index 9b38a0da407d..7012f9f796de 100644 --- a/tools/testing/selftests/arm64/fp/assembler.h +++ b/tools/testing/selftests/arm64/fp/assembler.h @@ -65,4 +65,19 @@ endfunction bl puts .endm +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +.macro enable_gcs + // Run with GCS + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, PR_SHADOW_STACK_ENABLE + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 +.endm + #endif /* ! ASSEMBLER_H */ diff --git a/tools/testing/selftests/arm64/fp/fpsimd-test.S b/tools/testing/selftests/arm64/fp/fpsimd-test.S index 8b960d01ed2e..b16fb7f42e3e 100644 --- a/tools/testing/selftests/arm64/fp/fpsimd-test.S +++ b/tools/testing/selftests/arm64/fp/fpsimd-test.S @@ -215,6 +215,8 @@ endfunction // Main program entry point .globl _start function _start + enable_gcs + mov x23, #0 // signal count mov w0, #SIGINT diff --git a/tools/testing/selftests/arm64/fp/sve-test.S b/tools/testing/selftests/arm64/fp/sve-test.S index 4328895dfc87..486634bc7def 100644 --- a/tools/testing/selftests/arm64/fp/sve-test.S +++ b/tools/testing/selftests/arm64/fp/sve-test.S @@ -378,6 +378,8 @@ endfunction // Main program entry point .globl _start function _start + enable_gcs + mov x23, #0 // Irritation signal count mov w0, #SIGINT diff --git a/tools/testing/selftests/arm64/fp/za-test.S b/tools/testing/selftests/arm64/fp/za-test.S index 9dcd70911397..f789694fa3ea 100644 --- a/tools/testing/selftests/arm64/fp/za-test.S +++ b/tools/testing/selftests/arm64/fp/za-test.S @@ -231,6 +231,8 @@ endfunction // Main program entry point .globl _start function _start + enable_gcs + mov x23, #0 // signal count mov w0, #SIGINT diff --git a/tools/testing/selftests/arm64/fp/zt-test.S b/tools/testing/selftests/arm64/fp/zt-test.S index d63286397638..ea5e55310705 100644 --- a/tools/testing/selftests/arm64/fp/zt-test.S +++ b/tools/testing/selftests/arm64/fp/zt-test.S @@ -200,6 +200,8 @@ endfunction // Main program entry point .globl _start function _start + enable_gcs + mov x23, #0 // signal count mov w0, #SIGINT