From patchwork Fri Feb 14 01:57:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865234 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 410DB13AA2F; Fri, 14 Feb 2025 02:01:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498462; cv=none; b=Wdqdcc++EE+ACzb7MPR/iOU1OGR1GgAUtu6Sv8kxk+gfIBmJOelCXlQLGtJFjauQ+6RFuxhhKH/D9CtgteksfKtPJv00PPaO+MR4XzX+oD6lozZMzMgOkRgeloZn7G44RJW2aZGlwOWzu2Zh7k64ZAlLb5bnUhxFFXsMbRSPFqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498462; c=relaxed/simple; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=th3kHeMueQFBCyo3kHFfr8/HZeM41g6mO4QeUcPN7bT0WJnM9oJlLFllNu9lY3k6IwpYDIcGZKcnVqhZRtKD046P5HxGHF4utsyl8IjeF10mF/GP6qPrJfjIAAHcld7AwEmzK2LT1ofvUeHZ9kWEi+uMd2aflQUjbErzhh+zBGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XfdNjUR5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XfdNjUR5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5A3F3C4CEE6; Fri, 14 Feb 2025 02:00:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498461; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XfdNjUR5WvKd0M+mir0toNA5usCYkyCvyL1IdKDsA6ruyoNu1d7Gr1c/MUY9aDfVy oJhxlRyV+i/cBnJO/uw/vQ2MgHtoPLAjA5ZaIObdSUZ8ckdEqFtZW5u+Xj030hRqWP lRWHVJfYZPKsTKbuPrSOes4f1O/EI/rXFa8VLXUUTjjWgq7SQtozf6qg8TqxNopI1y 1Uqf6QK24EdWKFD1AIsS5XkdYBjJH0wCWr3fgBa2Ab/U11o3XkT6x2LEwav5wY9zH7 mrBE4Cgvi8cZ2nDHmJYQebqiOsaXZwPjYzXNkUC/yiFOL9NbpkS1nQucyLqbYXevOd sdzV7iXYrt95w== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:44 +0000 Subject: [PATCH v4 01/27] arm64/fpsimd: Update FA64 and ZT0 enables when loading SME state Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-1-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5400; i=broonie@kernel.org; h=from:subject:message-id; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPBMdsgvM+gVTEg6r1ZGMsNCAoG4qXO215iqD2E bZBSAMyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jwQAKCRAk1otyXVSH0Es+B/ sFBDsFj8Nb12IFeon3+wHzalejhq9ZW8OSC65eCxkTfByjoBMXLrYOiM8srDG+ilcRsH3Egwn6EJ/n aaZqtKm913QXspxpRKDGq/x7TS1wjz3X60jIzKRXsN+O4vbA8+sQhjvO8+eoRPtjd9LeZiNG2N7CeA TWnm3W2lKZWZ9tbec2ixGdtmn/Y3aFTMkUZIBnLzPigM4F5Hzp5gbUZ4TcR+c4RxePznuVVs7YfOWO KWVl/44Uc2l56SKZsAFHTeZTxWedl+e0ycDe8LuzA+4SaL/QWlHjmL/W/V8fQgCQ+53Izz6JnI5ZCp XgJpqDl9qI3M1rNo8iZYaWHxmJ0vRO X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we enable EL0 and EL1 access to FA64 and ZT0 at boot and leave them enabled throughout the runtime of the system. When we add KVM support we will need to make this configuration dynamic, these features may be disabled for some KVM guests. Since the host kernel saves the floating point state for non-protected guests and we wish to avoid KVM having to reload the floating point state needlessly on guest reentry let's move the configuration of these enables to the floating point state reload. We provide a helper which does the configuration as part of a read/modify/write operation along with the configuration of the task VL, then update the floating point state load and SME access trap to use it. We also remove the setting of the enable bits from the CPU feature identification and resume paths. There will be a small overhead from setting the enables one at a time but this should be negligable in the context of the state load or access trap. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 12 +++++++++++ arch/arm64/kernel/cpufeature.c | 2 -- arch/arm64/kernel/fpsimd.c | 44 ++++++++++++----------------------------- 3 files changed, 25 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index f2a84efc361858d4deda99faf1967cc7cac386c1..95355892d47b3ec1c77a3ab19ccad0d7f9a8d621 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -429,6 +429,18 @@ static inline size_t sme_state_size(struct task_struct const *task) #endif /* ! CONFIG_ARM64_SME */ +#define sme_cond_update_smcr(vl, fa64, zt0, reg) \ + do { \ + u64 __old = read_sysreg_s((reg)); \ + u64 __new = vl; \ + if (fa64) \ + __new |= SMCR_ELx_FA64; \ + if (zt0) \ + __new |= SMCR_ELx_EZT0; \ + if (__old != __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index f0910f20fbf8c18fbeb63bcee18abf13371b1d5e..7ba2a554243fccfb94d34307821101bfcf6c07bb 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2857,7 +2857,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME_FA64, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_fa64, ARM64_CPUID_FIELDS(ID_AA64SMFR0_EL1, FA64, IMP) }, { @@ -2865,7 +2864,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME2, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_sme2, ARM64_CPUID_FIELDS(ID_AA64PFR1_EL1, SME, SME2) }, #endif /* CONFIG_ARM64_SME */ diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8370d55f035334edf9d4f01fb33e1054bddadf71..e52ec18f0fcab0e34123b7a115e886fca3fae210 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -399,9 +399,15 @@ static void task_fpsimd_load(void) if (system_supports_sme()) { unsigned long sme_vl = task_get_sme_vl(current); - /* Ensure VL is set up for restoring data */ + /* + * Ensure VL is set up for restoring data. KVM might + * disable subfeatures so we reset them each time. + */ if (test_thread_flag(TIF_SME)) - sme_set_vq(sve_vq_from_vl(sme_vl) - 1); + sme_cond_update_smcr(sve_vq_from_vl(sme_vl) - 1, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); write_sysreg_s(current->thread.svcr, SYS_SVCR); @@ -1267,26 +1273,6 @@ void cpu_enable_sme(const struct arm64_cpu_capabilities *__always_unused p) isb(); } -void cpu_enable_sme2(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME2 <= ARM64_SME); - - /* Allow use of ZT0 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_EZT0_MASK, - SYS_SMCR_EL1); -} - -void cpu_enable_fa64(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME_FA64 <= ARM64_SME); - - /* Allow use of FA64 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_FA64_MASK, - SYS_SMCR_EL1); -} - void __init sme_setup(void) { struct vl_info *info = &vl_info[ARM64_VEC_SME]; @@ -1330,17 +1316,9 @@ void __init sme_setup(void) void sme_suspend_exit(void) { - u64 smcr = 0; - if (!system_supports_sme()) return; - if (system_supports_fa64()) - smcr |= SMCR_ELx_FA64; - if (system_supports_sme2()) - smcr |= SMCR_ELx_EZT0; - - write_sysreg_s(smcr, SYS_SMCR_EL1); write_sysreg_s(0, SYS_SMPRI_EL1); } @@ -1457,7 +1435,11 @@ void do_sme_acc(unsigned long esr, struct pt_regs *regs) if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { unsigned long vq_minus_one = sve_vq_from_vl(task_get_sme_vl(current)) - 1; - sme_set_vq(vq_minus_one); + + sme_cond_update_smcr(vq_minus_one, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); fpsimd_bind_task_to_cpu(); } From patchwork Fri Feb 14 01:57:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865233 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0B9312B17C; Fri, 14 Feb 2025 02:01:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498471; cv=none; b=kW1+gLI6dlkvx/2AIb3VbizVjufQvi20rIOkam0iTggqvUvAhtDEPjI9uI7I0bFoprvRcgbtd6/2FN6naWlYy4PelcOWq0ZEwvgwJ3qLpwLciiJZxf1Ya1jhLrxo9Ai7h4vuWeEzbL/cXHzZFDTDP73SLTeWC/datmfnNWuMwOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498471; c=relaxed/simple; bh=XKkgc2ILOcIk5OZtippiLeHcza1bJLfSqPNREumRZo0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dus1GjujjYDrdhH5Dv5OzuuA6TmUFkQnsjPx5WspaoZMuo1Y3Xjz5jHwWFzxAVhoUp4NktNjx+tL9Cas+De25a53ThjmoCv+xsO7OSiM6dutqvyfl71OTBAX7FiuNTgpkO1e+31SNcS1mOSSHFTs/y0qMcG5Zq1cYD4xzG/2Bcs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TIwiYkpb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TIwiYkpb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18D08C4CEEB; Fri, 14 Feb 2025 02:01:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498471; bh=XKkgc2ILOcIk5OZtippiLeHcza1bJLfSqPNREumRZo0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=TIwiYkpbfwwAR8616SkmJGruf5XyQrNCGdKxu/NIsJDAEVgvK+gPhTrL2dsmJig0+ uhJb7W4STeUohsGvPAGGbSfP8U7Mn38w+hDl9N6xDa27kGl2Qtgt19hhNQNuDj/bAR T9NmyYhqPX06nmRJiWSrmbU+JgB9HzbaB4K/dIE8XI7GlFbWT8pbi8fZ+6cn4UyZAI IOsiqMvdPmVYTXHdM/dbtKyb1ZPYcT5X8Brlf8mNtNisCHrztHBh4bk3I8emn73ACf hgRjQJe0eZpgrg7krEs4ww1YKO1Absj+KZo8xs94kDx9Nk0dQY0wUGVdj4WZzqE1Vh LwMd+f1PSkgXg== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:46 +0000 Subject: [PATCH v4 03/27] arm64/fpsimd: Check enable bit for FA64 when saving EFI state Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-3-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=1524; i=broonie@kernel.org; h=from:subject:message-id; bh=XKkgc2ILOcIk5OZtippiLeHcza1bJLfSqPNREumRZo0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPCoUhVMtJEjvpA+QPVPDAXTGF2POU//l0shFXR 9FqDoS2JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jwgAKCRAk1otyXVSH0CQ1B/ 9AR+Gl+YtCX69mA6Ju8BvQmrPUol5Kqjmozm7grMCtnUYXnpKGXCfXTtFTOD+0KPTWtOuz7rbTMgPM PDbxzQn0WY8n2twP/Vqr8ZGlP35ntN8jLUQILxqHpXpvEeqVHPTlYYYnJgI7Fogyt6ACXZ0qlShy36 RDrsSyPT9joYNzzuPyoEl+LpLUqZZE+VleDeRcclR5c6gQl/UbG7aTAZv2Wr8UYoBVrg9mrHF2WDDv XYDcLzJlwMyJybV2A7rAD9gNxjJpmBWBT+Hes9LVAVx6wpH2KBZdMbf85qP0ZPdnuC2t4OL7ZwrL9U 1WHaTwS4MLi/SKd9EOMvaYhLR7r+LR X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently when deciding if we need to save FFR when in streaming mode prior to EFI calls we check if FA64 is supported by the system. Since KVM guest support will mean that FA64 might be enabled and disabled at runtime switch to checking if traps for FA64 are enabled in SMCR_EL1 instead. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 446a379d87539bb37a9d4eb7466a73d8819afc56..a97af9daea472cc57bafc0564fdfe4e327924db1 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1955,6 +1955,11 @@ static DEFINE_PER_CPU(bool, efi_sm_state); * either doing something wrong or you need to propose some refactoring. */ +static bool fa64_enabled(void) +{ + return read_sysreg_s(SYS_SMCR_EL1) & SMCR_ELx_FA64; +} + /* * __efi_fpsimd_begin(): prepare FPSIMD for making an EFI runtime services call */ @@ -1989,7 +1994,7 @@ void __efi_fpsimd_begin(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr = !(svcr & SVCR_SM_MASK); } @@ -2040,7 +2045,7 @@ void __efi_fpsimd_end(void) * Unless we have FA64 FFR does not * exist in streaming mode. */ - if (!system_supports_fa64()) + if (!fa64_enabled()) ffr = false; } } From patchwork Fri Feb 14 01:57:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865232 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21D2B12B17C; Fri, 14 Feb 2025 02:01:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498479; cv=none; b=nz6oPITe7g0DFJfUg8uDWxB8a3dx69J2bXsNm+Rvn8CtQy08yvJedphxJL1R7nBGEMdaYKpJ54dUvh1UsahxrZW2Fd0Pyzd+rUDUvZXlVdCARhSXs60JXbTUVFL6bM3kplj5ak3ux0kmJCWBRzHvnOSlCbj0PoMD2nlnw8Yat8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498479; c=relaxed/simple; bh=3PQifWZa0PYzxsXTtYT3uqYK+qICLHgWCa2IqwMJGYI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=m/zKjl9TSkhHUbWeyuMCZEZbs+3xbFyL20z6uBySyicHUWSS+QuYaRkqtCJ4QMIYLo8tBaMsLSt2K3tEMgo3/58pkS2elCKcwR+GRlNQvO/HO3c+2oEAzmubYX7/x7kYFv5xPXBEnNYQXf+i51lkpLzCslmLH6ymA7wU2UsHYqs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jzBV85hb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jzBV85hb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6129EC4CED1; Fri, 14 Feb 2025 02:01:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498479; bh=3PQifWZa0PYzxsXTtYT3uqYK+qICLHgWCa2IqwMJGYI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=jzBV85hbxeD88bGgqVz/W8uuHG/uvPe4kuWarE+38Oq82dsMUNSwtlEj6A0taOsG8 k4YZebjgAB98FHJU8CnRKLBe9MFgtkCa11zm5/BtEQKxMjH9FQZaGkFh03Y+EnRKp0 WAdtdDQ3Gt31mfyqB6jhPgwuBfUsTK9DBKEThDSmTZH3pYpNAZmLsRv3enC3HtW9EO lOl0MVhWSgmZvJaR95/dpDF1JjJf5d8NoxVD6iOkOk+0n5n10Sn8Wcxngmv8a678In u2t6tTS8n0Hj9XVPV4ivDUAZxvjRbwsHsKqmVuoO7DRoOryJcpwXqWbWojfkhJ8sL+ vKqpNlBFElZkg== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:48 +0000 Subject: [PATCH v4 05/27] KVM: arm64: Introduce non-UNDEF FGT control Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-5-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=2905; i=broonie@kernel.org; h=from:subject:message-id; bh=3PQifWZa0PYzxsXTtYT3uqYK+qICLHgWCa2IqwMJGYI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPEnFldtuIWxcKKrLml22ik1aL0J2nPamYq3D8M bIdk56qJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jxAAKCRAk1otyXVSH0K1aB/ 4t3+9jnwdck95RN7cJPA29m7VB8XNVNBH7EtfLO6PAzqjz7YMptZ6PJ2ceu5gO8JaXh2oQy7xe/clC G05wYyaIwOzkWIzP01yKpIs+BHtkTP2taZBfzFaGmI90aTBC9EgOUMy9vD/9UDliG6NZFYU9Qv8gUC o3mq+fciqKvLtYvDls8jrQQCh9i/Z8KBmcKyjlBV4pe/vLoNOzTyf7cz6ZFAUsu8eU2U3XMV0qk6Nt RAGPr3OaTzIl8Eu9d7bxgMzmgOeSoPiesFrDx89cepZ9zPpqNTA0RarQTia9eem07otBf/ieM8DtaW ZCiGLzwtJeqS10OXuju9M55meoz6Nj X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB We have support for determining a set of fine grained traps to enable for the guest which is tied to the support for injecting UNDEFs for undefined features. This means that we can't use the mechanism for system registers which should be present but need emulation, such as SMPRI_EL1 which should be accessible when SME is present but if SME priority support is absent SMPRI_EL1.Priority should be RAZ. Add an additional set of fine grained traps fgt, mirroring the existing fgu array. We use the same format where we always set the bit for the trap in the array as for FGU. This makes it clear what is being explicitly managed and keeps the code consistent. We do not convert the handling of ARM_WORKAROUND_AMPERE_ACO3_CPU_38 to this mechanism since this only enables a write trap and when implementing the existing UNDEF that we would share the read and write trap enablement (this being the overwhelmingly common case). Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 7 ++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index c77acc99045764cf5230f84ef5c9f8432c7d983e..64f2885cd4a87e9807fbbbbe6de937ed304ab917 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -289,6 +289,12 @@ struct kvm_arch { */ u64 fgu[__NR_FGT_GROUP_IDS__]; + /* + * Additional FGTs to enable for the guests, eg. for emulated + * registers, + */ + u64 fgt[__NR_FGT_GROUP_IDS__]; + /* * Stage 2 paging state for VMs with nested S2 using a virtual * VMID. diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index f5e882a358e2d6e6805d112ed646a112455012e8..c24e418cec101433f7484eb4f7af788c474eda3a 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -98,9 +98,9 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) id; \ }) -#define compute_undef_clr_set(vcpu, kvm, reg, clr, set) \ +#define compute_trap_clr_set(vcpu, kvm, trap, reg, clr, set) \ do { \ - u64 hfg = kvm->arch.fgu[reg_to_fgt_group_id(reg)]; \ + u64 hfg = kvm->arch.trap[reg_to_fgt_group_id(reg)]; \ set |= hfg & __ ## reg ## _MASK; \ clr |= hfg & __ ## reg ## _nMASK; \ } while(0) @@ -113,7 +113,8 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu)) \ compute_clr_set(vcpu, reg, c, s); \ \ - compute_undef_clr_set(vcpu, kvm, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgu, reg, c, s); \ + compute_trap_clr_set(vcpu, kvm, fgt, reg, c, s); \ \ s |= set; \ c |= clr; \ From patchwork Fri Feb 14 01:57:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865231 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF4F213C67E; Fri, 14 Feb 2025 02:01:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498486; cv=none; b=qJSRvt+vdjkPSf2IIFEZCqF8gWaEH1SrnuV4wIzhVblTLO7THLjJrRulM+REYyNxBCJUawvPrs5bJ3pkqx4hkE7SnAY9Oma5J8bIpMyYYuHzZiEd/7zKiHHnDRZjtuSv76ABD/dARYHGyPCPUgMEqb676eQbkWm4Svs97z9c3ik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498486; c=relaxed/simple; bh=ST8cvmEJlmmZZoglNVpZrkW7z4enEIPEfasznNtVTCQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i+lv4km+mjRkHLNt3uBvnXCCOnrm6RwwhBuM5oACPoEg8qOzFK7ytMR9s+gCo6eFPEF7DEzKXgZXRxo1Grjac+UmOYoIHHQpELdDDV39GsxSl9ls7tljU03ykGIptZI77UNfdYysQKktu05CDtIpsNAtAm4sq8TRBRwVLWEwtdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OFTcAcZZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OFTcAcZZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B4EDC4CED1; Fri, 14 Feb 2025 02:01:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498486; bh=ST8cvmEJlmmZZoglNVpZrkW7z4enEIPEfasznNtVTCQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OFTcAcZZd1OnkLlymvGvZ7PaeLBLX/x8RW3R9r40OtIad5SSN3FqHOnxl2DBojztY 4w9+unQOVdxfRhkOE9oX6KkGR5IUtY5Oxy5pQxMFX4RS4/fdkPMOU5AfE3tG3AnrSX ofolpzzj2ODEt6sjZXiVKZLnLHK7RqKkh1e0tgFtewRSinkd37JNdborIhKG9wVAA3 ZvKlT/PoJujmrDA0m6CeuTc3tvQ/WJWNWaVIxzVTl65aww6lrH4itmdbyl42NPzcDI SALjBGVmWcFxV0WlLUtYaw/13YM41S8bA/8vfDU2gysvKYXACEzEIsv91aX9RPr3yM cOCxn6PV7wLrQ== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:50 +0000 Subject: [PATCH v4 07/27] KVM: arm64: Pull ctxt_has_ helpers to start of sysreg-sr.h Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-7-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3314; i=broonie@kernel.org; h=from:subject:message-id; bh=ST8cvmEJlmmZZoglNVpZrkW7z4enEIPEfasznNtVTCQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPFTI+XTXydWt3dsh69De8TwYqweuNewVUYsHNV 073HGsSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jxQAKCRAk1otyXVSH0IvCB/ 95sal3p7n1BoGOj5DL94/59IWLjclRI+C8m4TkdMzSBzs2ARYfbmAtiElcpw1gKP5q321j10CNUZK0 EJp0hVcVx8dSzSMXD3WINQWtAmSR+X/S2ugEWcus1OwDxr4SOjWVvm8uCU4YNJFtGNEIaw+ZfZ3/Q7 p6n9V+Nqw3OSHBL7V8YE0CnO+egiIdjiro9m5SmSkI18AjCnHQn3r7Ighh51eHvLE++n6bjcQ2IVbn 6cHItw4+TgOKqpmzhmhxlDNvqAYSkSUIIUBF4vOBTo0ALd2WRpXm9WcU1NEgkCH/CkI+MW9YWKgRr4 6pmu3BF2wAdkbESkzTd1RzN1fgBNFy X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Rather than add earlier prototypes of specific ctxt_has_ helpers let's just pull all their definitions to the top of sysreg-sr.h so they're all available to all the individual save/restore functions. Signed-off-by: Mark Brown --- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 62 +++++++++++++++--------------- 1 file changed, 30 insertions(+), 32 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h index 76ff095c6b6ebf336aab3018ec1f384a5f19949e..7a4ad8a4c3727e4628b375cbefc5e0d3533687de 100644 --- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h @@ -16,8 +16,6 @@ #include #include -static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt); - static inline struct kvm_vcpu *ctxt_to_vcpu(struct kvm_cpu_context *ctxt) { struct kvm_vcpu *vcpu = ctxt->__hyp_running_vcpu; @@ -28,36 +26,6 @@ static inline struct kvm_vcpu *ctxt_to_vcpu(struct kvm_cpu_context *ctxt) return vcpu; } -static inline bool ctxt_is_guest(struct kvm_cpu_context *ctxt) -{ - return host_data_ptr(host_ctxt) != ctxt; -} - -static inline u64 *ctxt_mdscr_el1(struct kvm_cpu_context *ctxt) -{ - struct kvm_vcpu *vcpu = ctxt_to_vcpu(ctxt); - - if (ctxt_is_guest(ctxt) && kvm_host_owns_debug_regs(vcpu)) - return &vcpu->arch.external_mdscr_el1; - - return &ctxt_sys_reg(ctxt, MDSCR_EL1); -} - -static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) -{ - *ctxt_mdscr_el1(ctxt) = read_sysreg(mdscr_el1); - - // POR_EL0 can affect uaccess, so must be saved/restored early. - if (ctxt_has_s1poe(ctxt)) - ctxt_sys_reg(ctxt, POR_EL0) = read_sysreg_s(SYS_POR_EL0); -} - -static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) -{ - ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); - ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); -} - static inline bool ctxt_has_mte(struct kvm_cpu_context *ctxt) { struct kvm_vcpu *vcpu = ctxt_to_vcpu(ctxt); @@ -98,6 +66,36 @@ static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt) return kvm_has_s1poe(kern_hyp_va(vcpu->kvm)); } +static inline bool ctxt_is_guest(struct kvm_cpu_context *ctxt) +{ + return host_data_ptr(host_ctxt) != ctxt; +} + +static inline u64 *ctxt_mdscr_el1(struct kvm_cpu_context *ctxt) +{ + struct kvm_vcpu *vcpu = ctxt_to_vcpu(ctxt); + + if (ctxt_is_guest(ctxt) && kvm_host_owns_debug_regs(vcpu)) + return &vcpu->arch.external_mdscr_el1; + + return &ctxt_sys_reg(ctxt, MDSCR_EL1); +} + +static inline void __sysreg_save_common_state(struct kvm_cpu_context *ctxt) +{ + *ctxt_mdscr_el1(ctxt) = read_sysreg(mdscr_el1); + + // POR_EL0 can affect uaccess, so must be saved/restored early. + if (ctxt_has_s1poe(ctxt)) + ctxt_sys_reg(ctxt, POR_EL0) = read_sysreg_s(SYS_POR_EL0); +} + +static inline void __sysreg_save_user_state(struct kvm_cpu_context *ctxt) +{ + ctxt_sys_reg(ctxt, TPIDR_EL0) = read_sysreg(tpidr_el0); + ctxt_sys_reg(ctxt, TPIDRRO_EL0) = read_sysreg(tpidrro_el0); +} + static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) { ctxt_sys_reg(ctxt, SCTLR_EL1) = read_sysreg_el1(SYS_SCTLR); From patchwork Fri Feb 14 01:57:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865230 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD7F0146A7A; Fri, 14 Feb 2025 02:01:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498494; cv=none; b=UridUen/MCd1s/wqqX003bym3Anzv8mkcHybXw3MCrvS8ZpMT5FoUphh5hr5vJn2ay8W3upiIl7q8UP4Owuo7Lbq/1NUx0H57SLYFarjpr9+zkVEtWKxYAhPGXymabfC2eyTcUlkBXOYfSI1muSt1DgoSv0JBfjckrt5pRYsbJc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498494; c=relaxed/simple; bh=cmowkn5TVdFsHjj55S6rFAbnGcy4evtVKNvFJvkiRjM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lB7SeZQuOqQbG7fYr7NZVNiaW+zeVlRRo7conwAHYQ0BeWPyCg66d6A66mGnVzgewwDapvUTBrt+n8K3lOZrYUGer2Lu9w3o8INKn2xYML/G3vFE4OsnM/sakzsgsP+ckcFeVIOiN7xmod0laSkmOQMDk5Cp3tRubFlbjD8l3ho= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=k+LMlmS8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="k+LMlmS8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB5E2C4CED1; Fri, 14 Feb 2025 02:01:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498494; bh=cmowkn5TVdFsHjj55S6rFAbnGcy4evtVKNvFJvkiRjM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=k+LMlmS8Nw6aTGNc8BQjtQp9EwoEN7CYcn7FlU3Kwfe8ae9nfPBbQig8bewKa7slY ADLgtWmfuWjdg4FSl9gc5KqMT7UD6892fOqRDgeV14n9eIu8WpVlWLTtinbXgZ/nV2 5Q5tzMtAcswYAUipHq3OvRHzFFnj1ecvyZ4Ybv4uz1RniJW5YHaZmjUgifkq5iMw8i 7U1RwESFVe9XJQPIwkir6LoVgw4kO1LyV2xy6trqzPm1Q97jfa5Cc6Fif/XdWr/jle m86g3nCGsk47EXHfrlnTtGeZkZPNxGovVGIYvOYsfc8AWRyHnZotW15gyrq5Yh1eQB ontPt42i2ExSQ== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:52 +0000 Subject: [PATCH v4 09/27] KVM: arm64: Rename SVE finalization constants to be more general Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-9-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=7533; i=broonie@kernel.org; h=from:subject:message-id; bh=cmowkn5TVdFsHjj55S6rFAbnGcy4evtVKNvFJvkiRjM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPHl9LWxD3Ti32r74HX51zf+svqFuYjb1SkR9Ao yNv5q9aJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jxwAKCRAk1otyXVSH0HXHB/ 4ngTsPBa9zdPbFZIuRe1WzDihc8fbSHkAZt1PA40XfDeiSzkJGaEuAElPgWX+ogo+zp4ZK4oejdz/X rkItdo+IEDpAWSOvY8rwFkIttuSOr5DG8N1plAqZuTxfw6z/lpg8VzyG8SmlSYQYkXezPtraVJAgX0 t59K4jjeFHw1WWsLtkoQxJe8O2ylIvYhXXMCcdeEqH9rOhYyZ8PRDAOzUje/myqQzRz8piDhC5qgv4 Q3BGLYG5Dw5VovB6lKDuLoGEHPrFxUFutiXVyX8fJww/gM46II4MuAmVWLxe69qGb0yGE7B2Kbi5FI N/vceoRdfJMoLxb9/6VzXQs3StIwjU X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Due to the overlap between SVE and SME vector length configuration created by streaming mode SVE we will finalize both at once. Rename the existing finalization to use _VEC (vector) for the naming to avoid confusion. Since this includes the userspace API we create an alias KVM_ARM_VCPU_VEC for the existing KVM_ARM_VCPU_SVE capability, existing code which does not enable SME will be unaffected and any SME only code will not need to use SVE constants. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 6 ++++-- arch/arm64/include/uapi/asm/kvm.h | 6 ++++++ arch/arm64/kvm/guest.c | 10 +++++----- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 20 ++++++++++---------- 5 files changed, 26 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index cef03309fbe6b7f44ab30357ceb037e7e342fb0c..f12b13c3886a2b90d0f1e95f9d59f374d3c87398 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -874,7 +874,7 @@ struct kvm_vcpu_arch { /* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0)) /* SVE config completed */ -#define VCPU_SVE_FINALIZED __vcpu_single_flag(cflags, BIT(1)) +#define VCPU_VEC_FINALIZED __vcpu_single_flag(cflags, BIT(1)) /* Exception pending */ #define PENDING_EXCEPTION __vcpu_single_flag(iflags, BIT(0)) @@ -941,6 +941,8 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif +#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) + #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ ((cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH) || \ @@ -1423,7 +1425,7 @@ struct kvm *kvm_arch_alloc_vm(void); int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature); bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu); -#define kvm_arm_vcpu_sve_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_SVE_FINALIZED) +#define kvm_arm_vcpu_vec_finalized(vcpu) vcpu_get_flag(vcpu, VCPU_VEC_FINALIZED) #define kvm_has_mte(kvm) \ (system_supports_mte() && \ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 568bf858f3198e2a6f651eb8ae793b54fd49e67a..ea5aaa4b1cfe82862883678bbe47373d53c005f6 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -106,6 +106,12 @@ struct kvm_regs { #define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */ #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ +/* + * An alias for _SVE since we finalize VL configuration for both SVE and SME + * simultaneously. + */ +#define KVM_ARM_VCPU_VEC KVM_ARM_VCPU_SVE + struct kvm_vcpu_init { __u32 target; __u32 features[7]; diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 2196979a24a325311d6111404e4d089287c41bfe..73e714133bb647c1d50fc67d5c8bf0569520a537 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -342,7 +342,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; /* too late! */ if (WARN_ON(vcpu->arch.sve_state)) @@ -497,7 +497,7 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (ret) return ret; - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; if (copy_to_user(uptr, vcpu->arch.sve_state + region.koffset, @@ -523,7 +523,7 @@ static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (ret) return ret; - if (!kvm_arm_vcpu_sve_finalized(vcpu)) + if (!kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; if (copy_from_user(vcpu->arch.sve_state + region.koffset, uptr, @@ -657,7 +657,7 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu) return 0; /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */) + 1; /* KVM_REG_ARM64_SVE_VLS */ @@ -675,7 +675,7 @@ static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu, return 0; /* Policed by KVM_GET_REG_LIST: */ - WARN_ON(!kvm_arm_vcpu_sve_finalized(vcpu)); + WARN_ON(!kvm_arm_vcpu_vec_finalized(vcpu)); /* * Enumerate this first, so that userspace can save/restore in diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 3927fe52a3dde8f59ddb996d674dd252630613a3..5896e581c1fac943cf372f1df24311ef43529d62 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -381,7 +381,7 @@ static void pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu * struct kvm_vcpu *vcpu = &hyp_vcpu->vcpu; if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) - vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_clear_flag(vcpu, VCPU_VEC_FINALIZED); } static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 803e11b0dc8f5eb74b07b0ad745b0c4f666713d5..ce726b1d4e8e90cfd4459a6cb9c67b8805424e22 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -92,7 +92,7 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) * Finalize vcpu's maximum SVE vector length, allocating * vcpu->arch.sve_state as necessary. */ -static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) +static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { void *buf; unsigned int vl; @@ -122,21 +122,21 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) } vcpu->arch.sve_state = buf; - vcpu_set_flag(vcpu, VCPU_SVE_FINALIZED); + vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; } int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) { switch (feature) { - case KVM_ARM_VCPU_SVE: - if (!vcpu_has_sve(vcpu)) + case KVM_ARM_VCPU_VEC: + if (!vcpu_has_vec(vcpu)) return -EINVAL; - if (kvm_arm_vcpu_sve_finalized(vcpu)) + if (kvm_arm_vcpu_vec_finalized(vcpu)) return -EPERM; - return kvm_vcpu_finalize_sve(vcpu); + return kvm_vcpu_finalize_vec(vcpu); } return -EINVAL; @@ -144,7 +144,7 @@ int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) { - if (vcpu_has_sve(vcpu) && !kvm_arm_vcpu_sve_finalized(vcpu)) + if (vcpu_has_vec(vcpu) && !kvm_arm_vcpu_vec_finalized(vcpu)) return false; return true; @@ -161,7 +161,7 @@ void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) kfree(vcpu->arch.ccsidr); } -static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu) +static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); @@ -204,11 +204,11 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (loaded) kvm_arch_vcpu_put(vcpu); - if (!kvm_arm_vcpu_sve_finalized(vcpu)) { + if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); } else { - kvm_vcpu_reset_sve(vcpu); + kvm_vcpu_reset_vec(vcpu); } if (vcpu_el1_is_32bit(vcpu)) From patchwork Fri Feb 14 01:57:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865229 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE4EB14A630; Fri, 14 Feb 2025 02:01:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498502; cv=none; b=nc9Hj8OppOFJRdmCVx3Pb/0SGQy+VHxRmFgy3CVoGXrq5sNt32xcSlbX4/ZUSEzQckxp6X4cn96EttkaUTVhMic4I7bCQPiqqO3X9B0DtjK6n5MCK4bOG4fAyC1rrpL+VAcC0ehjlfBI6jZIvq8JySPCkqJtS/Kf8LjgZpcBI3k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498502; c=relaxed/simple; bh=pIXa80sqzHzFpWf+BtFQDhwZHyO3E94JaRZryQDM4fI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=VoG3B6/0KcRuauoedcjXywuR/oFwAtN4J01/oJ7/mSS8LpUBklRkdA/frsB9vtOZwpFSmJx2B4U5wDxRt8uWPLdQVo5sA3lg/q3JON2xpHRQ53YPvZk04klNdl2hUSmAVNoBdpHHY8RfVv8E7X/uhDFy+aoitk5vDLb4Y4gC7dU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=i6MtElX4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i6MtElX4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 697B7C4CEE8; Fri, 14 Feb 2025 02:01:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498501; bh=pIXa80sqzHzFpWf+BtFQDhwZHyO3E94JaRZryQDM4fI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=i6MtElX4fLodMybOfuMqg2AlpZFWQzfsrzHEfVCFp9gAD6y9aJLigtLYg7AwNJRsE Q3CbFPrQHO0X2UVtS/1L5YWuC/dkr3tm5q6gqod8zj8dfWo8wmAegh2+7G+TuRzKEx 7KKVECifyPq1YZBfgVjmhm4p3eX6Ds6zeHYdESEIMZqETp7BoNDd1EW5WfPfLcFeBB u2uCDxejaqKo5M9rhNZvI95TEd3G0y7wBl08NRc7n5W0d2AjgIBeC9IzPzHFd5l/v6 EwQYrnzuAV51Z+eZbpUNwe5arnWog9NpMtq0oVBBIpjpDGA+OOkb/5kPSDVrn1vSij dqlSsgjArBbTA== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:54 +0000 Subject: [PATCH v4 11/27] KVM: arm64: Define internal features for SME Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-11-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3280; i=broonie@kernel.org; h=from:subject:message-id; bh=pIXa80sqzHzFpWf+BtFQDhwZHyO3E94JaRZryQDM4fI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPJQPFdilei7rkQV+MBFJubRfiidnpeFvfS5+lL TCDF12+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jyQAKCRAk1otyXVSH0PVeB/ 9eM3Ub+1FRgxCLNlkjT+HrL5Nnh2dYhvBVpDCniMUhDBOplkdpcdpiYXPEbQGd+O+iQrjPeNj2+lez hZt2JqIP1IQu9Y43So1lybybs6Qi71jVRdj0amWpIZ3ylXBrucr8fdHkrPjqqNZE9PEWjrz4oPyqqh cTcBRpQ7p6GmqFl7npTSpsnZxo6jhkuwUFy6x/7VYPToenzHfqGhWZJfTvd6SpWTRLEcDPRN35MRLr 78ZNJpDvoCmeM7s7YASmvF4MHp363uHICFrFg9rP5v4jpqh09rxut2wrBGFrOufCFterrReCGr/K0b x6EkFNDWd/v0MzF6lw1ApczHd8TOoS X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order to simplify interdependencies in the rest of the series define the feature detection for SME and it's subfeatures. Due to the need for vector length configuration we define a flag for SME like for SVE. We also have two subfeatures which add architectural state, FA64 and SME2, which are configured via the normal ID register scheme. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 31 +++++++++++++++++++++++++++++-- arch/arm64/kvm/sys_regs.c | 2 +- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f12b13c3886a2b90d0f1e95f9d59f374d3c87398..55ada264b7383925269674a47446b8f659a7e4e6 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -39,7 +39,7 @@ #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS -#define KVM_VCPU_MAX_FEATURES 7 +#define KVM_VCPU_MAX_FEATURES 9 #define KVM_VCPU_VALID_FEATURES (BIT(KVM_VCPU_MAX_FEATURES) - 1) #define KVM_REQ_SLEEP \ @@ -340,6 +340,8 @@ struct kvm_arch { #define KVM_ARCH_FLAG_FGU_INITIALIZED 8 /* SVE exposed to guest */ #define KVM_ARCH_FLAG_GUEST_HAS_SVE 9 + /* SME exposed to guest */ +#define KVM_ARCH_FLAG_GUEST_HAS_SME 10 unsigned long flags; /* VM-wide vCPU feature set */ @@ -941,7 +943,16 @@ struct kvm_vcpu_arch { #define vcpu_has_sve(vcpu) kvm_has_sve((vcpu)->kvm) #endif -#define vcpu_has_vec(vcpu) vcpu_has_sve(vcpu) +#define kvm_has_sme(kvm) (system_supports_sme() && \ + test_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &(kvm)->arch.flags)) + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define vcpu_has_sme(vcpu) kvm_has_sme(kern_hyp_va((vcpu)->kvm)) +#else +#define vcpu_has_sme(vcpu) kvm_has_sme((vcpu)->kvm) +#endif + +#define vcpu_has_vec(vcpu) (vcpu_has_sve(vcpu) || vcpu_has_sme(vcpu)) #ifdef CONFIG_ARM64_PTR_AUTH #define vcpu_has_ptrauth(vcpu) \ @@ -1551,4 +1562,20 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); #define kvm_has_s1poe(k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) +#define kvm_has_fa64(k) \ + (system_supports_fa64() && \ + kvm_has_feat((k), ID_AA64SMFR0_EL1, FA64, IMP)) + +#define kvm_has_sme2(k) \ + (system_supports_sme2() && \ + kvm_has_feat((k), ID_AA64PFR1_EL1, SME, SME2)) + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define vcpu_has_sme2(vcpu) kvm_has_sme2(kern_hyp_va((vcpu)->kvm)) +#define vcpu_has_fa64(vcpu) kvm_has_fa64(kern_hyp_va((vcpu)->kvm)) +#else +#define vcpu_has_sme2(vcpu) kvm_has_sme2((vcpu)->kvm) +#define vcpu_has_fa64(vcpu) kvm_has_fa64((vcpu)->kvm) +#endif + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 82430c1e1dd02b1ac24fd2ddcd05a91272997fdb..49b7844af8a19467e7842347c4b05ceb44c4caaf 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1748,7 +1748,7 @@ static unsigned int sve_visibility(const struct kvm_vcpu *vcpu, static unsigned int sme_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { - if (kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, IMP)) + if (vcpu_has_sme(vcpu)) return 0; return REG_HIDDEN; From patchwork Fri Feb 14 01:57:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865228 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C786F151980; Fri, 14 Feb 2025 02:01:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498509; cv=none; b=g/DLH/ZT/vM544IjZriVMHADhLS8q9xjKEdlm+t8340nel4CFomefK7n0Ovn1naQcT1LE51SEecSVUozrfZ+ZrTUrZGICZPrHvVxtwza4/cYJiQ47BESp7auTuzsLqzPQzKCHXOOCvOsCjzK47Ey0AhjPf6r/HrGUlBbFt8Skc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498509; c=relaxed/simple; bh=GuWrMzYMv30MyOuz8QmLC5BbCjRiVvr/3ixjt4maNuo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rQwkXlbTbwnE8z39o47CLW84rNRcIh2YttXb72Q3G+AcI50eGAu/FOXiKJ+70ZjUOWk4pZRPi4GElWQTLCSzc277ixSBqWyJqcz/R/4FpYtFFttDOAFBVs9cGdfghlBQwDYslaaSt4QTTYGTXe4DRg8Sf3/SR/sE5kHXoqH9HtY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fVB5tNyX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fVB5tNyX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7F54C4CEE8; Fri, 14 Feb 2025 02:01:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498509; bh=GuWrMzYMv30MyOuz8QmLC5BbCjRiVvr/3ixjt4maNuo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=fVB5tNyXRl1qqEzv9qBQIMgOuegKIFNPeB6NAoLg+nDaO7TqgLd5FVBh5oB0gTHqG K1lBZgtSxQGOaga5W/C1M/nphDOSGCmp/I6H41mC/MJP2Ng/a7sk40WFXxQRDPxMou Rq/yLYH8sQyTE8Ps+MV6tcfooxuQuAFtejr/PhpuqKdBlyvv4SeNEcSqmin8aMs+zA waJhOwlnB+R3ubLHAxkhQlgII/b73EFWhfCfDb0twhKGgJeXwuJ/nmYT4wlKXkefIr XTJj17D5Aos4qQffoK/Pc7GF34pCoYnUCuIqnL+z48DKNE/zi+tuRknxgj4CKPS75B 0SmO+iGfd8F7Q== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:56 +0000 Subject: [PATCH v4 13/27] KVM: arm64: Store vector lengths in an array Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-13-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=12082; i=broonie@kernel.org; h=from:subject:message-id; bh=GuWrMzYMv30MyOuz8QmLC5BbCjRiVvr/3ixjt4maNuo=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPK9S643LEgi4MEPAawDBMbtXLW5be2jRkrjSWb Z47BF06JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jygAKCRAk1otyXVSH0AT8CA CDdDLivP9ekZLnciBIA474rLH9dZWrJYusSFoCaDv/IGW/bAv7mLDSAECI5ahDt8Z2o094hBJpSYfx eSAXgee+j6D7P+MvSzjXOpq4Rq218PR6zUwPWCCaRHt8v1ar0gfsWu13SV60YEAQW7bCNf1FlNbFhG RqBfNLHUh68+TQKklhlRrzeAuYTWotQA3SjtCm2jDoyuB6vId4g3YdWXEzGL2UNoSzQfkplPKszujG yMJkJwsLaDyVs1b/82SkOFnUiETKnd4JV8BX/sT9S4NSOSPDiBPmnxwf1qUrczZrW6zfFpxWDlDEX1 qgDVw0lQU2APpmiExmMYralfj0I55W X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME adds a second vector length configured in a very similar way to the SVE vector length, in order to facilitate future code sharing for SME refactor our storage of vector lengths to use an array like the host does. We do not yet take much advantage of this so the intermediate code is not as clean as might be. No functional change. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 17 +++++++++++------ arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/fpsimd.c | 2 +- arch/arm64/kvm/guest.c | 6 +++--- arch/arm64/kvm/hyp/include/hyp/switch.h | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 9 +++++---- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 22 +++++++++++----------- 9 files changed, 37 insertions(+), 31 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 55ada264b7383925269674a47446b8f659a7e4e6..84aa728718e13b99b4f17f5fdffa81e380834e69 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -74,8 +74,10 @@ enum kvm_mode kvm_get_mode(void); static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; #endif -extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; +extern unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; +DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); + int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); @@ -716,7 +718,7 @@ struct kvm_vcpu_arch { */ void *sve_state; enum fp_type fp_type; - unsigned int sve_max_vl; + unsigned int max_vl[ARM64_VEC_MAX]; /* Stage 2 paging state used by the hardware on next switch */ struct kvm_s2_mmu *hw_mmu; @@ -977,9 +979,12 @@ struct kvm_vcpu_arch { /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ #define vcpu_sve_pffr(vcpu) (kern_hyp_va((vcpu)->arch.sve_state) + \ - sve_ffr_offset((vcpu)->arch.sve_max_vl)) + sve_ffr_offset((vcpu)->arch.max_vl[ARM64_VEC_SVE])) + +#define vcpu_vec_max_vq(vcpu, type) sve_vq_from_vl((vcpu)->arch.max_vl[type]) + +#define vcpu_sve_max_vq(vcpu) vcpu_vec_max_vq(vcpu, ARM64_VEC_SVE) -#define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) @@ -988,7 +993,7 @@ struct kvm_vcpu_arch { size_t __size_ret; \ unsigned int __vcpu_vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ + if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SVE]))) { \ __size_ret = 0; \ } else { \ __vcpu_vq = vcpu_sve_max_vq(vcpu); \ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index c838309e4ec47e395d78127a8ee6bad8390c4411..21943cb98542750a1b626a8de6bbc095d7770ccf 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -143,6 +143,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_MAX]); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index eb65f12e81d90939de45f0c6e59f0777a1f3f78f..950aa6a7d2c0fd41ac9ddfc374681a41cca97d4f 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -159,7 +159,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]))); } struct pkvm_mapping { diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index f324990a400293976fd505442f9eaccc6d01ed8f..656e02312a5991419bfc3c22193b6b9b4354be64 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -106,7 +106,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) */ fp_state.st = &vcpu->arch.ctxt.fp_regs; fp_state.sve_state = vcpu->arch.sve_state; - fp_state.sve_vl = vcpu->arch.sve_max_vl; + fp_state.sve_vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; fp_state.sme_state = NULL; fp_state.svcr = &__vcpu_sys_reg(vcpu, SVCR); fp_state.fpmr = &__vcpu_sys_reg(vcpu, FPMR); diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index bb4b91e4392359026a4115468138f16f48a5b850..e9f17b8a6fba476a472d3f09691611228ee9d203 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -318,7 +318,7 @@ static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (!vcpu_has_sve(vcpu)) return -ENOENT; - if (WARN_ON(!sve_vl_valid(vcpu->arch.sve_max_vl))) + if (WARN_ON(!sve_vl_valid(vcpu->arch.max_vl[ARM64_VEC_SVE]))) return -EINVAL; memset(vqs, 0, sizeof(vqs)); @@ -356,7 +356,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (vq_present(vqs, vq)) max_vq = vq; - if (max_vq > sve_vq_from_vl(kvm_sve_max_vl)) + if (max_vq > sve_vq_from_vl(kvm_max_vl[ARM64_VEC_SVE])) return -EINVAL; /* @@ -375,7 +375,7 @@ static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return -EINVAL; /* vcpu->arch.sve_state will be alloc'd by kvm_vcpu_finalize_sve() */ - vcpu->arch.sve_max_vl = sve_vl_from_vq(max_vq); + vcpu->arch.max_vl[ARM64_VEC_SVE] = sve_vl_from_vq(max_vq); return 0; } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index c24e418cec101433f7484eb4f7af788c474eda3a..2c2e22d86a2353531efa49187fe80acb01d31d20 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -370,8 +370,8 @@ static inline void __hyp_sve_save_host(void) struct cpu_sve_state *sve_state = *host_data_ptr(sve_state); sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[ARM64_VEC_SVE]), &sve_state->fpsr, true); } @@ -426,7 +426,7 @@ static inline void fpsimd_lazy_switch_to_host(struct kvm_vcpu *vcpu) zcr_el2 = vcpu_sve_max_vq(vcpu) - 1; write_sysreg_el2(zcr_el2, SYS_ZCR); } else { - zcr_el2 = sve_vq_from_vl(kvm_host_sve_max_vl) - 1; + zcr_el2 = sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1; write_sysreg_el2(zcr_el2, SYS_ZCR); zcr_el1 = vcpu_sve_max_vq(vcpu) - 1; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 2c37680d954cf2c2aed5abe7c2225b682861869a..a4ba03380d6e1049a5453772fa5291b08c1f70c1 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -34,7 +34,7 @@ static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, true); - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); } static void __hyp_sve_restore_host(void) @@ -50,8 +50,8 @@ static void __hyp_sve_restore_host(void) * that was discovered, if we wish to use larger VLs this will * need to be revisited. */ - write_sysreg_s(sve_vq_from_vl(kvm_host_sve_max_vl) - 1, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + write_sysreg_s(sve_vq_from_vl(kvm_host_max_vl[ARM64_VEC_SVE]) - 1, SYS_ZCR_EL2); + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_max_vl[ARM64_VEC_SVE]), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); @@ -125,7 +125,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + hyp_vcpu->vcpu.arch.max_vl[ARM64_VEC_SVE] = min(host_vcpu->arch.max_vl[ARM64_VEC_SVE], + kvm_host_max_vl[ARM64_VEC_SVE]); hyp_vcpu->vcpu.arch.mdcr_el2 = host_vcpu->arch.mdcr_el2; hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE); diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 5896e581c1fac943cf372f1df24311ef43529d62..2a2f932143d5e5b2164162422ba6cf0e6731b13a 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -20,7 +20,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_max_vl[ARM64_VEC_MAX]; /* * The currently loaded hyp vCPU for each physical CPU. Used only when diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index ce726b1d4e8e90cfd4459a6cb9c67b8805424e22..3cb91dc6dc3dc5cc484900dbd9f4cdfedb3e2b4a 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; /* * ARMv8 Reset Values @@ -46,14 +46,14 @@ unsigned int __ro_after_init kvm_host_sve_max_vl; #define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \ PSR_AA32_I_BIT | PSR_AA32_F_BIT) -unsigned int __ro_after_init kvm_sve_max_vl; +unsigned int __ro_after_init kvm_max_vl[ARM64_VEC_MAX]; int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { - kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_max_vl[ARM64_VEC_SVE] = sve_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SVE] = sve_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SVE]) = kvm_host_max_vl[ARM64_VEC_SVE]; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need @@ -61,16 +61,16 @@ int __init kvm_arm_init_sve(void) * order to support vector lengths greater than * VL_ARCH_MAX: */ - if (WARN_ON(kvm_sve_max_vl > VL_ARCH_MAX)) - kvm_sve_max_vl = VL_ARCH_MAX; + if (WARN_ON(kvm_max_vl[ARM64_VEC_SVE] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SVE] = VL_ARCH_MAX; /* * Don't even try to make use of vector lengths that * aren't available on all CPUs, for now: */ - if (kvm_sve_max_vl < sve_max_vl()) + if (kvm_max_vl[ARM64_VEC_SVE] < sve_max_vl()) pr_warn("KVM: SVE vector length for guests limited to %u bytes\n", - kvm_sve_max_vl); + kvm_max_vl[ARM64_VEC_SVE]); } return 0; @@ -78,7 +78,7 @@ int __init kvm_arm_init_sve(void) static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { - vcpu->arch.sve_max_vl = kvm_sve_max_vl; + vcpu->arch.max_vl[ARM64_VEC_SVE] = kvm_max_vl[ARM64_VEC_SVE]; /* * Userspace can still customize the vector lengths by writing @@ -99,7 +99,7 @@ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) size_t reg_sz; int ret; - vl = vcpu->arch.sve_max_vl; + vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; /* * Responsibility for these properties is shared between From patchwork Fri Feb 14 01:57:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865227 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F0E01C8600; Fri, 14 Feb 2025 02:01:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498517; cv=none; b=uKL1qgAkjML1NY64+va/KofbWrypXasEtOVh5+Pu65jbKHZyVIIJa3flYPj2A/XbuVVi/FqrfP52/5gL/fDzQfhbLANiltC66rIcNyRxLT7egvFHB+BOXKOqvsMbk7SVsg5vn5/L5DoHjs0XztQ2GSmngZEjdDk2L6ec059NIlc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498517; c=relaxed/simple; bh=KthAFJE+SIF0dLE8S413GLUMvpdlQLSkbZ+hUct3NSo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DQYTX2LNYMpeB8cqx3CJwSQ4tPtngPq/gV1+14G2ET19sIWW1A2CSc4ZNpzNi07MyJGuu+l0bpZhUdccb92YoUDQUQRrbeujk4UVhHc/E/qLsmE1HXhr1ZcCH93o4bp3eZPwToLmmykeX7hf6umechKfeEIs5MuRe1JiTp2oyr0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=e0pVF4r0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="e0pVF4r0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77C7AC4CEE6; Fri, 14 Feb 2025 02:01:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498516; bh=KthAFJE+SIF0dLE8S413GLUMvpdlQLSkbZ+hUct3NSo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=e0pVF4r0fPdJWZkdIAgGuco0x/z7Shv/1wDuVq3xQo5zTEOtr/C9jV/WMYh3tJ3sA u0F/4LXoqIgGTp4xXLGnIVYcWIIFeCxP716BzyzXLJEiHiVUxlIYtA1qtn0UMgEWPI k7jeHjSbNtf9Rl6/UX9I5p0Q+Vg7I97wV71kufdQhPA0mV20DYVo81xY8fKAPp3sDm ZfOC+6KXLJtmvF21LH86Rc+7cgzj1Jg3Y0C8TU8k+S4aIBQJItqpbtOI0gYu9xO6us +8NxKSXwe9ZQZX3UKwRjcs5P0/Cf72w343Ep5sR9mLBvGdod03P9jd6aCtWkLU5W/0 PCgn8Io/xhTiQ== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:58 +0000 Subject: [PATCH v4 15/27] KVM: arm64: Support SME control registers Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-15-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=4930; i=broonie@kernel.org; h=from:subject:message-id; bh=KthAFJE+SIF0dLE8S413GLUMvpdlQLSkbZ+hUct3NSo=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPMmUgNls9piz0lfXFaB9EN3Q69vysm8e9EJxGl ESzaXtiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jzAAKCRAk1otyXVSH0GfqB/ 95txGd53UM9kLA6r0WdXratLvhDq7IoeWhql2U4CS8mxzIFKOks7tBOVNqZtuyH43abC/KusibJQQq 3PgG3PXkXCQL+hKZ3Fa54FoNosibLFeU3EQIsM533fwZ08GZa8+09BeuhQKXcF+KhTts2DdVusIGAZ UHj4SrD0z9+8GffzhpLND9I6CVkq9xCnEEYrXr3YS1YxsH07XUSOceX49E8xYwDJ5IzG7bpBxxE/h7 EuUu3JIqDHFVX5cM7N9HfZabiYaH6BWLxkVMh8ZvtZghGdy6knR7XnhL2mT5Z6lexucQrlTa85yqV2 ZoS+pAvSTgwExEEMfC2jjWtSkEwr5Y X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME is configured by the system registers SMCR_EL1 and SMCR_EL2, add definitions and userspace access for them. These control the SME vector length in a manner similar to that for SVE and also have feature enable bits for SME2 and FA64. A subsequent patch will add management of them for guests as part of the general floating point context switch, as is done for the equivalent SVE registers. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 37 ++++++++++++++++++++++++++++++++++- 3 files changed, 39 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 5cc1014120ca6de249093b3e34437a99bc8bf935..f987698f88acf7b01e08e44b46a0982e36cced95 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -485,6 +485,7 @@ enum vcpu_sysreg { CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ HACR_EL2, /* Hypervisor Auxiliary Control Register */ ZCR_EL2, /* SVE Control Register (EL2) */ + SMCR_EL2, /* SME Control Register (EL2) */ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ TCR_EL2, /* Translation Control Register (EL2) */ @@ -522,6 +523,7 @@ enum vcpu_sysreg { VNCR(ACTLR_EL1),/* Auxiliary Control Register */ VNCR(CPACR_EL1),/* Coprocessor Access Control */ VNCR(ZCR_EL1), /* SVE Control */ + VNCR(SMCR_EL1), /* SME Control */ VNCR(TTBR0_EL1),/* Translation Table Base Register 0 */ VNCR(TTBR1_EL1),/* Translation Table Base Register 1 */ VNCR(TCR_EL1), /* Translation Control Register */ diff --git a/arch/arm64/include/asm/vncr_mapping.h b/arch/arm64/include/asm/vncr_mapping.h index 4f9bbd4d6c2671753124599475e5138bf6b9c749..74fc7400efbc7de6b8dd81a485f1e9d545baf7a9 100644 --- a/arch/arm64/include/asm/vncr_mapping.h +++ b/arch/arm64/include/asm/vncr_mapping.h @@ -42,6 +42,7 @@ #define VNCR_HDFGWTR_EL2 0x1D8 #define VNCR_ZCR_EL1 0x1E0 #define VNCR_HAFGRTR_EL2 0x1E8 +#define VNCR_SMCR_EL1 0x1F0 #define VNCR_TTBR0_EL1 0x200 #define VNCR_TTBR1_EL1 0x210 #define VNCR_FAR_EL1 0x220 diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 49b7844af8a19467e7842347c4b05ceb44c4caaf..597d6a33826d001268d53174581ef8e61e7dd946 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -142,6 +142,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg, MAPPED_EL2_SYSREG(ELR_EL2, ELR_EL1, NULL ); MAPPED_EL2_SYSREG(SPSR_EL2, SPSR_EL1, NULL ); MAPPED_EL2_SYSREG(ZCR_EL2, ZCR_EL1, NULL ); + MAPPED_EL2_SYSREG(SMCR_EL2, SMCR_EL1, NULL ); MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL ); default: return false; @@ -2429,6 +2430,37 @@ static bool access_zcr_el2(struct kvm_vcpu *vcpu, return true; } +static unsigned int sme_el2_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + return __el2_visibility(vcpu, rd, sme_visibility); +} + +static bool access_smcr_el2(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + unsigned int vq; + u64 smcr; + + if (guest_hyp_sve_traps_enabled(vcpu)) { + kvm_inject_nested_sve_trap(vcpu); + return true; + } + + if (!p->is_write) { + p->regval = vcpu_read_sys_reg(vcpu, SMCR_EL2); + return true; + } + + smcr = p->regval; + vq = SYS_FIELD_GET(SMCR_ELx, LEN, smcr) + 1; + vq = min(vq, vcpu_sme_max_vq(vcpu)); + vcpu_write_sys_reg(vcpu, SYS_FIELD_PREP(SMCR_ELx, LEN, vq - 1), + SMCR_EL2); + return true; +} + static unsigned int s1poe_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd) { @@ -2695,7 +2727,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility = sve_visibility }, { SYS_DESC(SYS_TRFCR_EL1), undef_access }, { SYS_DESC(SYS_SMPRI_EL1), undef_access }, - { SYS_DESC(SYS_SMCR_EL1), undef_access }, + { SYS_DESC(SYS_SMCR_EL1), NULL, reset_val, SMCR_EL1, 0, .visibility = sme_visibility }, { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 }, { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 }, { SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 }, @@ -3047,6 +3079,9 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG_VNCR(HCRX_EL2, reset_val, 0), + EL2_REG_FILTERED(SMCR_EL2, access_smcr_el2, reset_val, 0, + sme_el2_visibility), + EL2_REG(TTBR0_EL2, access_rw, reset_val, 0), EL2_REG(TTBR1_EL2, access_rw, reset_val, 0), EL2_REG(TCR_EL2, access_rw, reset_val, TCR_EL2_RES1), From patchwork Fri Feb 14 01:58:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865226 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEE3A1519B1; Fri, 14 Feb 2025 02:02:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498525; cv=none; b=ZghO3GETih1SpCh8I0o8gkKIERt7TLW+nKmQhrp3TH8ERfEmGcnZUt01W8LEVKEAr62FCOTIgNKEkkqnTZCNtMv9nkPLTrBPIYubBsWahaKUFzyowt7EOMiDiHU59KT6VsVaZQLotaf2jLCqNqrAsARVo5f6ptfr5aJjGOhxTqw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498525; c=relaxed/simple; bh=6ZKeFmUwpIKsEmKKV2CccS5FiLEKpUEnHjqvTyJuW8I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=F1dpkgYo5vg/Z7B/M5lNEFlGcDxl0GdEJABgdSjclRG2PUlSU2/zPAAySlWDg+9OEaCgm4YKlx6mGIQwYi06SeJF0fyKRV4vo9T7d+zq5HtSNJaA5xvSblZSdrS87WpYJyiEYszcxImBo72LPuCtciUvxR23MIgSPkWraxK49hY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OiPMHHkl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OiPMHHkl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 131ABC4CEEB; Fri, 14 Feb 2025 02:02:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498524; bh=6ZKeFmUwpIKsEmKKV2CccS5FiLEKpUEnHjqvTyJuW8I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=OiPMHHkl1djeZ2lB8dMWaekSizYcbPZLeW5EMFvwXWfJxlVr9Ozru6+nq2Vt956Dd eZAOBTEEzW5IlTH03xQjanj/hRWglpLoT5ibBAEkfYvQECt7mt5o/5rJzl24JBfjz6 LTuxf6R9llGGhYjv/pmMjhZABXqtDZvz6rwdRGQnSXkp+8eLae/tj4ztdPL95S4jOF 5HvR0NxLmqhOmwa2+tJ5fB5ordXobTHaxdqzwMHFYyBNUE2fMynmKUBeKImP1utZ3b heUbWGv5x5gm8V/7NRr5xsJ6LkOWzhXNMcvmj6JhbjISVm3i2LE3lV0TDAwzCVefte RJvsEqNAHETnw== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:00 +0000 Subject: [PATCH v4 17/27] KVM: arm64: Support SME identification registers for guests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-17-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5505; i=broonie@kernel.org; h=from:subject:message-id; bh=6ZKeFmUwpIKsEmKKV2CccS5FiLEKpUEnHjqvTyJuW8I=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPN/4geYRKPPjBRvDXStddXp7T70d00Uznyao8d rOjSP3yJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jzQAKCRAk1otyXVSH0EzoB/ 9/03HAOXhIxlxxSR+RKChEm5O/Z3N4fyOwjIKdkaCQn7Dzkg6V/2YspMfs/9IbbqAecMoM/9iEvRSN 6U0yD5lZ/m6NenVl7xzYp/kRXl2LlEFrfuv8ijKIjqmX4ImidimNNSfRrE0bL6fYi7pxz7ILfJS1k5 0S0nGl1EXojymVEjL7pEesXPDahAu0wVCaHqxf4x5SRXwqulPyfWVb4xHYDoTdN2/Xd0UU+CabDWsV 7H/4tWHAKsDLS3gWz3s6e96TQ7n82t5M4uDDrdEsW99RPuQbN1a9TPYpwO6Mz5GeFTjKQeHlrIaKu/ v1DLIaSlaYDoRJ73v+bzA7czGM9udK X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The primary register for identifying SME is ID_AA64PFR1_EL1.SME. This is hidden from guests unless SME is enabled by the VMM. When it is visible it is writable and can be used to control the availability of SME2. There is also a new register ID_AA64SMFR0_EL1 which we make writable, forcing it to all bits 0 if SME is disabled. This includes the field SMEver giving the SME version, userspace is responsible for ensuring the value is consistent with ID_AA64PFR1_EL1.SME. It also includes FA64, a separately enableable extension which provides the full FPSIMD and SVE instruction set including FFR in streaming mode. Userspace can control the availability of FA64 by writing to this field. The other features enumerated there only add new instructions, there are no architectural controls for these. There is a further identification register SMIDR_EL1 which provides a basic description of the SME microarchitecture, in a manner similar to MIDR_EL1 for the PE. It also describes support for priority management and a basic affinity description for shared SME units, plus some RES0 space. We do not support priority management and affinity is not meaningful for guests so we mask out everything except for the microarchitecture description. As for MIDR_EL1 and REVIDR_EL1 we expose the implementer and revision information to guests with the raw value from the CPU we are running on, this may present issues for asymmetric systems or for migration as it does for the existing registers. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/kvm/sys_regs.c | 46 +++++++++++++++++++++++++++++++++++---- 2 files changed, 43 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index c770ed6138164fcd3e11b8517ef4120b4f4486b9..14615216845668bf1a614e9fbdb6112190fec054 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -473,6 +473,7 @@ enum vcpu_sysreg { /* FP/SIMD/SVE */ SVCR, FPMR, + SMIDR_EL1, /* Streaming Mode Identification Register */ /* 32bit specific registers. */ DACR32_EL2, /* Domain Access Control Register */ diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index eece67141480b8d4bbd2bac0f02f9208c7f86f8b..b515dfa1f27d0c744de996f6dcdff272aa184385 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -767,6 +767,38 @@ static u64 reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) return mpidr; } +static u64 reset_smidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) +{ + u64 smidr = 0; + + if (!system_supports_sme()) + return smidr; + + smidr = read_sysreg_s(SYS_SMIDR_EL1); + + /* + * Mask out everything except for the implementer and revison, + * in particular priority management is not implemented. + */ + smidr &= SMIDR_EL1_IMPLEMENTER_MASK | SMIDR_EL1_REVISION_MASK; + + vcpu_write_sys_reg(vcpu, smidr, SMIDR_EL1); + + return smidr; +} + +static bool access_smidr(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + if (p->is_write) + return write_to_read_only(vcpu, p, r); + + p->regval = vcpu_read_sys_reg(vcpu, r->reg); + + return true; +} + static unsigned int pmu_visibility(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) { @@ -1593,7 +1625,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, if (!kvm_has_mte(vcpu->kvm)) val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_MTE); - val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + if (!vcpu_has_sme(vcpu)) + val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_SME); + val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_RNDR_trap); val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_NMI); val &= ~ARM64_FEATURE_MASK(ID_AA64PFR1_EL1_MTE_frac); @@ -1697,6 +1731,10 @@ static unsigned int id_visibility(const struct kvm_vcpu *vcpu, if (!vcpu_has_sve(vcpu)) return REG_RAZ; break; + case SYS_ID_AA64SMFR0_EL1: + if (!vcpu_has_sme(vcpu)) + return REG_RAZ; + break; } return 0; @@ -2637,7 +2675,6 @@ static const struct sys_reg_desc sys_reg_descs[] = { ID_AA64PFR1_EL1_MTE_frac | ID_AA64PFR1_EL1_NMI | ID_AA64PFR1_EL1_RNDR_trap | - ID_AA64PFR1_EL1_SME | ID_AA64PFR1_EL1_RES0 | ID_AA64PFR1_EL1_MPAM_frac | ID_AA64PFR1_EL1_RAS_frac | @@ -2645,7 +2682,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { ID_WRITABLE(ID_AA64PFR2_EL1, ID_AA64PFR2_EL1_FPMR), ID_UNALLOCATED(4,3), ID_WRITABLE(ID_AA64ZFR0_EL1, ~ID_AA64ZFR0_EL1_RES0), - ID_HIDDEN(ID_AA64SMFR0_EL1), + ID_WRITABLE(ID_AA64SMFR0_EL1, ~ID_AA64SMFR0_EL1_RES0), ID_UNALLOCATED(4,6), ID_WRITABLE(ID_AA64FPFR0_EL1, ~ID_AA64FPFR0_EL1_RES0), @@ -2845,7 +2882,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_CLIDR_EL1), access_clidr, reset_clidr, CLIDR_EL1, .set_user = set_clidr, .val = ~CLIDR_EL1_RES0 }, { SYS_DESC(SYS_CCSIDR2_EL1), undef_access }, - { SYS_DESC(SYS_SMIDR_EL1), undef_access }, + { SYS_DESC(SYS_SMIDR_EL1), .access = access_smidr, .reset = reset_smidr, + .reg = SMIDR_EL1, .visibility = sme_visibility }, { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, ID_FILTERED(CTR_EL0, ctr_el0, CTR_EL0_DIC_MASK | From patchwork Fri Feb 14 01:58:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865225 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A1DF1DE89E; Fri, 14 Feb 2025 02:02:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498532; cv=none; b=tpu3zJqeefFT1R8A/JDC2vrzMIjcPek+4osaJjmxIu3zBMBNKeHSV7P1wo5a3SiVZoPAkOm9BvyBafjvYrx7rP2ZYv4aee/37269MXtF/ce+TO01Mccitj9DwEPHai3iNW35f1yft2hzKL132aRtVXeuaUOD02d46SsyvmoiQt8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498532; c=relaxed/simple; bh=hTVDnp0p6nmD43kAvlLpMQz5gcUT21vSNtNfafanNjA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CQQ2/ePH/PDqNl+yAncPhvJBs7MZ6jPSKlmbC2LjjmuBM8tb8wXSPGlajHwvoYE05XjZgNOVclVayKD0DIpP/2dRoRbqdutkhDhW0U9Y5+FZoF4a25vx2Gih36Tr9BoUucJKFLE7MC5QfWL53x+SCd9abRLa2Xs/8/zuR6almLY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ePLg2gQu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ePLg2gQu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9701CC4CEE8; Fri, 14 Feb 2025 02:02:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498531; bh=hTVDnp0p6nmD43kAvlLpMQz5gcUT21vSNtNfafanNjA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ePLg2gQu1d/23ahI4SHCbAFiOWOxSSpo2p9SvZHxjTr11X9256HuK4tkbcp9JNg7W mceo5hZqD2rwHroxUagmvA1KZKf/dMkS4nkDbmej80RcgygY79V8BBSvJ06ESsWjlH J45nJGqL2f+3AqdKkvXh6Xsdicbuw/aU9lga+X7A0S/Ctm4WBcOojqxWiE5H6B+kUd q08272WxckRA/6z98dbYdA2+t8lprgILaEGu+pI/5or6qIMUsDHSI3kmcCQ8un+yze 9l1g54vsIVnrPjqQNTl/h8mbpeFOM/JZqg3+BC3laVc/3EmoKqMIw11ZowU9ImEyQ7 /86j441616Zlw== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:02 +0000 Subject: [PATCH v4 19/27] KVM: arm64: Provide assembly for SME state restore Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-19-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=1651; i=broonie@kernel.org; h=from:subject:message-id; bh=hTVDnp0p6nmD43kAvlLpMQz5gcUT21vSNtNfafanNjA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPPqEmHRwyJlV+4taSwkW14T+B7Fla3GYl2iXUU d/Vh+z+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jzwAKCRAk1otyXVSH0BVyB/ 94PGxLphmD7LasFr7JLf4xmHrwfbkexncRmrlZKWhA+U8fKTy+dHyqOhsJAeU1uT6rL09lrOozq6vb b3pUGAZ3VCoO705G404h4QBPBi7Dlzu3aO9NdfyiJSTz+ATFeC/d1iSutGSgH0TFDYY+kVnHR4tFqF cTH6A8MdHa4o4H6iNGJChQsWc1ctwb2MUhaRUp6iu1dYF/lkhklXERxhqRj4LLGcxVGI2zgPL/BS/6 TE7nKBN9b4DGJtsV/2HeHzVXwUxLb9EAymsIl60BBi/V+qLbuLa5A0TrqEISq/DQ0Brk1/N+BKXt8x CteX9zKpoXnpiqlyz5R7Z8mIkFCKom X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a __sme_restore_state() for the hypervisor to allow it to restore ZA and ZT for guests. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 2 ++ arch/arm64/kvm/hyp/fpsimd.S | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 21943cb98542750a1b626a8de6bbc095d7770ccf..5a1f8e4be18624efa6b887f09c36f0e8ad318c40 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,8 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); +void __sme_restore_state(void const *state, bool restore_zt); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index 6e16cbfc5df27e63732655dffea60d7039c37332..b21809adfba78522410da053fd4eccbb66484532 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -29,3 +29,19 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) + +SYM_FUNC_START(__sme_restore_state) + _sme_rdsvl 2, 1 // x2 = VL/8 + sme_load_za 0, x2, 12 // Leaves x0 pointing to end of ZA + + cbz x1, 1f + _ldr_zt 0 + +1: + ret +SYM_FUNC_END(__sme_restore_state) From patchwork Fri Feb 14 01:58:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865224 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ACCE1FC10D; Fri, 14 Feb 2025 02:02:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498539; cv=none; b=KvaBtRqTxqdKGqDDpBrZmtll0SZGmbK6iLM5+Qc5dRAIS8eIL3a9Nrsv7fUtXeQRVgpULJkxL9FPrsZWGrMzH1YapVamtFlHU7Vdo5UVfXr14GUUg2z2sJYrFs3xTBRdu5l6XqM8c4QbpJMzsgS+5xlzD904Hux2sDIgNbaIj68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498539; c=relaxed/simple; bh=fnCQMptN1ycDRPB+hsWu9oBxkCMnhPzAMfclA65DM6Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CZK7qjsZGLge4U8hsRPXCbvufAJx4CA4Y2tB0kkmo6ACWD3InFh+JQ1XWaVPLx88frUJ7m3c+RO9whOGnlLVBjn8tb+uoiRM7RTAAXB0el1PegiNqxulNOQFDWk2Hk2U0XUO3O7ski/6xD1tT397HjLXSjAcWk0bO6ObthAdbdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FBV9xS6a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FBV9xS6a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 206ABC4CEE5; Fri, 14 Feb 2025 02:02:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498539; bh=fnCQMptN1ycDRPB+hsWu9oBxkCMnhPzAMfclA65DM6Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FBV9xS6advEsW5GLOU0FDPSCV33IEBjgVcG4VpPNDOnh3/IPo85wpJgWVmtvPssnA Swaix4faswwQxyCVSufhzsK0XMzB3AbXFWOhttEXr9FMxFptDZ1cTn+v7oCiB8joPS smv0lL0USmqMqWQuT/G5+IIBMxyS2dnu3M89fMdbvbfK8RboQlurcBCdUzH6lbwVqr QqkjonlTlES4Nl/5zj9n4U9iQg9U0DpDu62zijj9H4VxFK7R+PSU8dBFK4PrrYjDvD XfSvAzZbhz0JwcB7EHAZ760YCfQu2HkoKR2jVJ7MP6w7SfBGWEE23A+u5ELdrLOZ1M C+y0jHPnD41KQ== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:04 +0000 Subject: [PATCH v4 21/27] KVM: arm64: Expose SME specific state to userspace Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-21-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=8323; i=broonie@kernel.org; h=from:subject:message-id; bh=fnCQMptN1ycDRPB+hsWu9oBxkCMnhPzAMfclA65DM6Y=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPQ2aiAXrCUt457WEMctVDTOFzvg9+orir6Rf1F K99jIKSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66j0AAKCRAk1otyXVSH0B0mB/ 45/HObhST/tz6QF+/Wy8m1myRbqdAJbH1tEt2o+OkO//j+eVrqojveCXn8bV4Jk6+OXfKGfD+tMk4U hNKolDasx0BVMhd12H9MHM4GlCXKpkZoHa2qojvwXpz72qa6pbHC3hEB6j3Tot2qL78fZ4zQYDAO3l 7N0T96/00F5w00FtbDOf+/2E4oBY85o+aDNh+0HCOPJ/nd5NAMwl8UElVsR/gAhnGxKf/JuJBP7t2Z O5G/wrto/TRRVOrehW629wxdxccYUsMZQQmC6zDilkulZ+8ddTddTctmWU0MV84aj1zDf3vehAbgre Jg5YzZ54+9TxkHsxwZZibo1lXM62Pm X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB SME introduces two new registers, the ZA matrix register and the ZT0 LUT register. Both of these registers are only accessible when PSTATE.ZA is set and ZT0 is only present if SME2 is enabled for the guest. Provide support for configuring these from VMMs. The ZA matrix is a single SVL*SVL register which is available when PSTATE.ZA is set. We follow the pattern established by the architecture itself and expose this to userspace as a series of horizontal SVE vectors with the streaming mode vector length, using the format already established for the SVE vectors themselves. ZT0 is a single register with a refreshingly fixed size 512 bit register which is like ZA accessible only when PSTATE.ZA is set. Add support for it to the userspace API, as with ZA we allow the register to be read or written regardless of the state of PSTATE.ZA in order to simplify userspace usage. The value will be reset to 0 whenever PSTATE.ZA changes from 0 to 1, userspace can read stale values but these are not observable by the guest without manipulation of PSTATE.ZA by userspace. While there is currently only one ZT register the naming as ZT0 and the instruction encoding clearly leave room for future extensions adding more ZT registers. This encoding can readily support such an extension if one is introduced. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 21 +++++++ arch/arm64/include/uapi/asm/kvm.h | 17 ++++++ arch/arm64/kvm/guest.c | 114 +++++++++++++++++++++++++++++++++++++- 3 files changed, 150 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 67ac3e37fde31901513c2e692693d3ba1967b0bd..0b00b1e42b668367b89d7ffee2839fc53c14e753 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1020,6 +1020,24 @@ struct kvm_vcpu_arch { __size_ret; \ }) +#define vcpu_sme_state(vcpu) (kern_hyp_va((vcpu)->arch.sme_state)) + +#define vcpu_sme_state_size(vcpu) ({ \ + size_t __size_ret; \ + unsigned int __vcpu_vq; \ + \ + if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SME]))) { \ + __size_ret = 0; \ + } else { \ + __vcpu_vq = vcpu_sme_max_vq(vcpu); \ + __size_ret = ZA_SIG_REGS_SIZE(__vcpu_vq); \ + if (system_supports_sme2()) \ + __size_ret += ZT_SIG_REG_SIZE; \ + } \ + \ + __size_ret; \ +}) + /* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the * memory backed version of a register, and not the one most recently @@ -1603,4 +1621,7 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); #define vcpu_in_streaming_mode(vcpu) \ (__vcpu_sys_reg(vcpu, SVCR) & SVCR_SM_MASK) +#define vcpu_za_enabled(vcpu) \ + (__vcpu_sys_reg(vcpu, SVCR) & SVCR_ZA_MASK) + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index ff7c4bcee542befae6961fae9eec2eb9bc8e2eb3..1f75c9938f313e70a82d9792cb3c89caec2076a1 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -356,6 +356,23 @@ struct kvm_arm_counter_offset { /* SME registers */ #define KVM_REG_ARM64_SME (0x17 << KVM_REG_ARM_COPROC_SHIFT) +#define KVM_ARM64_SME_VQ_MIN __SVE_VQ_MIN +#define KVM_ARM64_SME_VQ_MAX __SVE_VQ_MAX + +/* ZA and ZTn occupy blocks at the following offsets within this range: */ +#define KVM_REG_ARM64_SME_ZA_BASE 0 +#define KVM_REG_ARM64_SME_ZT_BASE 0x600 + +#define KVM_ARM64_SME_MAX_ZAHREG (__SVE_VQ_BYTES * KVM_ARM64_SME_VQ_MAX) + +#define KVM_REG_ARM64_SME_ZAHREG(n, i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SME | KVM_REG_ARM64_SME_ZA_BASE | \ + KVM_REG_SIZE_U2048 | \ + (((n) & (KVM_ARM64_SME_MAX_ZAHREG - 1)) << 5) | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_REG_ARM64_SME_ZTREG_SIZE (512 / 8) + /* Vector lengths pseudo-register: */ #define KVM_REG_ARM64_SME_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SME | \ KVM_REG_SIZE_U512 | 0xffff) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 87a0a027e1e7e2487eff703dee2958137c9640ef..55459d1e7586291f8384e8073b5de2d6a8f5f2d9 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -600,23 +600,133 @@ static int set_sme_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) return set_vec_vls(ARM64_VEC_SME, vcpu, reg); } +/* + * Validate SVE register ID and get sanitised bounds for user/kernel SVE + * register copy + */ +static int sme_reg_to_region(struct vec_state_reg_region *region, + struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg) +{ + /* reg ID ranges for ZA.H[n] registers */ + unsigned int vq = vcpu_sme_max_vq(vcpu) - 1; + const u64 za_h_max = vq * __SVE_VQ_BYTES; + const u64 zah_id_min = KVM_REG_ARM64_SME_ZAHREG(0, 0); + const u64 zah_id_max = KVM_REG_ARM64_SME_ZAHREG(za_h_max - 1, + SVE_NUM_SLICES - 1); + unsigned int reg_num; + + unsigned int reqoffset, reqlen; /* User-requested offset and length */ + unsigned int maxlen; /* Maximum permitted length */ + + size_t sme_state_size; + + reg_num = (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT; + + if (reg->id >= zah_id_min && reg->id <= zah_id_max) { + if (!vcpu_has_sme(vcpu) || (reg->id & SVE_REG_SLICE_MASK) > 0) + return -ENOENT; + + /* ZA is exposed as SVE vectors ZA.H[n] */ + reqoffset = ZA_SIG_ZAV_OFFSET(vq, reg_num) - + ZA_SIG_REGS_OFFSET; + reqlen = KVM_SVE_ZREG_SIZE; + maxlen = SVE_SIG_ZREG_SIZE(vq); + } else if (reg->id == KVM_REG_ARM64_SME_ZT_BASE) { + /* ZA is exposed as SVE vectors ZA.H[n] */ + if (!kvm_has_feat(vcpu->kvm, ID_AA64PFR1_EL1, SME, SME2) || + (reg->id & SVE_REG_SLICE_MASK) > 0 || + reg_num > 0) + return -ENOENT; + + /* ZT0 is stored after ZA */ + reqlen = KVM_REG_ARM64_SME_ZTREG_SIZE; + maxlen = KVM_REG_ARM64_SME_ZTREG_SIZE; + } else { + return -EINVAL; + } + + sme_state_size = vcpu_sme_state_size(vcpu); + if (WARN_ON(!sme_state_size)) + return -EINVAL; + + region->koffset = array_index_nospec(reqoffset, sme_state_size); + region->klen = min(maxlen, reqlen); + region->upad = reqlen - region->klen; + + return 0; +} + +/* + * ZA is exposed as an array of horizontal vectors with the same + * format as SVE, mirroring the architecture's LDR ZA[Wv, offs], [Xn] + * instruction. + */ + static int get_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { + int ret; + struct vec_state_reg_region region; + char __user *uptr = (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id == KVM_REG_ARM64_SME_VLS) return get_sme_vls(vcpu, reg); - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret = sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } static int set_sme_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) { + int ret; + struct vec_state_reg_region region; + char __user *uptr = (char __user *)reg->addr; + /* Handle the KVM_REG_ARM64_SME_VLS pseudo-reg as a special case: */ if (reg->id == KVM_REG_ARM64_SME_VLS) return set_sme_vls(vcpu, reg); - return -EINVAL; + /* Try to interpret reg ID as an architectural SME register... */ + ret = sme_reg_to_region(®ion, vcpu, reg); + if (ret) + return ret; + + if (!kvm_arm_vcpu_vec_finalized(vcpu)) + return -EPERM; + + /* + * None of the SME specific registers are accessible unless + * PSTATE.ZA is set. + */ + if (!vcpu_za_enabled(vcpu)) + return -EINVAL; + + if (copy_from_user(vcpu->arch.sme_state + region.koffset, uptr, + region.klen)) + return -EFAULT; + + return 0; } + int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) { return -EINVAL; From patchwork Fri Feb 14 01:58:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865223 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31C9615748F; Fri, 14 Feb 2025 02:02:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498547; cv=none; b=C9cMjo5A2EvD5o0FedCcE4wYD7x8oJ2364IoUWRdXfM1/x0nykNj8tAbRGZF3XxaSIGgMrf+YmFquUPGn2lnMnI25fgPuvpvfTC/E3Nl3jBb9xHM3rL6ToqGKYKdkbQw/2HoiaPnb/okoTq/NHYeAwefgi6nPNBlb8OX+VBs3Xc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498547; c=relaxed/simple; bh=nqYjBb6f0F7ndi21l9+vc1B1CfMbKXQsnrY9kauVFRE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iUghMipXsrfCl01n1WTt8VKjB6wqfBbNxwr22BvC4vFNsgKBuqZd6C4gbNfKxhxSi2zVRkEYWKbOjkJL9DxtCKtWgM2jV2aEluvbA0DR8qGqBcE18pPEOk0mBNmVP1VdPkLAybeCo0MfeRG3RDXGO3jacRD674QPHxRsa2f49Jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lLHp4LjO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lLHp4LjO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABBF1C4CEEF; Fri, 14 Feb 2025 02:02:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498547; bh=nqYjBb6f0F7ndi21l9+vc1B1CfMbKXQsnrY9kauVFRE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lLHp4LjOqCUtf6L7oV7QbU1gZf1DNpCoUPxZjOIapuAYIuHU663js3YloJVtYjbZf 97Ggwfsebh3nzzDfhJ7Z6P7nieWYBPNpD0bFXFvOzOCCm0TClJ9gW+5N2pdUqylMma HUKQNicdN85tgu2HhrZgnXWWsMQnj6q1ln4A9P4xDw6KprXbUtmzZgukbtLyJgEeeX kJ41YcoudnxBMLpQlBea0/zCdDT90nojjADbObH6q2HqZX52Pm5BObgZnKoLLV7dw6 lqWxuyKc2Ps3oVX6RM8vETnsB+rii2HzBmmTL2vJ97zf6oPSe3xe/6Qf9ho6hTR9Tr IC8wIW9LLJRng== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:06 +0000 Subject: [PATCH v4 23/27] KVM: arm64: Handle SME exceptions Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-23-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=6490; i=broonie@kernel.org; h=from:subject:message-id; bh=nqYjBb6f0F7ndi21l9+vc1B1CfMbKXQsnrY9kauVFRE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPS+bT/31ABzQeA6aimPKYsZeU8ztn8c7lwhppv Kgww86WJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66j0gAKCRAk1otyXVSH0KtoB/ 9pwQzgkosbOPi2N66QXkMJi2imbqozvBuXpsKvfWYXE6SowpEZtsfKrI9s8PZUvLJgA6U49pDLsf0a 4dJtHdFYpk6v4DKXu/pQbxK8NYHDmJJgzxIVDn4ocb7wzxihUNJxmVoKnJ3Q+Y3Yq0OVAmkRU9yDiH +QWnfVmmzJNgd4k9ruMGQdolYG4CUp6PnOlfu0kXy3VLRGeSZ5kfMslQvdwWZOIq2RqDvXnswomCkO S3aFE7ZT7k6cx3nWXKxEzb1iTjTs7Jk+75FMoJPbf4XTVt/FlQcZ7zYLoCAdvQRHoo7q3NE4UVQWMs EK0W8humU1PpQ1CxjX41yJlkt94TQP X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The access control for SME follows the same structure as for the base FP and SVE extensions, with control being via CPACR_ELx.SMEN and CPTR_EL2.TSM mirroring the equivalent FPSIMD and SVE controls in those registers. Add handling for these controls and exceptions mirroring the existing handling for FPSIMD and SVE. When the hardware is in streaming mode guest operations that are invalid in in streaming mode will generate SME exceptions. Since these exceptions may be routed to EL1 with no opportunity for the hypervisor to intercept them we already have code which ensures that we exit streaming mode before running the guest. This ensures that guests do not receive unexpected SME exceptions. Signed-off-by: Mark Brown --- arch/arm64/kvm/handle_exit.c | 14 ++++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 11 ++++++----- arch/arm64/kvm/hyp/vhe/switch.c | 21 ++++++++++++++++----- 3 files changed, 36 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 512d152233ff20eb6c65abbf731b4bcb543a7d07..036e722a8f3ce03f0bd756a84a6fbffeae9bf359 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -227,6 +227,19 @@ static int handle_sve(struct kvm_vcpu *vcpu) return 1; } +/* + * Guest access to SME registers should be routed to this handler only + * when the system doesn't support SME. + */ +static int handle_sme(struct kvm_vcpu *vcpu) +{ + if (guest_hyp_sme_traps_enabled(vcpu)) + return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu)); + + kvm_inject_undefined(vcpu); + return 1; +} + /* * Two possibilities to handle a trapping ptrauth instruction: * @@ -310,6 +323,7 @@ static exit_handle_fn arm_exit_handlers[] = { [ESR_ELx_EC_SVC64] = handle_svc, [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg, [ESR_ELx_EC_SVE] = handle_sve, + [ESR_ELx_EC_SME] = handle_sme, [ESR_ELx_EC_ERET] = kvm_handle_eret, [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort, [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort, diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index eaeda9c8a1aa62f8f7eed36dbedc74513a453e4e..1f1883bc6f0cc270dd1ad38f96b9a0048e238e72 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -49,17 +49,16 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val |= CPACR_EL1_FPEN; if (vcpu_has_sve(vcpu)) val |= CPACR_EL1_ZEN; + if (vcpu_has_sme(vcpu)) + val |= CPACR_EL1_SMEN; } write_sysreg(val, cpacr_el1); } else { val |= CPTR_EL2_TTA | CPTR_NVHE_EL2_RES1; - /* - * Always trap SME since it's not supported in KVM. - * TSM is RES1 if SME isn't implemented. - */ - val |= CPTR_EL2_TSM; + if (!vcpu_has_sme(vcpu) || !guest_owns_fp_regs()) + val |= CPTR_EL2_TSM; if (!vcpu_has_sve(vcpu) || !guest_owns_fp_regs()) val |= CPTR_EL2_TZ; @@ -222,6 +221,7 @@ static const exit_handler_fn hyp_exit_handlers[] = { [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg, [ESR_ELx_EC_SVE] = kvm_hyp_handle_fpsimd, + [ESR_ELx_EC_SME] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] = kvm_hyp_handle_dabt_low, @@ -233,6 +233,7 @@ static const exit_handler_fn pvm_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_SYS64] = kvm_handle_pvm_sys64, [ESR_ELx_EC_SVE] = kvm_handle_pvm_restricted, + [ESR_ELx_EC_SME] = kvm_handle_pvm_restricted, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, [ESR_ELx_EC_DABT_LOW] = kvm_hyp_handle_dabt_low, diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 647737d6e8d0b5f41b2a8d25a06265e4703126b3..44e3f0577ceea2a1574038c97fd769fef30d7098 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -83,6 +83,8 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val |= CPACR_EL1_FPEN; if (vcpu_has_sve(vcpu)) val |= CPACR_EL1_ZEN; + if (vcpu_has_sme(vcpu)) + val |= CPACR_EL1_SMEN; } else { __activate_traps_fpsimd32(vcpu); } @@ -126,6 +128,8 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) val &= ~CPACR_EL1_FPEN; if (!(SYS_FIELD_GET(CPACR_EL1, ZEN, cptr) & BIT(0))) val &= ~CPACR_EL1_ZEN; + if (!(SYS_FIELD_GET(CPACR_EL1, SMEN, cptr) & BIT(0))) + val &= ~CPACR_EL1_SMEN; if (kvm_has_feat(vcpu->kvm, ID_AA64MMFR3_EL1, S2POE, IMP)) val |= cptr & CPACR_EL1_E0POE; @@ -486,22 +490,28 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } -static bool kvm_hyp_handle_zcr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) +static bool kvm_hyp_handle_vec_cr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) { u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); if (!vcpu_has_nv(vcpu)) return false; - if (sysreg != SYS_ZCR_EL2) + switch (sysreg) { + case SYS_ZCR_EL2: + case SYS_SMCR_EL2: + break; + default: return false; + } if (guest_owns_fp_regs()) return false; /* - * ZCR_EL2 traps are handled in the slow path, with the expectation - * that the guest's FP context has already been loaded onto the CPU. + * ZCR_EL2 and SMCR_EL2 traps are handled in the slow path, + * with the expectation that the guest's FP context has + * already been loaded onto the CPU. * * Load the guest's FP context and unconditionally forward to the * slow path for handling (i.e. return false). @@ -521,7 +531,7 @@ static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *vcpu, u64 *exit_code) if (kvm_hyp_handle_cpacr_el1(vcpu, exit_code)) return true; - if (kvm_hyp_handle_zcr_el2(vcpu, exit_code)) + if (kvm_hyp_handle_vec_cr_el2(vcpu, exit_code)) return true; return kvm_hyp_handle_sysreg(vcpu, exit_code); @@ -531,6 +541,7 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg_vhe, + [ESR_ELx_EC_SME] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_SVE] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, From patchwork Fri Feb 14 01:58:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865222 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BFA0155747; Fri, 14 Feb 2025 02:02:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498555; cv=none; b=tyYIk2+UQUC++9nRvhwnc6s9b2CxmvPBrw4/s+jpNnbfQvDcQE01lT5hPOjqMaxWCw2vdcQDleJfrTpxPZGBsT4pFu0G1GEBQSviQndVOVFmRjFqQSbP6MSmk05fwRw9kCF36qSAYbxtRScnF/cgce8q5lsjlZIc7JM/nBwFS0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498555; c=relaxed/simple; bh=5vUNsqQzFpYR2hxlnfok0ZSEZ+Jmv+A8lBVNLytG89w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CtvywUyEJsMriePtHPFdCbp5VKFz8J7c8BAUhSqoz+OkUGYqGbPpQuDDrMUo7iyLoHWGU9nGl519zpltsiNkYf4VZxbCfI7SZ/HidaDQRDxjtckayH/d0DHwXXLWbrqxTfz4j/6zd7xVnZwg3gY1MjV3GEiOD3CdTXumQTRmIPw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gpzdxS+7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gpzdxS+7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55F4EC4CED1; Fri, 14 Feb 2025 02:02:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498554; bh=5vUNsqQzFpYR2hxlnfok0ZSEZ+Jmv+A8lBVNLytG89w=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=gpzdxS+7Kj8Cb6dbdwkOAG3nCa/7Vr+7DW9xjVWa4H98Y4GwbR/E9ZEHj+VdBVoyb kmLuCH930CYdT22ulGeRi/UYTApkrTO0xD94n8EUNyDx2jqDisj8s8uGnhMLd7CSb5 787qqSHdoVMttebU8ZIHtG3k/GJlnLR3zDa/ZZlyXKvWogZTG9bTyNBz7zs9bdvbvY eM/f4c7NBEvrW1q3Yn+Pc+2f0lFQV1lB78Qro+UvYUfulh5Hz1iDF49ES2xZS+fJoC ywTXi7YnSOrw07lo5v4bj/xQnr1h/F6o5W4ZFmlDEjYV7A8dqfjLtsgP8dac5HSpAi cQUtsq01RNw5g== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:08 +0000 Subject: [PATCH v4 25/27] KVM: arm64: Provide interface for configuring and enabling SME for guests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-25-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=9724; i=broonie@kernel.org; h=from:subject:message-id; bh=5vUNsqQzFpYR2hxlnfok0ZSEZ+Jmv+A8lBVNLytG89w=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPTuXyW3nhrYqVz1LuQ8Do9radbsfrieoo5vNr3 KvwDi8GJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66j0wAKCRAk1otyXVSH0KbfB/ 0XzmrXv9SaksLxAKn+GDwGve+KZTCP88ni0EHELVtnt3I6zpzClmYbfVy2ZUyjmB0Er21on482fVsv 240FIebYfxfEbUTPlqU0sWOsKmkRZrdggA+5Bv8pVhVIg19qj37V9+3GBhN3PgWdkpP4vdXKfAmZiG 6iHUHrPBCvI34al/un6pMhwGCm+3pX+Lkp2dmOfx3scDPTtVZusOiXP2hfiB46d524T/Y0BXwbh3em PAjb2Z+gh8pKvdxksMsfxhU3y7nN6SthypSKnY7gomXomK2ANaSyO5mIbEmXe1RAD7c6zzJ+5hdd9I N82DEFGLgQOPI+eyY94RJniItMTgkU X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Since SME requires configuration of a vector length in order to know the size of both the streaming mode SVE state and ZA array we implement a capability for it and require that it be enabled and finalized before the SME specific state can be accessed, similarly to SVE. Due to the overlap with sizing the SVE state we finalise both SVE and SME with a single finalization, preventing any further changes to the SVE and SME configuration once KVM_ARM_VCPU_VEC (an alias for _VCPU_SVE) has been finalised. This is not a thing of great elegance but it ensures that we never have a state where one of SVE or SME is finalised and the other not, avoiding complexity. Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 5 +- arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arm.c | 10 ++++ arch/arm64/kvm/reset.c | 114 ++++++++++++++++++++++++++++++++------ include/uapi/linux/kvm.h | 1 + 5 files changed, 111 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index bbab22688a465dc0b5a4463cfaf061612b550fbd..ab7f0cddfcd2b57b742fd5193ebe44cbb8ca8677 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -79,6 +79,7 @@ extern unsigned int __ro_after_init kvm_host_max_vl[ARM64_VEC_MAX]; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); int __init kvm_arm_init_sve(void); +int __init kvm_arm_init_sme(void); u32 __attribute_const__ kvm_target_cpu(void); void kvm_reset_vcpu(struct kvm_vcpu *vcpu); @@ -1013,10 +1014,10 @@ struct kvm_vcpu_arch { size_t __size_ret; \ unsigned int __vcpu_vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.max_vl[ARM64_VEC_SVE]))) { \ + if (WARN_ON(!sve_vl_valid(vcpu_max_vl(vcpu)))) { \ __size_ret = 0; \ } else { \ - __vcpu_vq = vcpu_sve_max_vq(vcpu); \ + __vcpu_vq = sve_vl_from_vq(vcpu_max_vl(vcpu)); \ __size_ret = SVE_SIG_REGS_SIZE(__vcpu_vq); \ } \ \ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 1f75c9938f313e70a82d9792cb3c89caec2076a1..80a7d3119ec8421ca5c7050ed996fc9ba09ad7cf 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -105,6 +105,7 @@ struct kvm_regs { #define KVM_ARM_VCPU_PTRAUTH_ADDRESS 5 /* VCPU uses address authentication */ #define KVM_ARM_VCPU_PTRAUTH_GENERIC 6 /* VCPU uses generic authentication */ #define KVM_ARM_VCPU_HAS_EL2 7 /* Support nested virtualization */ +#define KVM_ARM_VCPU_SME 8 /* enable SME for this CPU */ /* * An alias for _SVE since we finalize VL configuration for both SVE and SME diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index b8e55a441282f57cb2d4bc55a43b41d5b774dfdd..5f44fe80af67369764d979e3cacd5bedacc5f7cb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -377,6 +377,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ARM_SVE: r = system_supports_sve(); break; + case KVM_CAP_ARM_SME: + r = system_supports_sme(); + break; case KVM_CAP_ARM_PTRAUTH_ADDRESS: case KVM_CAP_ARM_PTRAUTH_GENERIC: r = kvm_has_full_ptr_auth(); @@ -1394,6 +1397,9 @@ static unsigned long system_supported_vcpu_features(void) if (!system_supports_sve()) clear_bit(KVM_ARM_VCPU_SVE, &features); + if (!system_supports_sme()) + clear_bit(KVM_ARM_VCPU_SME, &features); + if (!kvm_has_full_ptr_auth()) { clear_bit(KVM_ARM_VCPU_PTRAUTH_ADDRESS, &features); clear_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, &features); @@ -2784,6 +2790,10 @@ static __init int kvm_arm_init(void) if (err) return err; + err = kvm_arm_init_sme(); + if (err) + return err; + err = kvm_arm_vmid_alloc_init(); if (err) { kvm_err("Failed to initialize VMID allocator.\n"); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 3cb91dc6dc3dc5cc484900dbd9f4cdfedb3e2b4a..78b5aa7976dbbfec35d83f5e373ef87636c2e3e4 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -76,6 +76,34 @@ int __init kvm_arm_init_sve(void) return 0; } +int __init kvm_arm_init_sme(void) +{ + if (system_supports_sme()) { + kvm_max_vl[ARM64_VEC_SME] = sme_max_virtualisable_vl(); + kvm_host_max_vl[ARM64_VEC_SME] = sme_max_vl(); + kvm_nvhe_sym(kvm_host_max_vl[ARM64_VEC_SME]) = kvm_host_max_vl[ARM64_VEC_SME]; + + /* + * The get_sve_reg()/set_sve_reg() ioctl interface will need + * to be extended with multiple register slice support in + * order to support vector lengths greater than + * VL_ARCH_MAX: + */ + if (WARN_ON(kvm_max_vl[ARM64_VEC_SME] > VL_ARCH_MAX)) + kvm_max_vl[ARM64_VEC_SME] = VL_ARCH_MAX; + + /* + * Don't even try to make use of vector lengths that + * aren't available on all CPUs, for now: + */ + if (kvm_max_vl[ARM64_VEC_SME] < sme_max_vl()) + pr_warn("KVM: SME vector length for guests limited to %u bytes\n", + kvm_max_vl[ARM64_VEC_SME]); + } + + return 0; +} + static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) { vcpu->arch.max_vl[ARM64_VEC_SVE] = kvm_max_vl[ARM64_VEC_SVE]; @@ -88,42 +116,84 @@ static void kvm_vcpu_enable_sve(struct kvm_vcpu *vcpu) set_bit(KVM_ARCH_FLAG_GUEST_HAS_SVE, &vcpu->kvm->arch.flags); } +static void kvm_vcpu_enable_sme(struct kvm_vcpu *vcpu) +{ + vcpu->arch.max_vl[ARM64_VEC_SME] = kvm_max_vl[ARM64_VEC_SME]; + + /* + * Userspace can still customize the vector lengths by writing + * KVM_REG_ARM64_SME_VLS. Allocation is deferred until + * kvm_arm_vcpu_finalize(), which freezes the configuration. + */ + set_bit(KVM_ARCH_FLAG_GUEST_HAS_SME, &vcpu->kvm->arch.flags); +} + /* - * Finalize vcpu's maximum SVE vector length, allocating - * vcpu->arch.sve_state as necessary. + * Finalize vcpu's maximum vector lengths, allocating + * vcpu->arch.sve_state and vcpu->arch.sme_state as necessary. */ static int kvm_vcpu_finalize_vec(struct kvm_vcpu *vcpu) { - void *buf; + void *sve_state, *sme_state; unsigned int vl; - size_t reg_sz; int ret; - vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; - /* * Responsibility for these properties is shared between * kvm_arm_init_sve(), kvm_vcpu_enable_sve() and * set_sve_vls(). Double-check here just to be sure: */ - if (WARN_ON(!sve_vl_valid(vl) || vl > sve_max_virtualisable_vl() || - vl > VL_ARCH_MAX)) - return -EIO; + if (vcpu_has_sve(vcpu)) { + vl = vcpu->arch.max_vl[ARM64_VEC_SVE]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sve_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } - reg_sz = vcpu_sve_state_size(vcpu); - buf = kzalloc(reg_sz, GFP_KERNEL_ACCOUNT); - if (!buf) + /* Similarly for SME */ + if (vcpu_has_sme(vcpu)) { + vl = vcpu->arch.max_vl[ARM64_VEC_SME]; + if (WARN_ON(!sve_vl_valid(vl) || + vl > sme_max_virtualisable_vl() || + vl > VL_ARCH_MAX)) + return -EIO; + } + + sve_state = kzalloc(vcpu_sve_state_size(vcpu), GFP_KERNEL_ACCOUNT); + if (!sve_state) return -ENOMEM; - ret = kvm_share_hyp(buf, buf + reg_sz); - if (ret) { - kfree(buf); - return ret; + ret = kvm_share_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); + if (ret) + goto err_sve_alloc; + + if (vcpu_has_sme(vcpu)) { + sme_state = kzalloc(vcpu_sme_state_size(vcpu), + GFP_KERNEL_ACCOUNT); + if (!sme_state) { + ret = -ENOMEM; + goto err_sve_map; + } + + ret = kvm_share_hyp(sme_state, + sme_state + vcpu_sme_state_size(vcpu)); + if (ret) + goto err_sve_map; + } else { + sme_state = NULL; } - - vcpu->arch.sve_state = buf; + + vcpu->arch.sve_state = sve_state; + vcpu->arch.sme_state = sme_state; vcpu_set_flag(vcpu, VCPU_VEC_FINALIZED); return 0; + +err_sve_map: + kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); +err_sve_alloc: + kfree(sve_state); + return ret; } int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) @@ -153,11 +223,15 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) { void *sve_state = vcpu->arch.sve_state; + void *sme_state = vcpu->arch.sme_state; kvm_unshare_hyp(vcpu, vcpu + 1); if (sve_state) kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); kfree(sve_state); + if (sme_state) + kvm_unshare_hyp(sme_state, sme_state + vcpu_sme_state_size(vcpu)); + kfree(sme_state); kfree(vcpu->arch.ccsidr); } @@ -165,6 +239,8 @@ static void kvm_vcpu_reset_vec(struct kvm_vcpu *vcpu) { if (vcpu_has_sve(vcpu)) memset(vcpu->arch.sve_state, 0, vcpu_sve_state_size(vcpu)); + if (vcpu_has_sme(vcpu)) + memset(vcpu->arch.sme_state, 0, vcpu_sme_state_size(vcpu)); } /** @@ -207,6 +283,8 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu) if (!kvm_arm_vcpu_vec_finalized(vcpu)) { if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) kvm_vcpu_enable_sve(vcpu); + if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_SME)) + kvm_vcpu_enable_sme(vcpu); } else { kvm_vcpu_reset_vec(vcpu); } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 45e6d8fca9b9969622dc15a6af924221782ac2e0..b1d9bd08e0c4643e5000dda5d2c5540e528fce7b 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -929,6 +929,7 @@ struct kvm_enable_cap { #define KVM_CAP_PRE_FAULT_MEMORY 236 #define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237 #define KVM_CAP_X86_GUEST_MODE 238 +#define KVM_CAP_ARM_SME 239 struct kvm_irq_routing_irqchip { __u32 irqchip; From patchwork Fri Feb 14 01:58:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865221 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C38631632E4; Fri, 14 Feb 2025 02:02:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498562; cv=none; b=DSAMAULkSKPQC7yo9kdRIkUbmzd3h0XM0/jTcELjZ/OdNJJcJjkGFRWXgLaF6qAqDYUqMFiYdskdtONrutwVmSi2IRnq/F7Zba6MIjtWF7K4D6Q7lFf6e0lgC47Pit4feFz2Ldx1vB9cNKRfwLCXNu6BCD747J7HiI1pxoc9g+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498562; c=relaxed/simple; bh=4XsjRO/Pqa3vD4B4LvvjSYJqhR/Q8my3VF4j5+psy/8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kyqTJmwaueHL9GCjn2xbDcCA3ZG2Z2AcnxjgvAyU0B96E/kvhh8kzHq0NgkGvCF6JJAQeMn9XXp5WqxEwHZ6YhVe3vF1w+sMEu7JjyOIE432E4zm/tH4ONfr+tXwLQqfHLZ05GvyjgvSUM4YDw6RsI9/5Vm0+cXMO8a0PChahiY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lbDoNZLH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lbDoNZLH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5148C4CEE2; Fri, 14 Feb 2025 02:02:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498562; bh=4XsjRO/Pqa3vD4B4LvvjSYJqhR/Q8my3VF4j5+psy/8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lbDoNZLH/npPV3CAYGJt7QFlOtgWhlVYENiulv/8XMb58njzpEUpl8nVZb7jbTURe 6hSK1MQZ8PfI4FHsXWEXTKNu70mG2nfcJCjqumUjpxiSdcjGV5xwg56OW9vQn3wB3G cSU/V2pMe2+17QVsaQuqdlbLMnLedMWjmjXm6L2ggdqiVvr833prCbL4KPo5QNYBuN FE68MnzV83bmP1rcaHN0WLwnEvAgBrqgfhTWvaE/H1KjzT2OvptiHv8Rc87iClPFGh +zxHphnXK8JX48q4yQkq2A0+THWp2ceF9+LdQYKM8BxfYxPfNb1pgTUT/cgTeCchqy G7MvHDbRKo3Qg== From: Mark Brown Date: Fri, 14 Feb 2025 01:58:10 +0000 Subject: [PATCH v4 27/27] KVM: arm64: selftests: Add SME to set_id_regs test Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-27-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=3734; i=broonie@kernel.org; h=from:subject:message-id; bh=4XsjRO/Pqa3vD4B4LvvjSYJqhR/Q8my3VF4j5+psy/8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPVhF/rm1Hv+EDc0crKIVE9eZzvnD7KQi7+Xcuq BToVxzeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66j1QAKCRAk1otyXVSH0EKyB/ 46ZNQvr34wuZ+41q3A8QM4rtKDn9mqBlEEHWAAKtOO/RahHBnmn7c3sABa+vN2iSS2Im3JQHca/3j1 xoEANRnDiA/WjBm9tuMO/yFu1B4QO9GjB+AGTQU8rqMw1iJBgDQpTBo+efrNS9uPxK+xkxXE5U5iSy 1gL6gNmOnu0LfBKPSiop8+oW5jOoF6/e/hAo0fRRE19TIjuDMoaeEpuP1cBS0XDLw4xr7Pb2jKbmrn o1OHN/XsbBGqPda0D46kRC9ZvnTvd9ndqHaxlce05AS4kJes9Fo1hu8ixMUaCaiND+vk5+QsIde4Uj unleftWfgAFAET/vRMBY81YzWBFgjF X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add coverage of the SME ID registers to set_id_regs, ID_AA64PFR1_EL1.SME becomes writable and we add ID_AA64SMFR_EL1 and it's subfields. Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/arm64/set_id_regs.c | 29 +++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/arm64/set_id_regs.c b/tools/testing/selftests/kvm/arm64/set_id_regs.c index 217541fe653601a82697134c8d773e901a205cfe..16a4cd84f45472451479c6a5502c67adb4d64490 100644 --- a/tools/testing/selftests/kvm/arm64/set_id_regs.c +++ b/tools/testing/selftests/kvm/arm64/set_id_regs.c @@ -138,6 +138,7 @@ static const struct reg_ftr_bits ftr_id_aa64pfr0_el1[] = { static const struct reg_ftr_bits ftr_id_aa64pfr1_el1[] = { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, CSV2_frac, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, SME, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, SSBS, ID_AA64PFR1_EL1_SSBS_NI), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64PFR1_EL1, BT, 0), REG_FTR_END, @@ -182,6 +183,28 @@ static const struct reg_ftr_bits ftr_id_aa64mmfr2_el1[] = { REG_FTR_END, }; +static const struct reg_ftr_bits ftr_id_aa64smfr0_el1[] = { + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, FA64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, LUTv2, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SMEver, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F64F64, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I16I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16B16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F16, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F8F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, I8I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, B16F32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, BI32I32, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, F32F32, 0) +, REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8FMA, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP4, 0), + REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64SMFR0_EL1, SF8DP2, 0), + REG_FTR_END, +}; + static const struct reg_ftr_bits ftr_id_aa64zfr0_el1[] = { REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F64MM, 0), REG_FTR_BITS(FTR_LOWER_SAFE, ID_AA64ZFR0_EL1, F32MM, 0), @@ -212,6 +235,7 @@ static struct test_feature_reg test_regs[] = { TEST_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0_el1), TEST_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1_el1), TEST_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2_el1), + TEST_REG(SYS_ID_AA64SMFR0_EL1, ftr_id_aa64smfr0_el1), TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64zfr0_el1), }; @@ -228,6 +252,7 @@ static void guest_code(void) GUEST_REG_SYNC(SYS_ID_AA64MMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR1_EL1); GUEST_REG_SYNC(SYS_ID_AA64MMFR2_EL1); + GUEST_REG_SYNC(SYS_ID_AA64SMFR0_EL1); GUEST_REG_SYNC(SYS_ID_AA64ZFR0_EL1); GUEST_REG_SYNC(SYS_CTR_EL0); @@ -675,8 +700,8 @@ int main(void) ARRAY_SIZE(ftr_id_aa64isar2_el1) + ARRAY_SIZE(ftr_id_aa64pfr0_el1) + ARRAY_SIZE(ftr_id_aa64pfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr0_el1) + ARRAY_SIZE(ftr_id_aa64mmfr1_el1) + ARRAY_SIZE(ftr_id_aa64mmfr2_el1) + - ARRAY_SIZE(ftr_id_aa64zfr0_el1) - ARRAY_SIZE(test_regs) + 2 + - MPAM_IDREG_TEST; + ARRAY_SIZE(ftr_id_aa64zfr0_el1) + ARRAY_SIZE(ftr_id_aa64smfr0_el1) - ARRAY_SIZE(test_regs) + + 2 + MPAM_IDREG_TEST; ksft_set_plan(test_cnt);