From patchwork Fri Feb 14 01:57:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 865234 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 410DB13AA2F; Fri, 14 Feb 2025 02:01:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498462; cv=none; b=Wdqdcc++EE+ACzb7MPR/iOU1OGR1GgAUtu6Sv8kxk+gfIBmJOelCXlQLGtJFjauQ+6RFuxhhKH/D9CtgteksfKtPJv00PPaO+MR4XzX+oD6lozZMzMgOkRgeloZn7G44RJW2aZGlwOWzu2Zh7k64ZAlLb5bnUhxFFXsMbRSPFqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739498462; c=relaxed/simple; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=th3kHeMueQFBCyo3kHFfr8/HZeM41g6mO4QeUcPN7bT0WJnM9oJlLFllNu9lY3k6IwpYDIcGZKcnVqhZRtKD046P5HxGHF4utsyl8IjeF10mF/GP6qPrJfjIAAHcld7AwEmzK2LT1ofvUeHZ9kWEi+uMd2aflQUjbErzhh+zBGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XfdNjUR5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XfdNjUR5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5A3F3C4CEE6; Fri, 14 Feb 2025 02:00:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739498461; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XfdNjUR5WvKd0M+mir0toNA5usCYkyCvyL1IdKDsA6ruyoNu1d7Gr1c/MUY9aDfVy oJhxlRyV+i/cBnJO/uw/vQ2MgHtoPLAjA5ZaIObdSUZ8ckdEqFtZW5u+Xj030hRqWP lRWHVJfYZPKsTKbuPrSOes4f1O/EI/rXFa8VLXUUTjjWgq7SQtozf6qg8TqxNopI1y 1Uqf6QK24EdWKFD1AIsS5XkdYBjJH0wCWr3fgBa2Ab/U11o3XkT6x2LEwav5wY9zH7 mrBE4Cgvi8cZ2nDHmJYQebqiOsaXZwPjYzXNkUC/yiFOL9NbpkS1nQucyLqbYXevOd sdzV7iXYrt95w== From: Mark Brown Date: Fri, 14 Feb 2025 01:57:44 +0000 Subject: [PATCH v4 01/27] arm64/fpsimd: Update FA64 and ZT0 enables when loading SME state Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-kvm-arm64-sme-v4-1-d64a681adcc2@kernel.org> References: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> In-Reply-To: <20250214-kvm-arm64-sme-v4-0-d64a681adcc2@kernel.org> To: Marc Zyngier , Oliver Upton , Joey Gouly , Catalin Marinas , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Jonathan Corbet , Shuah Khan Cc: Dave Martin , Fuad Tabba , Mark Rutland , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Mark Brown X-Mailer: b4 0.15-dev-1b0d6 X-Developer-Signature: v=1; a=openpgp-sha256; l=5400; i=broonie@kernel.org; h=from:subject:message-id; bh=kXbObTIYEyDWDXLdC1xsFRscizZQfM2JhsUqGeftO28=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBnrqPBMdsgvM+gVTEg6r1ZGMsNCAoG4qXO215iqD2E bZBSAMyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZ66jwQAKCRAk1otyXVSH0Es+B/ sFBDsFj8Nb12IFeon3+wHzalejhq9ZW8OSC65eCxkTfByjoBMXLrYOiM8srDG+ilcRsH3Egwn6EJ/n aaZqtKm913QXspxpRKDGq/x7TS1wjz3X60jIzKRXsN+O4vbA8+sQhjvO8+eoRPtjd9LeZiNG2N7CeA TWnm3W2lKZWZ9tbec2ixGdtmn/Y3aFTMkUZIBnLzPigM4F5Hzp5gbUZ4TcR+c4RxePznuVVs7YfOWO KWVl/44Uc2l56SKZsAFHTeZTxWedl+e0ycDe8LuzA+4SaL/QWlHjmL/W/V8fQgCQ+53Izz6JnI5ZCp XgJpqDl9qI3M1rNo8iZYaWHxmJ0vRO X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we enable EL0 and EL1 access to FA64 and ZT0 at boot and leave them enabled throughout the runtime of the system. When we add KVM support we will need to make this configuration dynamic, these features may be disabled for some KVM guests. Since the host kernel saves the floating point state for non-protected guests and we wish to avoid KVM having to reload the floating point state needlessly on guest reentry let's move the configuration of these enables to the floating point state reload. We provide a helper which does the configuration as part of a read/modify/write operation along with the configuration of the task VL, then update the floating point state load and SME access trap to use it. We also remove the setting of the enable bits from the CPU feature identification and resume paths. There will be a small overhead from setting the enables one at a time but this should be negligable in the context of the state load or access trap. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 12 +++++++++++ arch/arm64/kernel/cpufeature.c | 2 -- arch/arm64/kernel/fpsimd.c | 44 ++++++++++++----------------------------- 3 files changed, 25 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index f2a84efc361858d4deda99faf1967cc7cac386c1..95355892d47b3ec1c77a3ab19ccad0d7f9a8d621 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -429,6 +429,18 @@ static inline size_t sme_state_size(struct task_struct const *task) #endif /* ! CONFIG_ARM64_SME */ +#define sme_cond_update_smcr(vl, fa64, zt0, reg) \ + do { \ + u64 __old = read_sysreg_s((reg)); \ + u64 __new = vl; \ + if (fa64) \ + __new |= SMCR_ELx_FA64; \ + if (zt0) \ + __new |= SMCR_ELx_EZT0; \ + if (__old != __new) \ + write_sysreg_s(__new, (reg)); \ + } while (0) + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index f0910f20fbf8c18fbeb63bcee18abf13371b1d5e..7ba2a554243fccfb94d34307821101bfcf6c07bb 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2857,7 +2857,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME_FA64, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_fa64, ARM64_CPUID_FIELDS(ID_AA64SMFR0_EL1, FA64, IMP) }, { @@ -2865,7 +2864,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_SYSTEM_FEATURE, .capability = ARM64_SME2, .matches = has_cpuid_feature, - .cpu_enable = cpu_enable_sme2, ARM64_CPUID_FIELDS(ID_AA64PFR1_EL1, SME, SME2) }, #endif /* CONFIG_ARM64_SME */ diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8370d55f035334edf9d4f01fb33e1054bddadf71..e52ec18f0fcab0e34123b7a115e886fca3fae210 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -399,9 +399,15 @@ static void task_fpsimd_load(void) if (system_supports_sme()) { unsigned long sme_vl = task_get_sme_vl(current); - /* Ensure VL is set up for restoring data */ + /* + * Ensure VL is set up for restoring data. KVM might + * disable subfeatures so we reset them each time. + */ if (test_thread_flag(TIF_SME)) - sme_set_vq(sve_vq_from_vl(sme_vl) - 1); + sme_cond_update_smcr(sve_vq_from_vl(sme_vl) - 1, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); write_sysreg_s(current->thread.svcr, SYS_SVCR); @@ -1267,26 +1273,6 @@ void cpu_enable_sme(const struct arm64_cpu_capabilities *__always_unused p) isb(); } -void cpu_enable_sme2(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME2 <= ARM64_SME); - - /* Allow use of ZT0 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_EZT0_MASK, - SYS_SMCR_EL1); -} - -void cpu_enable_fa64(const struct arm64_cpu_capabilities *__always_unused p) -{ - /* This must be enabled after SME */ - BUILD_BUG_ON(ARM64_SME_FA64 <= ARM64_SME); - - /* Allow use of FA64 */ - write_sysreg_s(read_sysreg_s(SYS_SMCR_EL1) | SMCR_ELx_FA64_MASK, - SYS_SMCR_EL1); -} - void __init sme_setup(void) { struct vl_info *info = &vl_info[ARM64_VEC_SME]; @@ -1330,17 +1316,9 @@ void __init sme_setup(void) void sme_suspend_exit(void) { - u64 smcr = 0; - if (!system_supports_sme()) return; - if (system_supports_fa64()) - smcr |= SMCR_ELx_FA64; - if (system_supports_sme2()) - smcr |= SMCR_ELx_EZT0; - - write_sysreg_s(smcr, SYS_SMCR_EL1); write_sysreg_s(0, SYS_SMPRI_EL1); } @@ -1457,7 +1435,11 @@ void do_sme_acc(unsigned long esr, struct pt_regs *regs) if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { unsigned long vq_minus_one = sve_vq_from_vl(task_get_sme_vl(current)) - 1; - sme_set_vq(vq_minus_one); + + sme_cond_update_smcr(vq_minus_one, + system_supports_fa64(), + system_supports_sme2(), + SYS_SMCR_EL1); fpsimd_bind_task_to_cpu(); }