From patchwork Mon Mar 31 08:22:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877409 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC8E91EC009; Mon, 31 Mar 2025 08:24:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; cv=none; b=uYoqqOYBdShbNLT9rfaO7iFt6LJX6KzM0oFgJJluBhI8sTM/HYWMg6r/wpwsDuVznURKfmw2/zxaMG/TpjJ4pNNM1XMw0aJDnlMqOzRCAiMOBiV+W4wEa6zrqwcKL1Si2bfWhoGgEE9qj1/2nax9uqmgPh+7JyPlofVHnYdxtb8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; c=relaxed/simple; bh=Dt+JPai8+zal0PU5TNa9Qc5El9RrxyJOxdkK3MpXOcA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jsVG36jVD9+3QHaXi87FA06vQ6RCiOpaSiae+EuCrKhaq1fulFfkRVU/RxPw4Fu/eS28bLiKJLQ22nXEx1uBKLebMhT5sDsBf2asLjIwGGMoMA+HB5ukMT1SZFPYhcrYE7Lzyqiu+ao6lB4kIqwWC/v27KySpJ092xLScmiUTYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=DfikwSBt; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="DfikwSBt" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp013171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:00 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp013171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409382; bh=8NBvcv0ohTWU39bJ6aL9yuUsmviH/IXJjGBdBncqKAA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DfikwSBtepDkTPWSndBbjw9sQVnN64z4KXbHHV4byiheN5oN3c8108HUHo0WlTHiO fbuV4mKhMgYKnfiBPbYoVLpeAEW1C8xaUoDBQBJOqlka2Ux1zE5IQP96taH035GT/e xjt8OpAXfqCNG5GQQNWIsO0juYtYQXHFy1GfDKPCFncm+WTFt1opnlrPH8zpyO5lys AWQreJObo0N83zSyBr8Uut3butD6okD1jJ5CCQoAsoeWQqGbWx83aVOu3dhvPlDmYf JoUzckPNpFkOLgLe6na+VCr251PNcWOk98ANs3QkFTe88p9rHaeel6kOYEiLGzsU7J 1iQD+J8vO7ZXg== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 01/15] x86/msr: Replace __wrmsr() with native_wrmsrl() Date: Mon, 31 Mar 2025 01:22:37 -0700 Message-ID: <20250331082251.3171276-2-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 __wrmsr() is the lowest level primitive MSR write API, and its direct use is NOT preferred. Use its wrapper function native_wrmsrl() instead. No functional change intended. Signed-off-by: Xin Li (Intel) --- arch/x86/events/amd/brs.c | 2 +- arch/x86/include/asm/apic.h | 2 +- arch/x86/include/asm/msr.h | 6 ++++-- arch/x86/kernel/cpu/mce/core.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +++--- 5 files changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/events/amd/brs.c b/arch/x86/events/amd/brs.c index ec3427463382..4a47f3c108de 100644 --- a/arch/x86/events/amd/brs.c +++ b/arch/x86/events/amd/brs.c @@ -44,7 +44,7 @@ static inline unsigned int brs_to(int idx) static __always_inline void set_debug_extn_cfg(u64 val) { /* bits[4:3] must always be set to 11b */ - __wrmsr(MSR_AMD_DBG_EXTN_CFG, val | 3ULL << 3, val >> 32); + native_wrmsrl(MSR_AMD_DBG_EXTN_CFG, val | 3ULL << 3); } static __always_inline u64 get_debug_extn_cfg(void) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index c903d358405d..3345a819c859 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -214,7 +214,7 @@ static inline void native_apic_msr_write(u32 reg, u32 v) static inline void native_apic_msr_eoi(void) { - __wrmsr(APIC_BASE_MSR + (APIC_EOI >> 4), APIC_EOI_ACK, 0); + native_wrmsrl(APIC_BASE_MSR + (APIC_EOI >> 4), APIC_EOI_ACK); } static inline u32 native_apic_msr_read(u32 reg) diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index 9397a319d165..27ea8793705d 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -144,10 +144,12 @@ static inline unsigned long long native_read_msr_safe(unsigned int msr, static inline void notrace native_write_msr(unsigned int msr, u32 low, u32 high) { - __wrmsr(msr, low, high); + u64 val = (u64)high << 32 | low; + + native_wrmsrl(msr, val); if (tracepoint_enabled(write_msr)) - do_trace_write_msr(msr, ((u64)high << 32 | low), 0); + do_trace_write_msr(msr, val, 0); } /* Can be uninlined because referenced by paravirt */ diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 1f14c3308b6b..0eaeaba12df2 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -1306,7 +1306,7 @@ static noinstr bool mce_check_crashing_cpu(void) } if (mcgstatus & MCG_STATUS_RIPV) { - __wrmsr(MSR_IA32_MCG_STATUS, 0, 0); + native_wrmsrl(MSR_IA32_MCG_STATUS, 0); return true; } } diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 01fa7890b43f..55536120c8d1 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -481,7 +481,7 @@ int resctrl_arch_pseudo_lock_fn(void *_plr) * cache. */ saved_msr = __rdmsr(MSR_MISC_FEATURE_CONTROL); - __wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); + native_wrmsrl(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits); closid_p = this_cpu_read(pqr_state.cur_closid); rmid_p = this_cpu_read(pqr_state.cur_rmid); mem_r = plr->kmem; @@ -493,7 +493,7 @@ int resctrl_arch_pseudo_lock_fn(void *_plr) * pseudo-locked followed by reading of kernel memory to load it * into the cache. */ - __wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, plr->closid); + native_wrmsrl(MSR_IA32_PQR_ASSOC, (u64)plr->closid << 32 | rmid_p); /* * Cache was flushed earlier. Now access kernel memory to read it @@ -530,7 +530,7 @@ int resctrl_arch_pseudo_lock_fn(void *_plr) * Critical section end: restore closid with capacity bitmask that * does not overlap with pseudo-locked region. */ - __wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, closid_p); + native_wrmsrl(MSR_IA32_PQR_ASSOC, (u64)closid_p << 32 | rmid_p); /* Re-enable the hardware prefetcher(s) */ wrmsrl(MSR_MISC_FEATURE_CONTROL, saved_msr); From patchwork Mon Mar 31 08:22:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877617 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2986C41AAC; Mon, 31 Mar 2025 08:24:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409456; cv=none; b=mi3Zx2aE/ASD/VCwNlT97SoMiMZreK7klhiV7+h7Jnl1YPt8Nx1da+eFubkP65DjUcv3Jcu/EyEv49rE07kTVUyxigtAjw/yu0Ikn4IDnpdaMDAe8xCT1hEl7zTFt9p1wQWPNgRE8MjRsNJvbl3qrmuBBPvnw0nkft5OPwi69r0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409456; c=relaxed/simple; bh=BmINf9YXq0zwgNkhB9uzoU9LYnCfIvTtplUzvx0otZ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r8BVYpueX0/HcRm+t31YJ2rLhGgjyUmPB//An8duFl6moR4el/DGQjFTe6U5wzQq/mK8arGJLyXl7qLL6DzbVL4Yy038Zosb7uTALTD0hozQuFh9gLJRtQASY/EWWgilBZ10SL/yviPn+sw2RMNw3FIGbVJ9SL2JABYJz0ZcNWs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=GMjpnnYX; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="GMjpnnYX" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp023171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:02 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp023171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409384; bh=m4M8UGMKKd7tRa3nJstnKZwTxnwNx/YpFEKHVX35S1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GMjpnnYXMYpU67NGz93c7h9ws4Z3p3rE979fcI5Xs1NdNl+Td/mpmwbnPiYulmfj8 b8fSVsdt2ad5j0QSex3qUA/BNF0M8ziPcP4Mkz38x3S+XUwD4rvbU8+eDWfUPxnO7E Y70mhL72N7fvX3WqwqaTorgtiRS94zS/fVo8nOEjFVs9kCAUlFTEOFTIKmdPh072el C2aXZqY89xdoTLHuywMC03Z51D76GbRxilkwQRwDPUqIeAdLHWceDRXcmPCxXB7B88 vzqb8gdBtHAv76CahwdMT05nUSEwKm9KaXVsW8BCSDJ3odt76mR0w6Fy90TxcG4WqJ QtAekxEcU2fGw== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 02/15] x86/msr: Replace __rdmsr() with native_rdmsrl() Date: Mon, 31 Mar 2025 01:22:38 -0700 Message-ID: <20250331082251.3171276-3-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 __rdmsr() is the lowest level primitive MSR read API, and its direct use is NOT preferred. Use its wrapper function native_rdmsrl() instead. No functional change intended. Signed-off-by: Xin Li (Intel) --- arch/x86/coco/sev/core.c | 2 +- arch/x86/events/amd/brs.c | 2 +- arch/x86/hyperv/hv_vtl.c | 4 ++-- arch/x86/hyperv/ivm.c | 2 +- arch/x86/include/asm/mshyperv.h | 2 +- arch/x86/include/asm/msr.h | 5 +++++ arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/cpu/mce/core.c | 4 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kvm/vmx/vmx.c | 4 ++-- arch/x86/mm/mem_encrypt_identity.c | 4 ++-- 11 files changed, 19 insertions(+), 14 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index b0c1a7a57497..d38e6f0ff9c4 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -276,7 +276,7 @@ static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) static inline u64 sev_es_rd_ghcb_msr(void) { - return __rdmsr(MSR_AMD64_SEV_ES_GHCB); + return native_rdmsrl(MSR_AMD64_SEV_ES_GHCB); } static __always_inline void sev_es_wr_ghcb_msr(u64 val) diff --git a/arch/x86/events/amd/brs.c b/arch/x86/events/amd/brs.c index 4a47f3c108de..3ad7d87b5403 100644 --- a/arch/x86/events/amd/brs.c +++ b/arch/x86/events/amd/brs.c @@ -49,7 +49,7 @@ static __always_inline void set_debug_extn_cfg(u64 val) static __always_inline u64 get_debug_extn_cfg(void) { - return __rdmsr(MSR_AMD_DBG_EXTN_CFG); + return native_rdmsrl(MSR_AMD_DBG_EXTN_CFG); } static bool __init amd_brs_detect(void) diff --git a/arch/x86/hyperv/hv_vtl.c b/arch/x86/hyperv/hv_vtl.c index 13242ed8ff16..4a27e475d35f 100644 --- a/arch/x86/hyperv/hv_vtl.c +++ b/arch/x86/hyperv/hv_vtl.c @@ -149,11 +149,11 @@ static int hv_vtl_bringup_vcpu(u32 target_vp_index, int cpu, u64 eip_ignored) input->vp_context.rip = rip; input->vp_context.rsp = rsp; input->vp_context.rflags = 0x0000000000000002; - input->vp_context.efer = __rdmsr(MSR_EFER); + input->vp_context.efer = native_rdmsrl(MSR_EFER); input->vp_context.cr0 = native_read_cr0(); input->vp_context.cr3 = __native_read_cr3(); input->vp_context.cr4 = native_read_cr4(); - input->vp_context.msr_cr_pat = __rdmsr(MSR_IA32_CR_PAT); + input->vp_context.msr_cr_pat = native_rdmsrl(MSR_IA32_CR_PAT); input->vp_context.idtr.limit = idt_ptr.size; input->vp_context.idtr.base = idt_ptr.address; input->vp_context.gdtr.limit = gdt_ptr.size; diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 77bf05f06b9e..95cf2113a72a 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -110,7 +110,7 @@ u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_size) static inline u64 rd_ghcb_msr(void) { - return __rdmsr(MSR_AMD64_SEV_ES_GHCB); + return native_rdmsrl(MSR_AMD64_SEV_ES_GHCB); } static inline void wr_ghcb_msr(u64 val) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index bab5ccfc60a7..2ca6ef89530d 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -304,7 +304,7 @@ void hv_set_non_nested_msr(unsigned int reg, u64 value); static __always_inline u64 hv_raw_get_msr(unsigned int reg) { - return __rdmsr(reg); + return native_rdmsrl(reg); } #else /* CONFIG_HYPERV */ diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index 27ea8793705d..fb3d7c4cb774 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -106,6 +106,11 @@ do { \ (void)((val2) = (u32)(__val >> 32)); \ } while (0) +static __always_inline u64 native_rdmsrl(const u32 msr) +{ + return __rdmsr(msr); +} + #define native_wrmsr(msr, low, high) \ __wrmsr(msr, low, high) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 12126adbc3a9..a268db71d944 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -164,7 +164,7 @@ static void ppin_init(struct cpuinfo_x86 *c) /* Is the enable bit set? */ if (val & 2UL) { - c->ppin = __rdmsr(info->msr_ppin); + c->ppin = native_rdmsrl(info->msr_ppin); set_cpu_cap(c, info->feature); return; } diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 0eaeaba12df2..0e050af723f5 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -121,7 +121,7 @@ void mce_prep_record_common(struct mce *m) { m->cpuid = cpuid_eax(1); m->cpuvendor = boot_cpu_data.x86_vendor; - m->mcgcap = __rdmsr(MSR_IA32_MCG_CAP); + m->mcgcap = native_rdmsrl(MSR_IA32_MCG_CAP); /* need the internal __ version to avoid deadlocks */ m->time = __ktime_get_real_seconds(); } @@ -1298,7 +1298,7 @@ static noinstr bool mce_check_crashing_cpu(void) (crashing_cpu != -1 && crashing_cpu != cpu)) { u64 mcgstatus; - mcgstatus = __rdmsr(MSR_IA32_MCG_STATUS); + mcgstatus = native_rdmsrl(MSR_IA32_MCG_STATUS); if (boot_cpu_data.x86_vendor == X86_VENDOR_ZHAOXIN) { if (mcgstatus & MCG_STATUS_LMCES) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 55536120c8d1..675fd9f93e33 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -480,7 +480,7 @@ int resctrl_arch_pseudo_lock_fn(void *_plr) * the buffer and evict pseudo-locked memory read earlier from the * cache. */ - saved_msr = __rdmsr(MSR_MISC_FEATURE_CONTROL); + saved_msr = native_rdmsrl(MSR_MISC_FEATURE_CONTROL); native_wrmsrl(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits); closid_p = this_cpu_read(pqr_state.cur_closid); rmid_p = this_cpu_read(pqr_state.cur_rmid); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5c5766467a61..2a24060397cd 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -380,7 +380,7 @@ static __always_inline void vmx_disable_fb_clear(struct vcpu_vmx *vmx) if (!vmx->disable_fb_clear) return; - msr = __rdmsr(MSR_IA32_MCU_OPT_CTRL); + msr = native_rdmsrl(MSR_IA32_MCU_OPT_CTRL); msr |= FB_CLEAR_DIS; native_wrmsrl(MSR_IA32_MCU_OPT_CTRL, msr); /* Cache the MSR value to avoid reading it later */ @@ -7307,7 +7307,7 @@ void noinstr vmx_spec_ctrl_restore_host(struct vcpu_vmx *vmx, return; if (flags & VMX_RUN_SAVE_SPEC_CTRL) - vmx->spec_ctrl = __rdmsr(MSR_IA32_SPEC_CTRL); + vmx->spec_ctrl = native_rdmsrl(MSR_IA32_SPEC_CTRL); /* * If the guest/host SPEC_CTRL values differ, restore the host value. diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 5eecdd92da10..3005b07a0016 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -526,7 +526,7 @@ void __head sme_enable(struct boot_params *bp) me_mask = 1UL << (ebx & 0x3f); /* Check the SEV MSR whether SEV or SME is enabled */ - RIP_REL_REF(sev_status) = msr = __rdmsr(MSR_AMD64_SEV); + RIP_REL_REF(sev_status) = msr = native_rdmsrl(MSR_AMD64_SEV); feature_mask = (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT; /* @@ -557,7 +557,7 @@ void __head sme_enable(struct boot_params *bp) return; /* For SME, check the SYSCFG MSR */ - msr = __rdmsr(MSR_AMD64_SYSCFG); + msr = native_rdmsrl(MSR_AMD64_SYSCFG); if (!(msr & MSR_AMD64_SYSCFG_MEM_ENCRYPT)) return; } From patchwork Mon Mar 31 08:22:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877612 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2141A1D5178; Mon, 31 Mar 2025 08:24:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; cv=none; b=OWutZy+o3b4e0Y4kOznJl8kNGnVOYAcf2FMfpN+BiaaPGvnVW9zowXp1fK8bVZTAst6iIexSJ/E4ehKFORtiNEudZ3h4A3ZD392/duKIh1NH89O0UwUh7kYwvzM+x3l1jG6K6DjV2JmWWEom/aO3kuN01KsasVKKj/YiKaYXrNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; c=relaxed/simple; bh=6EPdz8aFjQ1ednZahrkNwc2//BReoieOSPgWhzZZxB0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LxudXpiOokoouJgFdxoCxJ3WYjAvrSK1vCqJtxkgyoxJE1T4QTG7uPmQaig4GJ3+fvr+8B/7kUvc39JS8gNdxFSSbYAeMBtQU/3IZkvY2yQywjim2eVinuobglDxu8EokglOBeB0t27VdBVJe16efpa5vv3gchyvJRuM/evbus8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=gZFEitNC; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="gZFEitNC" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp033171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:04 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp033171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409386; bh=5yXzAEblnWuewwiM+2a2MBWdk0hE50lz5zB2BZNESnk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gZFEitNCvpKZCut95fXZKAqZjYVexIRqqCsmMBp2CZW5W/F+F8YdgBRq7Jas48oEm jA/8QUCfk2D6k8vYesJGOnrzZv+XWetZsr8GnENjabojFMIx710C0/l2qUvpne52/U BKRmW6qbXZJEaaZotvS+ZyspM+SBZxcPiUz0OolK8l3PTAJGgcrhaoBd8FoI0axiFO UYJY19A43VO/nzu4JfIPKLbCPKuIQ+Fm3xRSaESxcKaXZ3m3Z/51sVm6ksmFV/kMOP i/pGDQotEM9rWQRUfcKCF7CHn9HUUQ60/HHqfKDisSlMtrPlhTUBKsDUSIOgBvisE7 KhHjRSIG4ujjQ== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 03/15] x86/msr: Simplify pmu_msr_{read,write}() Date: Mon, 31 Mar 2025 01:22:39 -0700 Message-ID: <20250331082251.3171276-4-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Remove calling native_{read,write}_msr{_safe}() in pmu_msr_{read,write}(), and have them return false to let their caller to do that instead. Refactor pmu_msr_write() to take the input MSR value in an u64 argument, replacing the current dual u32 arguments. Suggested-by: H. Peter Anvin (Intel) Sign-off-by: Xin Li (Intel) --- arch/x86/xen/enlighten_pv.c | 6 +++++- arch/x86/xen/pmu.c | 27 ++++----------------------- arch/x86/xen/xen-ops.h | 4 ++-- 3 files changed, 11 insertions(+), 26 deletions(-) diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index dcc2041f8e61..2bfe57469ac3 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1133,6 +1133,8 @@ static void set_seg(unsigned int which, unsigned int low, unsigned int high, static void xen_do_write_msr(unsigned int msr, unsigned int low, unsigned int high, int *err) { + u64 val; + switch (msr) { case MSR_FS_BASE: set_seg(SEGBASE_FS, low, high, err); @@ -1159,7 +1161,9 @@ static void xen_do_write_msr(unsigned int msr, unsigned int low, break; default: - if (!pmu_msr_write(msr, low, high, err)) { + val = (u64)high << 32 | low; + + if (!pmu_msr_write(msr, val)) { if (err) *err = native_write_msr_safe(msr, low, high); else diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c index f06987b0efc3..1364cd3fb3ef 100644 --- a/arch/x86/xen/pmu.c +++ b/arch/x86/xen/pmu.c @@ -313,37 +313,18 @@ static bool pmu_msr_chk_emulated(unsigned int msr, uint64_t *val, bool is_read, return true; } -bool pmu_msr_read(unsigned int msr, uint64_t *val, int *err) +bool pmu_msr_read(u32 msr, u64 *val, int *err) { bool emulated; - if (!pmu_msr_chk_emulated(msr, val, true, &emulated)) - return false; - - if (!emulated) { - *val = err ? native_read_msr_safe(msr, err) - : native_read_msr(msr); - } - - return true; + return pmu_msr_chk_emulated(msr, val, true, &emulated) && emulated; } -bool pmu_msr_write(unsigned int msr, uint32_t low, uint32_t high, int *err) +bool pmu_msr_write(u32 msr, u64 val) { - uint64_t val = ((uint64_t)high << 32) | low; bool emulated; - if (!pmu_msr_chk_emulated(msr, &val, false, &emulated)) - return false; - - if (!emulated) { - if (err) - *err = native_write_msr_safe(msr, low, high); - else - native_write_msr(msr, low, high); - } - - return true; + return pmu_msr_chk_emulated(msr, &val, false, &emulated) && emulated; } static unsigned long long xen_amd_read_pmc(int counter) diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index 63c13a2ccf55..4a0a1d73d8b8 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -274,8 +274,8 @@ void xen_pmu_finish(int cpu); static inline void xen_pmu_init(int cpu) {} static inline void xen_pmu_finish(int cpu) {} #endif -bool pmu_msr_read(unsigned int msr, uint64_t *val, int *err); -bool pmu_msr_write(unsigned int msr, uint32_t low, uint32_t high, int *err); +bool pmu_msr_read(u32 msr, u64 *val, int *err); +bool pmu_msr_write(u32 msr, u64 val); int pmu_apic_update(uint32_t reg); unsigned long long xen_read_pmc(int counter); From patchwork Mon Mar 31 08:22:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877616 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 727801DE2A4; Mon, 31 Mar 2025 08:24:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; cv=none; b=c6qKDJo8/LHt/dymASd4EV8IR4xiivJbVAPA2U0+ASgU2gCjUEUHF1b4uJmGLeM4HOCygWFPi7a+x55kMcPXiTFr6xi/16EghS+gED8r17qkULVXec4UCwI4tJwLC0RwRNWCkunM6ljln58UKmS0j9S89IN5/DdstE1E8XW1sRY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; c=relaxed/simple; bh=tME7mojeJezeDUpP+ObCQecGa9XLxXIzWAI0QCCNPoE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JPPXu+1ujDztgrhlBfzPYIFX11bxMnzwbr65ekQwc1CF29e0F6cDOLi3vk9kkNhW3SAosy/nbsKbvdY8yO526ufUx17MPe39iSp1ZFVT9xM1VAN/ytDKMNzRsoKG3fNKR7CFQ84Lr+NUItYd0ffiLXQOkeeasfLTaWpSX8DiWQw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=cjtZINut; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="cjtZINut" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp043171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:06 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp043171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409388; bh=R5lkZK1BttlgoDRIcsGie/EvPC/1EsVxCj2GWRAyo+M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cjtZINutBa+FD44rvEKBI/bQn/6JnE37jJg8UfQt76je9oer8CW+KU4+GASTM697n 43XgUKC7AUoEwu3/CB/moVDg2HKzbhmtehqO4aY0qEvC/qHCgIZb3cKa7RdsVrgwnl AUil1g8TJnz5dWM5rTRr4QNzdiKcsuNrqQZvGQjKzzHfZaQiw5xA2m5iYN6sK7sImI oREOYUP88D8O6Cp/ISnEuVJDEMudARxBOJb7gcgFfJ1/rgsxpmKmOjcw84nWzZkB3C VsITyaqW7KXt9Q1SvVjBLyJNoLTgoCY1Ptdb7Cq8X9iccW9VZz7LSfWoqKxg9r1F67 TUz6F6A4Ni/IA== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 04/15] x86/msr: Let pv_cpu_ops.write_msr{_safe}() take an u64 instead of two u32 Date: Mon, 31 Mar 2025 01:22:40 -0700 Message-ID: <20250331082251.3171276-5-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Refactor pv_cpu_ops.write_msr{_safe}() to take the input MSR value in a single u64 argument, replacing the current dual u32 arguments. No functional change intended. Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/msr.h | 33 ++++++++++++--------------- arch/x86/include/asm/paravirt.h | 10 ++++---- arch/x86/include/asm/paravirt_types.h | 4 ++-- arch/x86/kernel/kvmclock.c | 2 +- arch/x86/kvm/svm/svm.c | 15 +++--------- arch/x86/xen/enlighten_pv.c | 13 +++++------ 6 files changed, 30 insertions(+), 47 deletions(-) diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index fb3d7c4cb774..121597fc5d41 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -91,12 +91,12 @@ static __always_inline unsigned long long __rdmsr(unsigned int msr) return EAX_EDX_VAL(val, low, high); } -static __always_inline void __wrmsr(unsigned int msr, u32 low, u32 high) +static __always_inline void __wrmsr(u32 msr, u64 val) { asm volatile("1: wrmsr\n" "2:\n" _ASM_EXTABLE_TYPE(1b, 2b, EX_TYPE_WRMSR) - : : "c" (msr), "a"(low), "d" (high) : "memory"); + : : "c" (msr), "a"((u32)val), "d" ((u32)(val >> 32)) : "memory"); } #define native_rdmsr(msr, val1, val2) \ @@ -112,11 +112,10 @@ static __always_inline u64 native_rdmsrl(const u32 msr) } #define native_wrmsr(msr, low, high) \ - __wrmsr(msr, low, high) + __wrmsr((msr), ((u64)(high) << 32) | (low)) #define native_wrmsrl(msr, val) \ - __wrmsr((msr), (u32)((u64)(val)), \ - (u32)((u64)(val) >> 32)) + __wrmsr((msr), (val)) static inline unsigned long long native_read_msr(unsigned int msr) { @@ -146,11 +145,8 @@ static inline unsigned long long native_read_msr_safe(unsigned int msr, } /* Can be uninlined because referenced by paravirt */ -static inline void notrace -native_write_msr(unsigned int msr, u32 low, u32 high) +static inline void notrace native_write_msr(u32 msr, u64 val) { - u64 val = (u64)high << 32 | low; - native_wrmsrl(msr, val); if (tracepoint_enabled(write_msr)) @@ -158,8 +154,7 @@ native_write_msr(unsigned int msr, u32 low, u32 high) } /* Can be uninlined because referenced by paravirt */ -static inline int notrace -native_write_msr_safe(unsigned int msr, u32 low, u32 high) +static inline int notrace native_write_msr_safe(u32 msr, u64 val) { int err; @@ -167,10 +162,10 @@ native_write_msr_safe(unsigned int msr, u32 low, u32 high) "2:\n\t" _ASM_EXTABLE_TYPE_REG(1b, 2b, EX_TYPE_WRMSR_SAFE, %[err]) : [err] "=a" (err) - : "c" (msr), "0" (low), "d" (high) + : "c" (msr), "0" ((u32)val), "d" ((u32)(val >> 32)) : "memory"); if (tracepoint_enabled(write_msr)) - do_trace_write_msr(msr, ((u64)high << 32 | low), err); + do_trace_write_msr(msr, val, err); return err; } @@ -258,23 +253,23 @@ do { \ (void)((high) = (u32)(__val >> 32)); \ } while (0) -static inline void wrmsr(unsigned int msr, u32 low, u32 high) +static inline void wrmsr(u32 msr, u32 low, u32 high) { - native_write_msr(msr, low, high); + native_write_msr(msr, (u64)high << 32 | low); } #define rdmsrl(msr, val) \ ((val) = native_read_msr((msr))) -static inline void wrmsrl(unsigned int msr, u64 val) +static inline void wrmsrl(u32 msr, u64 val) { - native_write_msr(msr, (u32)(val & 0xffffffffULL), (u32)(val >> 32)); + native_write_msr(msr, val); } /* wrmsr with exception handling */ -static inline int wrmsr_safe(unsigned int msr, u32 low, u32 high) +static inline int wrmsr_safe(u32 msr, u32 low, u32 high) { - return native_write_msr_safe(msr, low, high); + return native_write_msr_safe(msr, (u64)high << 32 | low); } /* rdmsr with exception handling */ diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index c4c23190925c..f3d6e8394d38 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -180,10 +180,9 @@ static inline u64 paravirt_read_msr(unsigned msr) return PVOP_CALL1(u64, cpu.read_msr, msr); } -static inline void paravirt_write_msr(unsigned msr, - unsigned low, unsigned high) +static inline void paravirt_write_msr(u32 msr, u32 low, u32 high) { - PVOP_VCALL3(cpu.write_msr, msr, low, high); + PVOP_VCALL2(cpu.write_msr, msr, (u64)high << 32 | low); } static inline u64 paravirt_read_msr_safe(unsigned msr, int *err) @@ -191,10 +190,9 @@ static inline u64 paravirt_read_msr_safe(unsigned msr, int *err) return PVOP_CALL2(u64, cpu.read_msr_safe, msr, err); } -static inline int paravirt_write_msr_safe(unsigned msr, - unsigned low, unsigned high) +static inline int paravirt_write_msr_safe(u32 msr, u32 low, u32 high) { - return PVOP_CALL3(int, cpu.write_msr_safe, msr, low, high); + return PVOP_CALL2(int, cpu.write_msr_safe, msr, (u64)high << 32 | low); } #define rdmsr(msr, val1, val2) \ diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 631c306ce1ff..78777b78da12 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -92,14 +92,14 @@ struct pv_cpu_ops { /* Unsafe MSR operations. These will warn or panic on failure. */ u64 (*read_msr)(unsigned int msr); - void (*write_msr)(unsigned int msr, unsigned low, unsigned high); + void (*write_msr)(u32 msr, u64 val); /* * Safe MSR operations. * read sets err to 0 or -EIO. write returns 0 or -EIO. */ u64 (*read_msr_safe)(unsigned int msr, int *err); - int (*write_msr_safe)(unsigned int msr, unsigned low, unsigned high); + int (*write_msr_safe)(u32 msr, u64 val); u64 (*read_pmc)(int counter); diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 5b2c15214a6b..6b4102365ae5 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -196,7 +196,7 @@ static void kvm_setup_secondary_clock(void) void kvmclock_disable(void) { if (msr_kvm_system_time) - native_write_msr(msr_kvm_system_time, 0, 0); + native_write_msr(msr_kvm_system_time, 0); } static void __init kvmclock_init_mem(void) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d5d0c5c3300b..5cbc4ccb145c 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -475,7 +475,6 @@ static void svm_inject_exception(struct kvm_vcpu *vcpu) static void svm_init_erratum_383(void) { - u32 low, high; int err; u64 val; @@ -489,10 +488,7 @@ static void svm_init_erratum_383(void) val |= (1ULL << 47); - low = lower_32_bits(val); - high = upper_32_bits(val); - - native_write_msr_safe(MSR_AMD64_DC_CFG, low, high); + native_write_msr_safe(MSR_AMD64_DC_CFG, val); erratum_383_found = true; } @@ -2167,17 +2163,12 @@ static bool is_erratum_383(void) /* Clear MCi_STATUS registers */ for (i = 0; i < 6; ++i) - native_write_msr_safe(MSR_IA32_MCx_STATUS(i), 0, 0); + native_write_msr_safe(MSR_IA32_MCx_STATUS(i), 0); value = native_read_msr_safe(MSR_IA32_MCG_STATUS, &err); if (!err) { - u32 low, high; - value &= ~(1ULL << 2); - low = lower_32_bits(value); - high = upper_32_bits(value); - - native_write_msr_safe(MSR_IA32_MCG_STATUS, low, high); + native_write_msr_safe(MSR_IA32_MCG_STATUS, value); } /* Flush tlb to evict multi-match entries */ diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 2bfe57469ac3..7401cce19939 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1165,9 +1165,9 @@ static void xen_do_write_msr(unsigned int msr, unsigned int low, if (!pmu_msr_write(msr, val)) { if (err) - *err = native_write_msr_safe(msr, low, high); + *err = native_write_msr_safe(msr, val); else - native_write_msr(msr, low, high); + native_write_msr(msr, val); } } } @@ -1177,12 +1177,11 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err) return xen_do_read_msr(msr, err); } -static int xen_write_msr_safe(unsigned int msr, unsigned int low, - unsigned int high) +static int xen_write_msr_safe(u32 msr, u64 val) { int err = 0; - xen_do_write_msr(msr, low, high, &err); + xen_do_write_msr(msr, val, (u32)(val >> 32), &err); return err; } @@ -1194,11 +1193,11 @@ static u64 xen_read_msr(unsigned int msr) return xen_do_read_msr(msr, xen_msr_safe ? &err : NULL); } -static void xen_write_msr(unsigned int msr, unsigned low, unsigned high) +static void xen_write_msr(u32 msr, u64 val) { int err; - xen_do_write_msr(msr, low, high, xen_msr_safe ? &err : NULL); + xen_do_write_msr(msr, val, (u32)(val >> 32), xen_msr_safe ? &err : NULL); } /* This is called once we have the cpu_possible_mask */ From patchwork Mon Mar 31 08:22:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877611 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C3111EEA37; Mon, 31 Mar 2025 08:24:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409469; cv=none; b=jmwj7KwBbR6y8vW+5gS34JYDZP+mm1jJhC1JaMjMz/MwspO0uaBS8SEJvksBbX6YrTOj9TxEYS1CTpxmhJwWxBPdYh97RHvLM6cUOILPUN4MYUQ2xIqv5A+LvUBzeT0aSaRAIQdQq932TWZAJ2MbebhKPrOpBGbHP3lO3KcYf5I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409469; c=relaxed/simple; bh=AoEHiNomUXTLBsZt4rMkNOk4470R/zK+4XLCkfKkmCw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z34d4oU7uiIok76wbKfTssq31WUaLC+M0ipzWiDwuIyjh6Xc0a0KouhBXxv77XjTx65cXY3kLPaZs8ErM4GbwwOExF1aj92eGMLFdFCjilBfFf8ZjUn3gc335SAWzF/2ouYfgCTG+/D8dU6+VuhTXJtAdXOiZHdVkO/WGiAfnUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=dojm+Dpg; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="dojm+Dpg" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp053171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:08 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp053171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409390; bh=S3swNgI99EyAKvvqMZiCU9IrqVA5GQts4Tvl6BiIxb4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dojm+DpgfYvbJxe4bv9ERf6lObdkKGws+H++OPxcP81M9c542DdQA77cLignnUuwF E0OyDovwZjUMlVfVuV/OGBGyfcYMnrDHCs4DbWXSLge6Hw07JbhQ3e4+A4YufL5+Vf KAcevYltBLXe8AzQ5l74EGDn8RcUVpT9sw3r7Q1O3VLmXW/ZMyQIqONrrgqJyt3R6Y hmw39rO0oJ0Tyqs5MFI9ajtUWEagdP4qRYRQ54wbuw8Vdoma5mWmh+l7VI4ay0q41q +p194X/jk0oM3oQCUzLC8VAVfkXKe7oge/M/8qV3o6il81Z3+5gOJ9d1cP9haH6+35 Xkyjs8fsNRJGA== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 05/15] x86/msr: Replace wrmsr(msr, low, 0) with wrmsrl(msr, value) Date: Mon, 31 Mar 2025 01:22:41 -0700 Message-ID: <20250331082251.3171276-6-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Xin Li (Intel) --- arch/x86/hyperv/hv_apic.c | 6 +++--- arch/x86/include/asm/apic.h | 2 +- arch/x86/include/asm/switch_to.h | 2 +- arch/x86/kernel/cpu/amd.c | 2 +- arch/x86/kernel/cpu/common.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 ++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- arch/x86/kernel/cpu/umwait.c | 4 ++-- arch/x86/kernel/kvm.c | 2 +- drivers/ata/pata_cs5535.c | 12 ++++++------ drivers/ata/pata_cs5536.c | 6 +++--- drivers/cpufreq/acpi-cpufreq.c | 2 +- drivers/cpufreq/e_powersaver.c | 2 +- drivers/cpufreq/powernow-k6.c | 8 ++++---- 14 files changed, 31 insertions(+), 31 deletions(-) diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c index 6d91ac5f9836..284e16fe359b 100644 --- a/arch/x86/hyperv/hv_apic.c +++ b/arch/x86/hyperv/hv_apic.c @@ -75,10 +75,10 @@ static void hv_apic_write(u32 reg, u32 val) { switch (reg) { case APIC_EOI: - wrmsr(HV_X64_MSR_EOI, val, 0); + wrmsrl(HV_X64_MSR_EOI, val); break; case APIC_TASKPRI: - wrmsr(HV_X64_MSR_TPR, val, 0); + wrmsrl(HV_X64_MSR_TPR, val); break; default: native_apic_mem_write(reg, val); @@ -92,7 +92,7 @@ static void hv_apic_eoi_write(void) if (hvp && (xchg(&hvp->apic_assist, 0) & 0x1)) return; - wrmsr(HV_X64_MSR_EOI, APIC_EOI_ACK, 0); + wrmsrl(HV_X64_MSR_EOI, APIC_EOI_ACK); } static bool cpu_is_self(int cpu) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 3345a819c859..003b2cd2266b 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -209,7 +209,7 @@ static inline void native_apic_msr_write(u32 reg, u32 v) reg == APIC_LVR) return; - wrmsr(APIC_BASE_MSR + (reg >> 4), v, 0); + wrmsrl(APIC_BASE_MSR + (reg >> 4), v); } static inline void native_apic_msr_eoi(void) diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h index 75248546403d..525896a18028 100644 --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -59,7 +59,7 @@ static inline void refresh_sysenter_cs(struct thread_struct *thread) return; this_cpu_write(cpu_tss_rw.x86_tss.ss1, thread->sysenter_cs); - wrmsr(MSR_IA32_SYSENTER_CS, thread->sysenter_cs, 0); + wrmsrl(MSR_IA32_SYSENTER_CS, thread->sysenter_cs); } #endif diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index 79569f72b8ee..2f70cd525043 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -1200,7 +1200,7 @@ void amd_set_dr_addr_mask(unsigned long mask, unsigned int dr) if (per_cpu(amd_dr_addr_mask, cpu)[dr] == mask) return; - wrmsr(amd_msr_dr_addr_masks[dr], mask, 0); + wrmsrl(amd_msr_dr_addr_masks[dr], mask); per_cpu(amd_dr_addr_mask, cpu)[dr] = mask; } diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a268db71d944..9b53f92df21c 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1982,9 +1982,9 @@ void enable_sep_cpu(void) */ tss->x86_tss.ss1 = __KERNEL_CS; - wrmsr(MSR_IA32_SYSENTER_CS, tss->x86_tss.ss1, 0); - wrmsr(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_entry_stack(cpu) + 1), 0); - wrmsr(MSR_IA32_SYSENTER_EIP, (unsigned long)entry_SYSENTER_32, 0); + wrmsrl(MSR_IA32_SYSENTER_CS, tss->x86_tss.ss1); + wrmsrl(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_entry_stack(cpu) + 1)); + wrmsrl(MSR_IA32_SYSENTER_EIP, (unsigned long)entry_SYSENTER_32); put_cpu(); } @@ -2198,7 +2198,7 @@ static inline void setup_getcpu(int cpu) struct desc_struct d = { }; if (boot_cpu_has(X86_FEATURE_RDTSCP) || boot_cpu_has(X86_FEATURE_RDPID)) - wrmsr(MSR_TSC_AUX, cpudata, 0); + wrmsrl(MSR_TSC_AUX, cpudata); /* Store CPU and node number in limit. */ d.limit0 = cpudata; diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 675fd9f93e33..44a9ac87b7be 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -903,7 +903,7 @@ int resctrl_arch_measure_cycles_lat_fn(void *_plr) * Disable hardware prefetchers. */ rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); - wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); + wrmsrl(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits); mem_r = READ_ONCE(plr->kmem); /* * Dummy execute of the time measurement to load the needed @@ -999,7 +999,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr, * Disable hardware prefetchers. */ rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); - wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); + wrmsrl(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits); /* Initialize rest of local variables */ /* diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index c6274d40b217..e5a4c283c924 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1697,7 +1697,7 @@ void resctrl_arch_mon_event_config_write(void *_config_info) pr_warn_once("Invalid event id %d\n", config_info->evtid); return; } - wrmsr(MSR_IA32_EVT_CFG_BASE + index, config_info->mon_config, 0); + wrmsrl(MSR_IA32_EVT_CFG_BASE + index, config_info->mon_config); } static void mbm_config_write_domain(struct rdt_resource *r, diff --git a/arch/x86/kernel/cpu/umwait.c b/arch/x86/kernel/cpu/umwait.c index 2293efd6ffa6..0f5d5d9f3352 100644 --- a/arch/x86/kernel/cpu/umwait.c +++ b/arch/x86/kernel/cpu/umwait.c @@ -33,7 +33,7 @@ static DEFINE_MUTEX(umwait_lock); static void umwait_update_control_msr(void * unused) { lockdep_assert_irqs_disabled(); - wrmsr(MSR_IA32_UMWAIT_CONTROL, READ_ONCE(umwait_control_cached), 0); + wrmsrl(MSR_IA32_UMWAIT_CONTROL, READ_ONCE(umwait_control_cached)); } /* @@ -71,7 +71,7 @@ static int umwait_cpu_offline(unsigned int cpu) * the original control MSR value in umwait_init(). So there * is no race condition here. */ - wrmsr(MSR_IA32_UMWAIT_CONTROL, orig_umwait_control_cached, 0); + wrmsrl(MSR_IA32_UMWAIT_CONTROL, orig_umwait_control_cached); return 0; } diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 3be9b3342c67..1b74aa64a1bc 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -399,7 +399,7 @@ static void kvm_disable_steal_time(void) if (!has_steal_clock) return; - wrmsr(MSR_KVM_STEAL_TIME, 0, 0); + wrmsrl(MSR_KVM_STEAL_TIME, 0); } static u64 kvm_steal_clock(int cpu) diff --git a/drivers/ata/pata_cs5535.c b/drivers/ata/pata_cs5535.c index d793fc441b46..b0ebd0fe31ed 100644 --- a/drivers/ata/pata_cs5535.c +++ b/drivers/ata/pata_cs5535.c @@ -102,16 +102,16 @@ static void cs5535_set_piomode(struct ata_port *ap, struct ata_device *adev) cmdmode = min(mode, pairmode); /* Write the other drive timing register if it changed */ if (cmdmode < pairmode) - wrmsr(ATAC_CH0D0_PIO + 2 * pair->devno, - pio_cmd_timings[cmdmode] << 16 | pio_timings[pairmode], 0); + wrmsrl(ATAC_CH0D0_PIO + 2 * pair->devno, + pio_cmd_timings[cmdmode] << 16 | pio_timings[pairmode]); } /* Write the drive timing register */ - wrmsr(ATAC_CH0D0_PIO + 2 * adev->devno, - pio_cmd_timings[cmdmode] << 16 | pio_timings[mode], 0); + wrmsrl(ATAC_CH0D0_PIO + 2 * adev->devno, + pio_cmd_timings[cmdmode] << 16 | pio_timings[mode]); /* Set the PIO "format 1" bit in the DMA timing register */ rdmsr(ATAC_CH0D0_DMA + 2 * adev->devno, reg, dummy); - wrmsr(ATAC_CH0D0_DMA + 2 * adev->devno, reg | 0x80000000UL, 0); + wrmsrl(ATAC_CH0D0_DMA + 2 * adev->devno, reg | 0x80000000UL); } /** @@ -138,7 +138,7 @@ static void cs5535_set_dmamode(struct ata_port *ap, struct ata_device *adev) reg |= udma_timings[mode - XFER_UDMA_0]; else reg |= mwdma_timings[mode - XFER_MW_DMA_0]; - wrmsr(ATAC_CH0D0_DMA + 2 * adev->devno, reg, 0); + wrmsrl(ATAC_CH0D0_DMA + 2 * adev->devno, reg); } static const struct scsi_host_template cs5535_sht = { diff --git a/drivers/ata/pata_cs5536.c b/drivers/ata/pata_cs5536.c index b811efd2cc34..0578f3046b51 100644 --- a/drivers/ata/pata_cs5536.c +++ b/drivers/ata/pata_cs5536.c @@ -34,9 +34,9 @@ module_param_named(msr, use_msr, int, 0644); MODULE_PARM_DESC(msr, "Force using MSR to configure IDE function (Default: 0)"); #else #undef rdmsr /* avoid accidental MSR usage on, e.g. x86-64 */ -#undef wrmsr +#undef wrmsrl #define rdmsr(x, y, z) do { } while (0) -#define wrmsr(x, y, z) do { } while (0) +#define wrmsrl(x, y) do { } while (0) #define use_msr 0 #endif @@ -98,7 +98,7 @@ static int cs5536_read(struct pci_dev *pdev, int reg, u32 *val) static int cs5536_write(struct pci_dev *pdev, int reg, int val) { if (unlikely(use_msr)) { - wrmsr(MSR_IDE_CFG + reg, val, 0); + wrmsrl(MSR_IDE_CFG + reg, val); return 0; } diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c index 924314cdeebc..937c07f0839f 100644 --- a/drivers/cpufreq/acpi-cpufreq.c +++ b/drivers/cpufreq/acpi-cpufreq.c @@ -271,7 +271,7 @@ static u32 cpu_freq_read_amd(struct acpi_pct_register *not_used) static void cpu_freq_write_amd(struct acpi_pct_register *not_used, u32 val) { - wrmsr(MSR_AMD_PERF_CTL, val, 0); + wrmsrl(MSR_AMD_PERF_CTL, val); } static u32 cpu_freq_read_io(struct acpi_pct_register *reg) diff --git a/drivers/cpufreq/e_powersaver.c b/drivers/cpufreq/e_powersaver.c index d23a97ba6478..08bf293eb4bb 100644 --- a/drivers/cpufreq/e_powersaver.c +++ b/drivers/cpufreq/e_powersaver.c @@ -123,7 +123,7 @@ static int eps_set_state(struct eps_cpu_data *centaur, } } /* Set new multiplier and voltage */ - wrmsr(MSR_IA32_PERF_CTL, dest_state & 0xffff, 0); + wrmsrl(MSR_IA32_PERF_CTL, dest_state & 0xffff); /* Wait until transition end */ i = 0; do { diff --git a/drivers/cpufreq/powernow-k6.c b/drivers/cpufreq/powernow-k6.c index 99d2244e03b0..d22a0b981797 100644 --- a/drivers/cpufreq/powernow-k6.c +++ b/drivers/cpufreq/powernow-k6.c @@ -88,10 +88,10 @@ static int powernow_k6_get_cpu_multiplier(void) local_irq_disable(); msrval = POWERNOW_IOPORT + 0x1; - wrmsr(MSR_K6_EPMR, msrval, 0); /* enable the PowerNow port */ + wrmsrl(MSR_K6_EPMR, msrval); /* enable the PowerNow port */ invalue = inl(POWERNOW_IOPORT + 0x8); msrval = POWERNOW_IOPORT + 0x0; - wrmsr(MSR_K6_EPMR, msrval, 0); /* disable it again */ + wrmsrl(MSR_K6_EPMR, msrval); /* disable it again */ local_irq_enable(); @@ -118,13 +118,13 @@ static void powernow_k6_set_cpu_multiplier(unsigned int best_i) outvalue = (1<<12) | (1<<10) | (1<<9) | (index_to_register[best_i]<<5); msrval = POWERNOW_IOPORT + 0x1; - wrmsr(MSR_K6_EPMR, msrval, 0); /* enable the PowerNow port */ + wrmsrl(MSR_K6_EPMR, msrval); /* enable the PowerNow port */ invalue = inl(POWERNOW_IOPORT + 0x8); invalue = invalue & 0x1f; outvalue = outvalue | invalue; outl(outvalue, (POWERNOW_IOPORT + 0x8)); msrval = POWERNOW_IOPORT + 0x0; - wrmsr(MSR_K6_EPMR, msrval, 0); /* disable it again */ + wrmsrl(MSR_K6_EPMR, msrval); /* disable it again */ write_cr0(cr0); local_irq_enable(); From patchwork Mon Mar 31 08:22:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877413 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB7591D5178; Mon, 31 Mar 2025 08:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; cv=none; b=HkD246eHWBdXWAl+dZdXvzP6sRofLJ6ufi0bBomNPdOsbLAsWRF04na0oyqvPnHGmcZ0S/RE2grKXZbCCZngX88ypLPMGyXDheq2d14ivMZFKgZDJPCCCOMZsXFwuL6h9MR7KwvMiJ9GVm1PpEHbBTSJ1ynT3UpRNxU5FXEEXbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; c=relaxed/simple; bh=vi8GjX+r00xIyasO6KKKsl413uOtTqQMjo6S+m99RGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AifxDEr6AHkHHSn17cVpQ2ZD0S0r7Df29hX4QHV2Y3WORwg2e/fLmROZyRuuTILW5cj6RVQQvR8WHF/ibEYJ+OqRRPi9WkLghl3tozpcriN6+/8PKmK2NhgbW6+idxKG+Vpuwd+8DFJZpJZSrajmHWx1ThwPPrPELj9I4gd+Svs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=LmuvsR6J; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="LmuvsR6J" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp063171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:10 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp063171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409392; bh=VWVlHnRAQ/URpBmLS0sboRyyUviRP0Ft54tlbnwKLbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LmuvsR6JU0wFg45aoeU1BmH+vTdjhSivLbBzn3nlmIbxQ1T8sqwq82VC74O3yYItX S9sFYj9ZngvuYrNW7sHGljfyh5Gqzhf6VwZ/aRtGB720vFbJ9EDDOtQ9E0xXh2PU0+ 7FTWRNpDPjfEZWo0yQCIVXmiHAQ6gMo2/Xgl8on4LA7UNyUv6fE4T91P+HIZpg7iX1 KkWnpYwKYYQj9Ur2Oo6+kBlMb0LDXwYZNWNFnRncQNe82SMqi3NRZs9EH5QHNmpSzd 3/S95C0un5h+YfWI71+I8tTNlrvTtz2S7X75vWWrxF52F7m0pJIvPFhhq4Yw7MnVfu weMHDHw5OzZNA== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 06/15] x86/msr: Remove MSR write APIs that take the MSR value in two u32 arguments Date: Mon, 31 Mar 2025 01:22:42 -0700 Message-ID: <20250331082251.3171276-7-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/msr.h | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index 121597fc5d41..da4f2f6d127f 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -111,9 +111,6 @@ static __always_inline u64 native_rdmsrl(const u32 msr) return __rdmsr(msr); } -#define native_wrmsr(msr, low, high) \ - __wrmsr((msr), ((u64)(high) << 32) | (low)) - #define native_wrmsrl(msr, val) \ __wrmsr((msr), (val)) @@ -253,11 +250,6 @@ do { \ (void)((high) = (u32)(__val >> 32)); \ } while (0) -static inline void wrmsr(u32 msr, u32 low, u32 high) -{ - native_write_msr(msr, (u64)high << 32 | low); -} - #define rdmsrl(msr, val) \ ((val) = native_read_msr((msr))) @@ -266,12 +258,6 @@ static inline void wrmsrl(u32 msr, u64 val) native_write_msr(msr, val); } -/* wrmsr with exception handling */ -static inline int wrmsr_safe(u32 msr, u32 low, u32 high) -{ - return native_write_msr_safe(msr, (u64)high << 32 | low); -} - /* rdmsr with exception handling */ #define rdmsr_safe(msr, low, high) \ ({ \ @@ -321,7 +307,7 @@ static __always_inline void wrmsrns(u32 msr, u64 val) */ static inline int wrmsrl_safe(u32 msr, u64 val) { - return wrmsr_safe(msr, (u32)val, (u32)(val >> 32)); + return native_write_msr_safe(msr, val); } struct msr __percpu *msrs_alloc(void); @@ -380,7 +366,7 @@ static inline int rdmsr_safe_on_cpu(unsigned int cpu, u32 msr_no, } static inline int wrmsr_safe_on_cpu(unsigned int cpu, u32 msr_no, u32 l, u32 h) { - return wrmsr_safe(msr_no, l, h); + return wrmsrl_safe(msr_no, (u64)h << 32 | l); } static inline int rdmsrl_safe_on_cpu(unsigned int cpu, u32 msr_no, u64 *q) { From patchwork Mon Mar 31 08:22:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877408 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3E9B1F03C9; Mon, 31 Mar 2025 08:24:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409470; cv=none; b=fdYw+R82rqsVQVM+WEr1N8Yr/L4qGW48pgyXMQkdX6mQW8Ea+nOHzPXiuuBxs6e6LgKIJp40bJkFXsZQbh1tFz2SSghfuJ+dyZE3yLdLIq9s/XmRz+KPQ08kD4zYtnkFJd7x5YAru7Zx0aZxetT5Nccz9RWQAh01rY0bZg0WYVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409470; c=relaxed/simple; bh=oYmM0CCry6cTlflAhfomz2ZkehnIFifV1gvK1EPpSk8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y7gySEXhbix7tjkWBUsXuhH5bPqKngVeqAaCEfQlrWNPfr318XDxM7S0iCcRYvNAH6cpuQZHiFfclMC67kQrJBJ5/ak5Hfed3DgE8FE7qXwAzmoZNZNF56dIelNHb/ZdOS5S7GM0i/YVZ2+sX9PiJQemWoRk8FEspSx+xtsWOFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=EVNNysVl; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="EVNNysVl" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp073171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:12 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp073171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409394; bh=4oMqi/mNiX/dP3yUNXOh0KHKk5eO4mtyZ/S9nY+rYCU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EVNNysVl8UKZVEUX8F7MbPFwX2epn0yrpmH2w+9q6c8158NF8qnthafY3Pkh8Ilxh 9M0ASaWQUpbHnDRwUDQa7VXHXLDjd40aeBCbAqUiExQtgZ+wsRgJSE1VkqGpirm+UR 5prXvLiFohME5Rh9uOv5Cw+cBYozzUd9b4kenjhMrVaaMr45h9Bi4/VNNbAxT8bv2/ Z8Dx3/k762OqkBR7GvNLYD2WZ0BEYXT78x4FpDvDc8rCAR5k7TIawz4/K3bBBGMueL W/Wh/zFdQkonvF67LZZkVO833IcrPosTMqlS6T0maUanzmLHMlKbYhc9HxoZ+xAFPV Y0GoucFLKizNQ== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 07/15] x86/msr: Remove pmu_msr_{read,write}() Date: Mon, 31 Mar 2025 01:22:43 -0700 Message-ID: <20250331082251.3171276-8-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now pmu_msr_{read,write}() just do pmu_msr_chk_emulated(), so remove them and call pmu_msr_chk_emulated() directly. Suggested-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li (Intel) --- arch/x86/xen/enlighten_pv.c | 17 ++++++++++------- arch/x86/xen/pmu.c | 24 ++++-------------------- arch/x86/xen/xen-ops.h | 3 +-- 3 files changed, 15 insertions(+), 29 deletions(-) diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 7401cce19939..a047dadf4511 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1090,8 +1090,9 @@ static void xen_write_cr4(unsigned long cr4) static u64 xen_do_read_msr(unsigned int msr, int *err) { u64 val = 0; /* Avoid uninitialized value for safe variant. */ + bool emulated; - if (pmu_msr_read(msr, &val, err)) + if (pmu_msr_chk_emulated(msr, &val, true, &emulated) && emulated) return val; if (err) @@ -1134,6 +1135,7 @@ static void xen_do_write_msr(unsigned int msr, unsigned int low, unsigned int high, int *err) { u64 val; + bool emulated; switch (msr) { case MSR_FS_BASE: @@ -1163,12 +1165,13 @@ static void xen_do_write_msr(unsigned int msr, unsigned int low, default: val = (u64)high << 32 | low; - if (!pmu_msr_write(msr, val)) { - if (err) - *err = native_write_msr_safe(msr, val); - else - native_write_msr(msr, val); - } + if (pmu_msr_chk_emulated(msr, &val, false, &emulated) && emulated) + return; + + if (err) + *err = native_write_msr_safe(msr, val); + else + native_write_msr(msr, val); } } diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c index 1364cd3fb3ef..4d20503430dd 100644 --- a/arch/x86/xen/pmu.c +++ b/arch/x86/xen/pmu.c @@ -128,7 +128,7 @@ static inline uint32_t get_fam15h_addr(u32 addr) return addr; } -static inline bool is_amd_pmu_msr(unsigned int msr) +static bool is_amd_pmu_msr(u32 msr) { if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD && boot_cpu_data.x86_vendor != X86_VENDOR_HYGON) @@ -194,8 +194,7 @@ static bool is_intel_pmu_msr(u32 msr_index, int *type, int *index) } } -static bool xen_intel_pmu_emulate(unsigned int msr, u64 *val, int type, - int index, bool is_read) +static bool xen_intel_pmu_emulate(u32 msr, u64 *val, int type, int index, bool is_read) { uint64_t *reg = NULL; struct xen_pmu_intel_ctxt *ctxt; @@ -257,7 +256,7 @@ static bool xen_intel_pmu_emulate(unsigned int msr, u64 *val, int type, return false; } -static bool xen_amd_pmu_emulate(unsigned int msr, u64 *val, bool is_read) +static bool xen_amd_pmu_emulate(u32 msr, u64 *val, bool is_read) { uint64_t *reg = NULL; int i, off = 0; @@ -298,8 +297,7 @@ static bool xen_amd_pmu_emulate(unsigned int msr, u64 *val, bool is_read) return false; } -static bool pmu_msr_chk_emulated(unsigned int msr, uint64_t *val, bool is_read, - bool *emul) +bool pmu_msr_chk_emulated(u32 msr, u64 *val, bool is_read, bool *emul) { int type, index = 0; @@ -313,20 +311,6 @@ static bool pmu_msr_chk_emulated(unsigned int msr, uint64_t *val, bool is_read, return true; } -bool pmu_msr_read(u32 msr, u64 *val, int *err) -{ - bool emulated; - - return pmu_msr_chk_emulated(msr, val, true, &emulated) && emulated; -} - -bool pmu_msr_write(u32 msr, u64 val) -{ - bool emulated; - - return pmu_msr_chk_emulated(msr, &val, false, &emulated) && emulated; -} - static unsigned long long xen_amd_read_pmc(int counter) { struct xen_pmu_amd_ctxt *ctxt; diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index 4a0a1d73d8b8..6545661010ce 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -274,8 +274,7 @@ void xen_pmu_finish(int cpu); static inline void xen_pmu_init(int cpu) {} static inline void xen_pmu_finish(int cpu) {} #endif -bool pmu_msr_read(u32 msr, u64 *val, int *err); -bool pmu_msr_write(u32 msr, u64 val); +bool pmu_msr_chk_emulated(u32 msr, u64 *val, bool is_read, bool *emul); int pmu_apic_update(uint32_t reg); unsigned long long xen_read_pmc(int counter); From patchwork Mon Mar 31 08:22:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877411 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9C4D1EB5CA; Mon, 31 Mar 2025 08:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409466; cv=none; b=u5f6pP53XCtskYLwDh4x7r02vYZOFStTcl1mXs/nvwnJ5jPcUD0mqHlYVZVlRaonfAE3HPz/hFqGRoq3kW1bebYWt//djE3viHTSrQHBTXwHOTlE6E/GUXJlPF85TMN9T0QPIOMD7aggyF9kuXkwKnrTwDle0zn9aE3L3tbg/LM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409466; c=relaxed/simple; bh=FN5g8giJJxRRxCOVXKjUG367o7XXihVRjnOAVof1uB0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WSa83U4b6t0/eOTnBjucFOxOxCDUgTxSlT0C+Mp6OecvukFNRzH5yjLMvdAhmDQ75eVVcXcxhyCBojDRK8j5v8FjND5/ygsoo0BrUglwdBurlfhzNnhEI7QmKlGLUKrik+KoujYT2WnmDUt36zBleJzdOjekAWEcWs7M/00sDKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=jg4QkVHW; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="jg4QkVHW" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp083171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:14 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp083171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409396; bh=LUf6J4OHKn6Yqz/cuiPFW5gyz7UWQzJR/hPRWef8Heg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jg4QkVHWmuNxTSaNr3aLQCDAx0pfc5MzZ6zHM+wqM8DSTcQ3v/RsKHxoUQpqWtNbJ m458JdmtvOkoM5D9ageqtKl9oHpcXzkGBie1zx4Z64jnAGFmPZKoyTrDoCRBO1dSLT mR0rsGe9H8URcFY2CBgpai+T0h9txzTfzfHRjKd6U95Ungph1Kwv+yEPzL4fhScW5l Ab8OKE5fJ02Tagt2yFWbB7GGJh/377QIshI1xuLWxMSYTo8yAwJfG0LGkBemhVlVjc tTZHvLjVyjNvdRXxd7vUQNm4YDAIWKVIT4qPZoTRRR4mD09RG0QuehmydgeXZydpxd 2Z/C61/mQ2MUw== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 08/15] x86/cpufeatures: Add a CPU feature bit for MSR immediate form instructions Date: Mon, 31 Mar 2025 01:22:44 -0700 Message-ID: <20250331082251.3171276-9-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The immediate form of MSR access instructions are primarily motivated by performance, not code size: by having the MSR number in an immediate, it is available *much* earlier in the pipeline, which allows the hardware much more leeway about how a particular MSR is handled. Use a scattered CPU feature bit for MSR immediate form instructions. Suggested-by: Borislav Petkov Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/cpufeatures.h | 19 ++++++++++--------- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 6c2c152d8a67..a742a3d34712 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -472,15 +472,16 @@ * * Reuse free bits when adding new feature flags! */ -#define X86_FEATURE_AMD_LBR_PMC_FREEZE (21*32+ 0) /* "amd_lbr_pmc_freeze" AMD LBR and PMC Freeze */ -#define X86_FEATURE_CLEAR_BHB_LOOP (21*32+ 1) /* Clear branch history at syscall entry using SW loop */ -#define X86_FEATURE_BHI_CTRL (21*32+ 2) /* BHI_DIS_S HW control available */ -#define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */ -#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */ -#define X86_FEATURE_AMD_FAST_CPPC (21*32 + 5) /* Fast CPPC */ -#define X86_FEATURE_AMD_HETEROGENEOUS_CORES (21*32 + 6) /* Heterogeneous Core Topology */ -#define X86_FEATURE_AMD_WORKLOAD_CLASS (21*32 + 7) /* Workload Classification */ -#define X86_FEATURE_PREFER_YMM (21*32 + 8) /* Avoid ZMM registers due to downclocking */ +#define X86_FEATURE_AMD_LBR_PMC_FREEZE (21*32+ 0) /* "amd_lbr_pmc_freeze" AMD LBR and PMC Freeze */ +#define X86_FEATURE_CLEAR_BHB_LOOP (21*32+ 1) /* Clear branch history at syscall entry using SW loop */ +#define X86_FEATURE_BHI_CTRL (21*32+ 2) /* BHI_DIS_S HW control available */ +#define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */ +#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */ +#define X86_FEATURE_AMD_FAST_CPPC (21*32+ 5) /* Fast CPPC */ +#define X86_FEATURE_AMD_HETEROGENEOUS_CORES (21*32+ 6) /* Heterogeneous Core Topology */ +#define X86_FEATURE_AMD_WORKLOAD_CLASS (21*32+ 7) /* Workload Classification */ +#define X86_FEATURE_PREFER_YMM (21*32+ 8) /* Avoid ZMM registers due to downclocking */ +#define X86_FEATURE_MSR_IMM (21*32+ 9) /* MSR immediate form instructions */ /* * BUG word(s) diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c index 16f3ca30626a..9eda656e9793 100644 --- a/arch/x86/kernel/cpu/scattered.c +++ b/arch/x86/kernel/cpu/scattered.c @@ -27,6 +27,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_APERFMPERF, CPUID_ECX, 0, 0x00000006, 0 }, { X86_FEATURE_EPB, CPUID_ECX, 3, 0x00000006, 0 }, { X86_FEATURE_INTEL_PPIN, CPUID_EBX, 0, 0x00000007, 1 }, + { X86_FEATURE_MSR_IMM, CPUID_ECX, 5, 0x00000007, 1 }, { X86_FEATURE_RRSBA_CTRL, CPUID_EDX, 2, 0x00000007, 2 }, { X86_FEATURE_BHI_CTRL, CPUID_EDX, 4, 0x00000007, 2 }, { X86_FEATURE_CQM_LLC, CPUID_EDX, 1, 0x0000000f, 0 }, From patchwork Mon Mar 31 08:22:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877618 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 298B215624D; Mon, 31 Mar 2025 08:24:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409454; cv=none; b=NvxnjxFZkujXxJj4T043/f+JkekR4VRwz9E0GkdljnxIwA/FjUeBXcLxK+q/hKd4xNPy8Ri7EazyV2VlT0iWmfkoBXMQh06S7Faq+uowv+OA1yEANneonkU7dWT2ZhCkjaM25MnuWYh+KLpA7hL/7TCrMEtAvt7dN8KwZC56hGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409454; c=relaxed/simple; bh=MlzZP2PcWDZrFa9hDnXv30fqYMM04NkLvN5K6Ig/4KA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=exGMydajc7kaCtwk5Oy+QFAR2wOShgW/Q1+m3uC+iWvj39z0eW1zMljZ9XrOVDf3cnDfY1034rtzFKEThammFXytTUs+jv5UVbUaNe4NL5wlOd9IOp8HJixs5tk/vvtOWUufC1pYoQ3tEQtgaJIuzUwhcbxOmdpOzMw0nqzvfsI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=m3OGyhoJ; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="m3OGyhoJ" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp093171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:16 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp093171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409398; bh=XaAm3SgdeuQpqxiLf3TgiBnvvEMb/sqRjazcaLW8DyU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=m3OGyhoJNr0Zsep5p1ViUs89IdqwpoEtbWuo2OsC8y9JAzzZbZXvjUjaRqzx5AUYK lJcmsGoVuGakNOlQ489XYhpn7UoCG2qfRQMcP3vMYw+NnWAKYOXHRdzMSDVDxQLIJJ kKOmdnvGQYVe7ioNqEmxq2m51haFE/snyyuHDZ7lhQTKPP3O+McTy73wS5FrEXpbEA GSpcgMRq/tDS9ikpfmvoHA9qWJyh5e7k4bNtkTdyO72QoWkxhdrF5ZM+BjwqxGvb8o Fxagag3lgbW6Apcxom4HGKuHb9Jh6FSWSc4zA4lH3CcX3khjRVDzEBBBmSENC1+wB9 70W8KGHT8R63g== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 09/15] x86/opcode: Add immediate form MSR instructions to x86-opcode-map Date: Mon, 31 Mar 2025 01:22:45 -0700 Message-ID: <20250331082251.3171276-10-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add the instruction opcodes used by the immediate form WRMSRNS/RDMSR to x86-opcode-map. Signed-off-by: Xin Li (Intel) --- arch/x86/lib/x86-opcode-map.txt | 5 +++-- tools/arch/x86/lib/x86-opcode-map.txt | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt index caedb3ef6688..e64f52321d6d 100644 --- a/arch/x86/lib/x86-opcode-map.txt +++ b/arch/x86/lib/x86-opcode-map.txt @@ -839,7 +839,7 @@ f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2) f2: ANDN Gy,By,Ey (v) f3: Grp17 (1A) f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) | WRUSSD/Q My,Gy (66) -f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSSD/Q My,Gy +f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSSD/Q My,Gy | RDMSR Rq,Gq (F2),(11B) | WRMSRNS Gq,Rq (F3),(11B) f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v) f8: MOVDIR64B Gv,Mdqq (66) | ENQCMD Gv,Mdqq (F2) | ENQCMDS Gv,Mdqq (F3) | URDMSR Rq,Gq (F2),(11B) | UWRMSR Gq,Rq (F3),(11B) f9: MOVDIRI My,Gy @@ -1014,7 +1014,7 @@ f1: CRC32 Gy,Ey (es) | CRC32 Gy,Ey (66),(es) | INVVPID Gy,Mdq (F3),(ev) f2: INVPCID Gy,Mdq (F3),(ev) f4: TZCNT Gv,Ev (es) | TZCNT Gv,Ev (66),(es) f5: LZCNT Gv,Ev (es) | LZCNT Gv,Ev (66),(es) -f6: Grp3_1 Eb (1A),(ev) +f6: Grp3_1 Eb (1A),(ev) | RDMSR Rq,Gq (F2),(11B),(ev) | WRMSRNS Gq,Rq (F3),(11B),(ev) f7: Grp3_2 Ev (1A),(es) f8: MOVDIR64B Gv,Mdqq (66),(ev) | ENQCMD Gv,Mdqq (F2),(ev) | ENQCMDS Gv,Mdqq (F3),(ev) | URDMSR Rq,Gq (F2),(11B),(ev) | UWRMSR Gq,Rq (F3),(11B),(ev) f9: MOVDIRI My,Gy (ev) @@ -1103,6 +1103,7 @@ EndTable Table: VEX map 7 Referrer: AVXcode: 7 +f6: RDMSR Rq,Id (F2),(v1),(11B) | WRMSRNS Id,Rq (F3),(v1),(11B) f8: URDMSR Rq,Id (F2),(v1),(11B) | UWRMSR Id,Rq (F3),(v1),(11B) EndTable diff --git a/tools/arch/x86/lib/x86-opcode-map.txt b/tools/arch/x86/lib/x86-opcode-map.txt index caedb3ef6688..e64f52321d6d 100644 --- a/tools/arch/x86/lib/x86-opcode-map.txt +++ b/tools/arch/x86/lib/x86-opcode-map.txt @@ -839,7 +839,7 @@ f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2) f2: ANDN Gy,By,Ey (v) f3: Grp17 (1A) f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) | WRUSSD/Q My,Gy (66) -f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSSD/Q My,Gy +f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSSD/Q My,Gy | RDMSR Rq,Gq (F2),(11B) | WRMSRNS Gq,Rq (F3),(11B) f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v) f8: MOVDIR64B Gv,Mdqq (66) | ENQCMD Gv,Mdqq (F2) | ENQCMDS Gv,Mdqq (F3) | URDMSR Rq,Gq (F2),(11B) | UWRMSR Gq,Rq (F3),(11B) f9: MOVDIRI My,Gy @@ -1014,7 +1014,7 @@ f1: CRC32 Gy,Ey (es) | CRC32 Gy,Ey (66),(es) | INVVPID Gy,Mdq (F3),(ev) f2: INVPCID Gy,Mdq (F3),(ev) f4: TZCNT Gv,Ev (es) | TZCNT Gv,Ev (66),(es) f5: LZCNT Gv,Ev (es) | LZCNT Gv,Ev (66),(es) -f6: Grp3_1 Eb (1A),(ev) +f6: Grp3_1 Eb (1A),(ev) | RDMSR Rq,Gq (F2),(11B),(ev) | WRMSRNS Gq,Rq (F3),(11B),(ev) f7: Grp3_2 Ev (1A),(es) f8: MOVDIR64B Gv,Mdqq (66),(ev) | ENQCMD Gv,Mdqq (F2),(ev) | ENQCMDS Gv,Mdqq (F3),(ev) | URDMSR Rq,Gq (F2),(11B),(ev) | UWRMSR Gq,Rq (F3),(11B),(ev) f9: MOVDIRI My,Gy (ev) @@ -1103,6 +1103,7 @@ EndTable Table: VEX map 7 Referrer: AVXcode: 7 +f6: RDMSR Rq,Id (F2),(v1),(11B) | WRMSRNS Id,Rq (F3),(v1),(11B) f8: URDMSR Rq,Id (F2),(v1),(11B) | UWRMSR Id,Rq (F3),(v1),(11B) EndTable From patchwork Mon Mar 31 08:22:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877614 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3A511EA7D7; Mon, 31 Mar 2025 08:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409465; cv=none; b=qRVE1pZPaIBYHTJ+0DOcGy9L5vPOJs4jXsd0IgqyMe78rm7Z8vPTLvkL4EcRQWXLX7ne07EmF1/yY6njGtkSzWcnTum+HoXBMGo32lVydD/uBToirjJ1HkDpgHU//R8nJLl8gSK+rCMB8V1XY2w7EgJFyWt5n2fXfLX7QXsLHMo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409465; c=relaxed/simple; bh=a/q0kPgNYF3+FXDewvszN3tEL/ZiGdovN3OV/WHf8Rc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BehquKuiQFeIRuSVX2Ccc1YqoIJBh7OejEYGfIL0ivQalJFaHFBJa0Ck7nw5Y9v2I/aqNJyvvM4Q5hQjOdrcqJ70S6rwDSHbWqiltxgjvw/HQp5M/osx2u+VjicGcbSkOJ2o+0vwYo/wvGquINpEae4pULA6wGyMnpK7gYE0NuQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=fYJJM8Xj; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="fYJJM8Xj" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0A3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:18 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0A3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409400; bh=K1q53eo/E11QGaBNCmuuiEewoEjYIhJFKtCNyzdMWfk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fYJJM8Xj8oLRQMrpq1oZ9Rnts/L5sB1pkc/gbLLiuGLqwOG9qwxAGu/4BG68QY0E9 wfBdSwh4eQvvh2/Y3Z6WaX7t7pE5x/AyF9EYYVoeYUsrgjy5aQsLhKl3r8wbyqsS8o qh1pRd13tdt4PmpmLxqluvo/m8RcpuW2x76HH/JRrb1yV8WiUIE4hCpEtDNED/Yh+6 sNVHvloh/CyqgKztI+Tc3MQqU27RJiZ6HQl+pwJmYWKADo2kfVmQTIr7agdhUDKEDb anOJ6HwZplUeZ1KVIk1BWCIusZV2JYQ1KiyYhKxPqQ6qOrnhQXCF8X6edaiE7NigVe yW2IFXjim5dUQ== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 10/15] KVM: VMX: Use WRMSRNS or its immediate form when available Date: Mon, 31 Mar 2025 01:22:46 -0700 Message-ID: <20250331082251.3171276-11-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/msr-index.h | 6 ++++++ arch/x86/kvm/vmx/vmenter.S | 28 ++++++++++++++++++++++++---- 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index e6134ef2263d..04244c3ba374 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1226,4 +1226,10 @@ * a #GP */ +/* Instruction opcode for WRMSRNS supported in binutils >= 2.40 */ +#define ASM_WRMSRNS _ASM_BYTES(0x0f,0x01,0xc6) + +/* Instruction opcode for the immediate form RDMSR/WRMSRNS */ +#define ASM_WRMSRNS_RAX _ASM_BYTES(0xc4,0xe7,0x7a,0xf6,0xc0) + #endif /* _ASM_X86_MSR_INDEX_H */ diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index f6986dee6f8c..9fae43723c44 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -64,6 +64,29 @@ RET .endm +/* + * Write EAX to MSR_IA32_SPEC_CTRL. + * + * Choose the best WRMSR instruction based on availability. + * + * Replace with 'wrmsrns' and 'wrmsrns %rax, $MSR_IA32_SPEC_CTRL' once binutils support them. + */ +.macro WRITE_EAX_TO_MSR_IA32_SPEC_CTRL + ALTERNATIVE_2 __stringify(mov $MSR_IA32_SPEC_CTRL, %ecx; \ + xor %edx, %edx; \ + mov %edi, %eax; \ + ds wrmsr), \ + __stringify(mov $MSR_IA32_SPEC_CTRL, %ecx; \ + xor %edx, %edx; \ + mov %edi, %eax; \ + ASM_WRMSRNS), \ + X86_FEATURE_WRMSRNS, \ + __stringify(xor %_ASM_AX, %_ASM_AX; \ + mov %edi, %eax; \ + ASM_WRMSRNS_RAX; .long MSR_IA32_SPEC_CTRL), \ + X86_FEATURE_MSR_IMM +.endm + .section .noinstr.text, "ax" /** @@ -123,10 +146,7 @@ SYM_FUNC_START(__vmx_vcpu_run) movl PER_CPU_VAR(x86_spec_ctrl_current), %esi cmp %edi, %esi je .Lspec_ctrl_done - mov $MSR_IA32_SPEC_CTRL, %ecx - xor %edx, %edx - mov %edi, %eax - wrmsr + WRITE_EAX_TO_MSR_IA32_SPEC_CTRL .Lspec_ctrl_done: From patchwork Mon Mar 31 08:22:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877613 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC99E1EC00C; Mon, 31 Mar 2025 08:24:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409467; cv=none; b=Eic+EO844hK7JLGqFhW391oAh3ZW4P55IsgbmFP0EzxaKPAAOTWn0E1XbbkeeUtgv07yCVddsYfSYR/C4wRuQqnNvOF/hzFhDgY5COU/Qc61tZ5R75vEjy3LZ33/6LtolmwMU/bQ0nCBi9zyRgjkk4U2nWWaA1fpkLWwOKwPMf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409467; c=relaxed/simple; bh=o9yCUyHRvdZVCtPxYF8n3kIoSQXy7DN2Skw+gaOuRpY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LRxqdXoRIukHh8pTKSWrXv/ZLdyVZFGombsMN002SVXgfB6SFrYW8fMTWvmaR0zRixw7ydpzzJfu4wFwgaQ+Rp7PuvhmyXLl67FRHjUlDQign9hvJJwDKSURkCdFE2H8oDRQdDKsbCEkSIK3iFTywZMP/ynIDjehb1wZlpyWPOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=PnRobvTi; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="PnRobvTi" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0B3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:20 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0B3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409402; bh=AliY8eRLGXH0M9T1YLICUMbSBraUzdLknMsijvvtjSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PnRobvTiAXfnZAA5cVvRKq7KE/T/TrORn+EtVp1jCTqA8ycun7Wucl8DJLIO71N2/ uSEpnYPaUafO8DpxST6jufpuHWqLkzTNOnaZtZniIJpVjolE8yipeVlu40jxS3AxVz SQXxQMsab++sLuUC9KkA/PcH3r95hKodFUqGxFZQ9js7SuO8Is0I08vsCNtKya5bjI mij9VKqWcRygtaCmMG+i1TdPNItSumUg5xfXgt/1D8fSuKbEzPD6bPqWH0ZclRLlBQ yjIIY+aIXe/tviO8O3oyYbGBtYyk40Hm4yPMyYxfnoNqsAUOTo4DSLZ+1BUmRmGqR4 89R6xqzhUgPLQ== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 11/15] x86/extable: Implement EX_TYPE_FUNC_REWIND Date: Mon, 31 Mar 2025 01:22:47 -0700 Message-ID: <20250331082251.3171276-12-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "H. Peter Anvin (Intel)" Add a new exception type, which allows emulating an exception as if it had happened at or near the call site of a function. This allows a function call inside an alternative for instruction emulation to "kick back" the exception into the alternatives pattern, possibly invoking a different exception handling pattern there, or at least indicating the "real" location of the fault. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/asm.h | 6 + arch/x86/include/asm/extable_fixup_types.h | 1 + arch/x86/mm/extable.c | 135 +++++++++++++-------- 3 files changed, 91 insertions(+), 51 deletions(-) diff --git a/arch/x86/include/asm/asm.h b/arch/x86/include/asm/asm.h index cc2881576c2c..c05c33653194 100644 --- a/arch/x86/include/asm/asm.h +++ b/arch/x86/include/asm/asm.h @@ -243,5 +243,11 @@ register unsigned long current_stack_pointer asm(_ASM_SP); #define _ASM_EXTABLE_FAULT(from, to) \ _ASM_EXTABLE_TYPE(from, to, EX_TYPE_FAULT) +#define _ASM_EXTABLE_FUNC_REWIND(from, ipdelta, spdelta) \ + _ASM_EXTABLE_TYPE(from, from /* unused */, \ + EX_TYPE_FUNC_REWIND | \ + EX_DATA_REG(spdelta) | \ + EX_DATA_IMM(ipdelta)) + #endif /* __KERNEL__ */ #endif /* _ASM_X86_ASM_H */ diff --git a/arch/x86/include/asm/extable_fixup_types.h b/arch/x86/include/asm/extable_fixup_types.h index 906b0d5541e8..9cd1cea45052 100644 --- a/arch/x86/include/asm/extable_fixup_types.h +++ b/arch/x86/include/asm/extable_fixup_types.h @@ -67,5 +67,6 @@ #define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ #define EX_TYPE_ERETU 21 +#define EX_TYPE_FUNC_REWIND 22 #endif diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 51986e8a9d35..eb9331240a88 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -290,6 +290,27 @@ static bool ex_handler_eretu(const struct exception_table_entry *fixup, } #endif +/* + * Emulate a fault taken at the call site of a function. + * + * The combined reg and flags field are used as an unsigned number of + * machine words to pop off the stack before the return address, then + * the signed imm field is used as a delta from the return IP address. + */ +static bool ex_handler_func_rewind(struct pt_regs *regs, int data) +{ + const long ipdelta = FIELD_GET(EX_DATA_IMM_MASK, data); + const unsigned long pops = FIELD_GET(EX_DATA_REG_MASK | EX_DATA_FLAG_MASK, data); + unsigned long *sp; + + sp = (unsigned long *)regs->sp; + sp += pops; + regs->ip = *sp++ + ipdelta; + regs->sp = (unsigned long)sp; + + return true; +} + int ex_get_fixup_type(unsigned long ip) { const struct exception_table_entry *e = search_exception_tables(ip); @@ -302,6 +323,7 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, { const struct exception_table_entry *e; int type, reg, imm; + bool again; #ifdef CONFIG_PNPBIOS if (unlikely(SEGMENT_IS_PNP_CODE(regs->cs))) { @@ -317,60 +339,71 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, } #endif - e = search_exception_tables(regs->ip); - if (!e) - return 0; - - type = FIELD_GET(EX_DATA_TYPE_MASK, e->data); - reg = FIELD_GET(EX_DATA_REG_MASK, e->data); - imm = FIELD_GET(EX_DATA_IMM_MASK, e->data); - - switch (type) { - case EX_TYPE_DEFAULT: - case EX_TYPE_DEFAULT_MCE_SAFE: - return ex_handler_default(e, regs); - case EX_TYPE_FAULT: - case EX_TYPE_FAULT_MCE_SAFE: - return ex_handler_fault(e, regs, trapnr); - case EX_TYPE_UACCESS: - return ex_handler_uaccess(e, regs, trapnr, fault_addr); - case EX_TYPE_CLEAR_FS: - return ex_handler_clear_fs(e, regs); - case EX_TYPE_FPU_RESTORE: - return ex_handler_fprestore(e, regs); - case EX_TYPE_BPF: - return ex_handler_bpf(e, regs); - case EX_TYPE_WRMSR: - return ex_handler_msr(e, regs, true, false, reg); - case EX_TYPE_RDMSR: - return ex_handler_msr(e, regs, false, false, reg); - case EX_TYPE_WRMSR_SAFE: - return ex_handler_msr(e, regs, true, true, reg); - case EX_TYPE_RDMSR_SAFE: - return ex_handler_msr(e, regs, false, true, reg); - case EX_TYPE_WRMSR_IN_MCE: - ex_handler_msr_mce(regs, true); - break; - case EX_TYPE_RDMSR_IN_MCE: - ex_handler_msr_mce(regs, false); - break; - case EX_TYPE_POP_REG: - regs->sp += sizeof(long); - fallthrough; - case EX_TYPE_IMM_REG: - return ex_handler_imm_reg(e, regs, reg, imm); - case EX_TYPE_FAULT_SGX: - return ex_handler_sgx(e, regs, trapnr); - case EX_TYPE_UCOPY_LEN: - return ex_handler_ucopy_len(e, regs, trapnr, fault_addr, reg, imm); - case EX_TYPE_ZEROPAD: - return ex_handler_zeropad(e, regs, fault_addr); + do { + e = search_exception_tables(regs->ip); + if (!e) + return 0; + + again = false; + + type = FIELD_GET(EX_DATA_TYPE_MASK, e->data); + reg = FIELD_GET(EX_DATA_REG_MASK, e->data); + imm = FIELD_GET(EX_DATA_IMM_MASK, e->data); + + switch (type) { + case EX_TYPE_DEFAULT: + case EX_TYPE_DEFAULT_MCE_SAFE: + return ex_handler_default(e, regs); + case EX_TYPE_FAULT: + case EX_TYPE_FAULT_MCE_SAFE: + return ex_handler_fault(e, regs, trapnr); + case EX_TYPE_UACCESS: + return ex_handler_uaccess(e, regs, trapnr, fault_addr); + case EX_TYPE_CLEAR_FS: + return ex_handler_clear_fs(e, regs); + case EX_TYPE_FPU_RESTORE: + return ex_handler_fprestore(e, regs); + case EX_TYPE_BPF: + return ex_handler_bpf(e, regs); + case EX_TYPE_WRMSR: + return ex_handler_msr(e, regs, true, false, reg); + case EX_TYPE_RDMSR: + return ex_handler_msr(e, regs, false, false, reg); + case EX_TYPE_WRMSR_SAFE: + return ex_handler_msr(e, regs, true, true, reg); + case EX_TYPE_RDMSR_SAFE: + return ex_handler_msr(e, regs, false, true, reg); + case EX_TYPE_WRMSR_IN_MCE: + ex_handler_msr_mce(regs, true); + break; + case EX_TYPE_RDMSR_IN_MCE: + ex_handler_msr_mce(regs, false); + break; + case EX_TYPE_POP_REG: + regs->sp += sizeof(long); + fallthrough; + case EX_TYPE_IMM_REG: + return ex_handler_imm_reg(e, regs, reg, imm); + case EX_TYPE_FAULT_SGX: + return ex_handler_sgx(e, regs, trapnr); + case EX_TYPE_UCOPY_LEN: + return ex_handler_ucopy_len(e, regs, trapnr, fault_addr, reg, imm); + case EX_TYPE_ZEROPAD: + return ex_handler_zeropad(e, regs, fault_addr); #ifdef CONFIG_X86_FRED - case EX_TYPE_ERETU: - return ex_handler_eretu(e, regs, error_code); + case EX_TYPE_ERETU: + return ex_handler_eretu(e, regs, error_code); #endif - } + case EX_TYPE_FUNC_REWIND: + again = ex_handler_func_rewind(regs, e->data); + break; + default: + break; /* Will BUG() */ + } + } while (again); + BUG(); + return 0; } extern unsigned int early_recursion_flag; From patchwork Mon Mar 31 08:22:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877410 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33B901E7C03; Mon, 31 Mar 2025 08:24:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; cv=none; b=EAdRub3EgrsqpvJBrfwQky2hFYcysC8JARzQWBdGcKoLpQilfluiMYgFrLFXVigCf5RqisWv2hVDBHOYrVM3wM703OAIxVII2UnNKnVm5i+H5cBxT6ekn8xqE2Ar26FcgUH9TVLrp1UuRQp3L6zCjRxC1Z38xX7zMcl0lRySjIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409468; c=relaxed/simple; bh=bDcjpXAKJC+ghOaegd6KXIC08jV5/IcjH/5FkEK+PDc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YYynPxRlUyMLwStLvVLYyPzqeEA+osOpRxeOTne9HP7NAz4/59FphCA4PNliIW646VsdPbbinRjP++C4kpTl/2qiD+rxwt70GpYRZFl4Z6oO42xsa+NSE6PaMVagZK7np4sQfT8qSVVwXKCbKVSqp9GzmHdEXDNnRUtROxhiElk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=YQm0yq2/; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="YQm0yq2/" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0C3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:22 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0C3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409404; bh=NQD+TN3+0whhA/vOhooP6nsjLYQCN7OafGSYujp9CME=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YQm0yq2/abEsmYWAyVQbq5tQv70sDODfxwc7WwdlTgBlPxMIbMLpLMH0zVC9Lcc7p Qvuu6AGlbcJx1X1yi5tNXQ5AKhi8sR5rEB9RkE6QsQL5YjMdPz8fU2pc5ce/HEbaab pUpzIeMuYD0iFpFUHOnJVdPXd2WtqkGUYx9CVAc5El2OjF8wD56yIUXJ4kaoH1bad9 SJRD+yopao3aWS761ht+LyTO99joEjJ6c9vMcU/A45DJvYphj3Yt7CI/55kNcF9glQ eECNCo/JvLAwqkbaYdbrxsCEMwYktqL28wYgtJYsHjujTsS41z8JotrbNwa4TedKYG jPVVW5zxNNySA== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 12/15] x86/msr: Use the alternatives mechanism to write MSR Date: Mon, 31 Mar 2025 01:22:48 -0700 Message-ID: <20250331082251.3171276-13-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Also add support for the immediate form MSR write support. Originally-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/fred.h | 2 +- arch/x86/include/asm/msr.h | 340 ++++++++++++++++++++++---- arch/x86/include/asm/paravirt.h | 22 -- arch/x86/include/asm/paravirt_types.h | 2 - arch/x86/kernel/paravirt.c | 2 - arch/x86/xen/enlighten_pv.c | 63 ++--- arch/x86/xen/xen-asm.S | 55 +++++ arch/x86/xen/xen-ops.h | 2 + 8 files changed, 362 insertions(+), 126 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 2a29e5216881..e6eab64095d4 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -100,7 +100,7 @@ static __always_inline void fred_update_rsp0(void) unsigned long rsp0 = (unsigned long) task_stack_page(current) + THREAD_SIZE; if (cpu_feature_enabled(X86_FEATURE_FRED) && (__this_cpu_read(fred_rsp0) != rsp0)) { - wrmsrns(MSR_IA32_FRED_RSP0, rsp0); + native_wrmsrl(MSR_IA32_FRED_RSP0, rsp0); __this_cpu_write(fred_rsp0, rsp0); } } diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index da4f2f6d127f..066cde11254a 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -8,6 +8,7 @@ #include #include +#include #include #include #include @@ -72,13 +73,83 @@ static inline void do_trace_read_msr(unsigned int msr, u64 val, int failed) {} static inline void do_trace_rdpmc(unsigned int msr, u64 val, int failed) {} #endif +#ifdef CONFIG_CC_IS_GCC +#define ASM_WRMSRNS_IMM \ + " .insn VEX.128.F3.M7.W0 0xf6 /0, %[val], %[msr]%{:u32}\n\t" +#endif + +#ifdef CONFIG_CC_IS_CLANG /* - * __rdmsr() and __wrmsr() are the two primitives which are the bare minimum MSR - * accessors and should not have any tracing or other functionality piggybacking - * on them - those are *purely* for accessing MSRs and nothing more. So don't even - * think of extending them - you will be slapped with a stinking trout or a frozen - * shark will reach you, wherever you are! You've been warned. + * clang doesn't support the insn directive. + * + * The register operand is encoded as %rax because all uses of the immediate + * form MSR access instructions reference %rax as the register operand. */ +#define ASM_WRMSRNS_IMM \ + " .byte 0xc4,0xe7,0x7a,0xf6,0xc0; .long %c[msr]" +#endif + +#define PREPARE_RDX_FOR_WRMSR \ + "mov %%rax, %%rdx\n\t" \ + "shr $0x20, %%rdx\n\t" + +#define PREPARE_RCX_RDX_FOR_WRMSR \ + "mov %[msr], %%ecx\n\t" \ + PREPARE_RDX_FOR_WRMSR + +enum pv_msr_action { + PV_MSR_NATIVE, + PV_MSR_PV, + PV_MSR_IGNORE, +}; + +#ifdef CONFIG_XEN_PV +static __always_inline enum pv_msr_action get_pv_msr_action(const u32 msr) +{ + if (!__builtin_constant_p(msr)) { + /* Is it safe to blindly do so? */ + return PV_MSR_NATIVE; + } + + switch (msr) { + case MSR_FS_BASE: + case MSR_KERNEL_GS_BASE: + case MSR_GS_BASE: + case MSR_CORE_PERF_GLOBAL_OVF_CTRL: + case MSR_CORE_PERF_GLOBAL_STATUS: + case MSR_CORE_PERF_GLOBAL_CTRL: + case MSR_CORE_PERF_FIXED_CTR_CTRL: + case MSR_IA32_APICBASE: + return PV_MSR_PV; + + case MSR_STAR: + case MSR_CSTAR: + case MSR_LSTAR: + case MSR_SYSCALL_MASK: + case MSR_IA32_SYSENTER_CS: + case MSR_IA32_SYSENTER_ESP: + case MSR_IA32_SYSENTER_EIP: + return PV_MSR_IGNORE; + + default: + /* + * MSR access instructions RDMSR/WRMSR/WRMSRNS will be used. + * + * The hypervisor will trap and inject #GP into the guest and + * the MSR access instruction will be skipped. + */ + return PV_MSR_NATIVE; + } +} + +extern void asm_xen_write_msr(void); +#else +static __always_inline enum pv_msr_action get_pv_msr_action(const u32 msr) +{ + return PV_MSR_NATIVE; +} +#endif + static __always_inline unsigned long long __rdmsr(unsigned int msr) { DECLARE_ARGS(val, low, high); @@ -91,14 +162,6 @@ static __always_inline unsigned long long __rdmsr(unsigned int msr) return EAX_EDX_VAL(val, low, high); } -static __always_inline void __wrmsr(u32 msr, u64 val) -{ - asm volatile("1: wrmsr\n" - "2:\n" - _ASM_EXTABLE_TYPE(1b, 2b, EX_TYPE_WRMSR) - : : "c" (msr), "a"((u32)val), "d" ((u32)(val >> 32)) : "memory"); -} - #define native_rdmsr(msr, val1, val2) \ do { \ u64 __val = __rdmsr((msr)); \ @@ -111,9 +174,6 @@ static __always_inline u64 native_rdmsrl(const u32 msr) return __rdmsr(msr); } -#define native_wrmsrl(msr, val) \ - __wrmsr((msr), (val)) - static inline unsigned long long native_read_msr(unsigned int msr) { unsigned long long val; @@ -141,31 +201,232 @@ static inline unsigned long long native_read_msr_safe(unsigned int msr, return EAX_EDX_VAL(val, low, high); } -/* Can be uninlined because referenced by paravirt */ -static inline void notrace native_write_msr(u32 msr, u64 val) +/* + * Non-serializing WRMSR, when available. + * Falls back to a serializing WRMSR. + */ +static __always_inline bool __native_wrmsr_variable(const u32 msr, const u64 val, const int type) +{ +#ifdef CONFIG_X86_64 + BUILD_BUG_ON(__builtin_constant_p(msr)); +#endif + + asm_inline volatile goto( + "1:\n" + ALTERNATIVE("ds wrmsr", + ASM_WRMSRNS, + X86_FEATURE_WRMSRNS) + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) + + : + : "c" (msr), "a" ((u32)val), "d" ((u32)(val >> 32)), [type] "i" (type) + : "memory" + : badmsr); + + return false; + +badmsr: + return true; +} + +#ifdef CONFIG_X86_64 +/* + * Non-serializing WRMSR or its immediate form, when available. + * Falls back to a serializing WRMSR. + */ +static __always_inline bool __native_wrmsr_constant(const u32 msr, const u64 val, const int type) +{ + BUILD_BUG_ON(!__builtin_constant_p(msr)); + + /* + * WRMSR is 2 bytes. WRMSRNS is 3 bytes. Pad WRMSR with a redundant + * DS prefix to avoid a trailing NOP. + */ + asm_inline volatile goto( + "1:\n" + ALTERNATIVE_2(PREPARE_RCX_RDX_FOR_WRMSR + "2: ds wrmsr", + PREPARE_RCX_RDX_FOR_WRMSR + ASM_WRMSRNS, + X86_FEATURE_WRMSRNS, + ASM_WRMSRNS_IMM, + X86_FEATURE_MSR_IMM) + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For WRMSRNS immediate */ + _ASM_EXTABLE_TYPE(2b, %l[badmsr], %c[type]) /* For WRMSR(NS) */ + + : + : [val] "a" (val), [msr] "i" (msr), [type] "i" (type) + : "memory", "ecx", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +} +#endif + +static __always_inline bool __native_wrmsr(const u32 msr, const u64 val, const int type) +{ +#ifdef CONFIG_X86_64 + if (__builtin_constant_p(msr)) + return __native_wrmsr_constant(msr, val, type); +#endif + + return __native_wrmsr_variable(msr, val, type); +} + +static __always_inline void native_wrmsr(const u32 msr, const u32 low, const u32 high) +{ + __native_wrmsr(msr, (u64)high << 32 | low, EX_TYPE_WRMSR); +} + +static __always_inline void native_wrmsrl(const u32 msr, const u64 val) +{ + __native_wrmsr(msr, val, EX_TYPE_WRMSR); +} + +static inline void notrace native_write_msr(const u32 msr, const u64 val) { - native_wrmsrl(msr, val); + __native_wrmsr(msr, val, EX_TYPE_WRMSR); if (tracepoint_enabled(write_msr)) do_trace_write_msr(msr, val, 0); } -/* Can be uninlined because referenced by paravirt */ -static inline int notrace native_write_msr_safe(u32 msr, u64 val) +static inline int notrace native_write_msr_safe(const u32 msr, const u64 val) { - int err; + int err = __native_wrmsr(msr, val, EX_TYPE_WRMSR_SAFE) ? -EIO : 0; - asm volatile("1: wrmsr ; xor %[err],%[err]\n" - "2:\n\t" - _ASM_EXTABLE_TYPE_REG(1b, 2b, EX_TYPE_WRMSR_SAFE, %[err]) - : [err] "=a" (err) - : "c" (msr), "0" ((u32)val), "d" ((u32)(val >> 32)) - : "memory"); if (tracepoint_enabled(write_msr)) do_trace_write_msr(msr, val, err); + return err; } +static __always_inline bool __wrmsr_variable(const u32 msr, const u64 val, const int type) +{ +#ifdef CONFIG_X86_64 + BUILD_BUG_ON(__builtin_constant_p(msr)); + + asm_inline volatile goto( + ALTERNATIVE(PREPARE_RDX_FOR_WRMSR, + "call asm_xen_write_msr\n\t" + "jnz 2f\n\t", + X86_FEATURE_XENPV) + ALTERNATIVE("1: ds wrmsr", + ASM_WRMSRNS, + X86_FEATURE_WRMSRNS) + "2:\n" + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For WRMSR(NS) */ + + : ASM_CALL_CONSTRAINT + : "a" (val), "c" (msr), [type] "i" (type) + : "memory", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +#else + return __native_wrmsr_variable(msr, val, type); +#endif +} + +static __always_inline bool __wrmsr_variable_all(const u32 msr, const u64 val, const int type) +{ + const enum pv_msr_action action = get_pv_msr_action(msr); + + if (action == PV_MSR_PV) { + return __wrmsr_variable(msr, val, type); + } else if (action == PV_MSR_IGNORE) { + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + } + + return __native_wrmsr_variable(msr, val, type); +} + +#ifdef CONFIG_X86_64 +static __always_inline bool __wrmsr_constant(const u32 msr, const u64 val, const int type) +{ + BUILD_BUG_ON(!__builtin_constant_p(msr)); + + asm_inline volatile goto( + "1:\n" + ALTERNATIVE_2(PREPARE_RCX_RDX_FOR_WRMSR, + "", + X86_FEATURE_MSR_IMM, + "mov %[msr], %%ecx\n\t" + "call asm_xen_write_msr\n\t" + "jnz 3f\n\t", + X86_FEATURE_XENPV) + ALTERNATIVE_2("2: ds wrmsr", + ASM_WRMSRNS, + X86_FEATURE_WRMSRNS, + ASM_WRMSRNS_IMM, + X86_FEATURE_MSR_IMM) + "3:\n" + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For WRMSRNS immediate */ + _ASM_EXTABLE_TYPE(2b, %l[badmsr], %c[type]) /* For WRMSR(NS) */ + + : ASM_CALL_CONSTRAINT + : [val] "a" (val), [msr] "i" (msr), [type] "i" (type) + : "memory", "ecx", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +} + +static __always_inline bool __wrmsr_constant_all(const u32 msr, const u64 val, const int type) +{ + const enum pv_msr_action action = get_pv_msr_action(msr); + + if (action == PV_MSR_PV) { + return __wrmsr_constant(msr, val, type); + } else if (action == PV_MSR_IGNORE) { + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + } + + return __native_wrmsr_constant(msr, val, type); +} +#endif + +static __always_inline bool __wrmsr(const u32 msr, const u64 val, const int type) +{ +#ifdef CONFIG_X86_64 + if (__builtin_constant_p(msr)) + return __wrmsr_constant_all(msr, val, type); +#endif + + return __wrmsr_variable_all(msr, val, type); +} + +static __always_inline void wrmsr(const u32 msr, const u32 low, const u32 high) +{ + __wrmsr(msr, (u64)high << 32 | low, EX_TYPE_WRMSR); +} + +static __always_inline void wrmsrl(const u32 msr, const u64 val) +{ + __wrmsr(msr, val, EX_TYPE_WRMSR); +} + +static __always_inline int wrmsr_safe(const u32 msr, const u32 low, const u32 high) +{ + return __wrmsr(msr, (u64)high << 32 | low, EX_TYPE_WRMSR_SAFE) ? -EIO : 0; +} + +static __always_inline int wrmsrl_safe(const u32 msr, const u64 val) +{ + return __wrmsr(msr, val, EX_TYPE_WRMSR_SAFE) ? -EIO : 0; +} + extern int rdmsr_safe_regs(u32 regs[8]); extern int wrmsr_safe_regs(u32 regs[8]); @@ -287,29 +548,6 @@ do { \ #endif /* !CONFIG_PARAVIRT_XXL */ -/* Instruction opcode for WRMSRNS supported in binutils >= 2.40 */ -#define WRMSRNS _ASM_BYTES(0x0f,0x01,0xc6) - -/* Non-serializing WRMSR, when available. Falls back to a serializing WRMSR. */ -static __always_inline void wrmsrns(u32 msr, u64 val) -{ - /* - * WRMSR is 2 bytes. WRMSRNS is 3 bytes. Pad WRMSR with a redundant - * DS prefix to avoid a trailing NOP. - */ - asm volatile("1: " ALTERNATIVE("ds wrmsr", WRMSRNS, X86_FEATURE_WRMSRNS) - "2: " _ASM_EXTABLE_TYPE(1b, 2b, EX_TYPE_WRMSR) - : : "c" (msr), "a" ((u32)val), "d" ((u32)(val >> 32))); -} - -/* - * 64-bit version of wrmsr_safe(): - */ -static inline int wrmsrl_safe(u32 msr, u64 val) -{ - return native_write_msr_safe(msr, val); -} - struct msr __percpu *msrs_alloc(void); void msrs_free(struct msr __percpu *msrs); int msr_set_bit(u32 msr, u8 bit); diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index f3d6e8394d38..351feb890ab0 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -180,21 +180,11 @@ static inline u64 paravirt_read_msr(unsigned msr) return PVOP_CALL1(u64, cpu.read_msr, msr); } -static inline void paravirt_write_msr(u32 msr, u32 low, u32 high) -{ - PVOP_VCALL2(cpu.write_msr, msr, (u64)high << 32 | low); -} - static inline u64 paravirt_read_msr_safe(unsigned msr, int *err) { return PVOP_CALL2(u64, cpu.read_msr_safe, msr, err); } -static inline int paravirt_write_msr_safe(u32 msr, u32 low, u32 high) -{ - return PVOP_CALL2(int, cpu.write_msr_safe, msr, (u64)high << 32 | low); -} - #define rdmsr(msr, val1, val2) \ do { \ u64 _l = paravirt_read_msr(msr); \ @@ -202,23 +192,11 @@ do { \ val2 = _l >> 32; \ } while (0) -#define wrmsr(msr, val1, val2) \ -do { \ - paravirt_write_msr(msr, val1, val2); \ -} while (0) - #define rdmsrl(msr, val) \ do { \ val = paravirt_read_msr(msr); \ } while (0) -static inline void wrmsrl(unsigned msr, u64 val) -{ - wrmsr(msr, (u32)val, (u32)(val>>32)); -} - -#define wrmsr_safe(msr, a, b) paravirt_write_msr_safe(msr, a, b) - /* rdmsr with exception handling */ #define rdmsr_safe(msr, a, b) \ ({ \ diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 78777b78da12..8a563576d70e 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -92,14 +92,12 @@ struct pv_cpu_ops { /* Unsafe MSR operations. These will warn or panic on failure. */ u64 (*read_msr)(unsigned int msr); - void (*write_msr)(u32 msr, u64 val); /* * Safe MSR operations. * read sets err to 0 or -EIO. write returns 0 or -EIO. */ u64 (*read_msr_safe)(unsigned int msr, int *err); - int (*write_msr_safe)(u32 msr, u64 val); u64 (*read_pmc)(int counter); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 1ccd05d8999f..ffb04445f97e 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -129,9 +129,7 @@ struct paravirt_patch_template pv_ops = { .cpu.write_cr0 = native_write_cr0, .cpu.write_cr4 = native_write_cr4, .cpu.read_msr = native_read_msr, - .cpu.write_msr = native_write_msr, .cpu.read_msr_safe = native_read_msr_safe, - .cpu.write_msr_safe = native_write_msr_safe, .cpu.read_pmc = native_read_pmc, .cpu.load_tr_desc = native_load_tr_desc, .cpu.set_ldt = native_set_ldt, diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index a047dadf4511..d02f55bfa869 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1112,43 +1112,33 @@ static u64 xen_do_read_msr(unsigned int msr, int *err) return val; } -static void set_seg(unsigned int which, unsigned int low, unsigned int high, - int *err) +static void set_seg(u32 which, u64 base) { - u64 base = ((u64)high << 32) | low; - - if (HYPERVISOR_set_segment_base(which, base) == 0) - return; - - if (err) - *err = -EIO; - else + if (HYPERVISOR_set_segment_base(which, base)) WARN(1, "Xen set_segment_base(%u, %llx) failed\n", which, base); } /* - * Support write_msr_safe() and write_msr() semantics. - * With err == NULL write_msr() semantics are selected. - * Supplying an err pointer requires err to be pre-initialized with 0. + * Return true to indicate the requested MSR write has been done successfully, + * otherwise return false to have the calling MSR write primitives in msr.h to + * fail. */ -static void xen_do_write_msr(unsigned int msr, unsigned int low, - unsigned int high, int *err) +bool xen_do_write_msr(u32 msr, u64 val) { - u64 val; bool emulated; switch (msr) { case MSR_FS_BASE: - set_seg(SEGBASE_FS, low, high, err); - break; + set_seg(SEGBASE_FS, val); + return true; case MSR_KERNEL_GS_BASE: - set_seg(SEGBASE_GS_USER, low, high, err); - break; + set_seg(SEGBASE_GS_USER, val); + return true; case MSR_GS_BASE: - set_seg(SEGBASE_GS_KERNEL, low, high, err); - break; + set_seg(SEGBASE_GS_KERNEL, val); + return true; case MSR_STAR: case MSR_CSTAR: @@ -1160,18 +1150,13 @@ static void xen_do_write_msr(unsigned int msr, unsigned int low, /* Fast syscall setup is all done in hypercalls, so these are all ignored. Stub them out here to stop Xen console noise. */ - break; + return true; default: - val = (u64)high << 32 | low; - if (pmu_msr_chk_emulated(msr, &val, false, &emulated) && emulated) - return; + return true; - if (err) - *err = native_write_msr_safe(msr, val); - else - native_write_msr(msr, val); + return false; } } @@ -1180,15 +1165,6 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err) return xen_do_read_msr(msr, err); } -static int xen_write_msr_safe(u32 msr, u64 val) -{ - int err = 0; - - xen_do_write_msr(msr, val, (u32)(val >> 32), &err); - - return err; -} - static u64 xen_read_msr(unsigned int msr) { int err; @@ -1196,13 +1172,6 @@ static u64 xen_read_msr(unsigned int msr) return xen_do_read_msr(msr, xen_msr_safe ? &err : NULL); } -static void xen_write_msr(u32 msr, u64 val) -{ - int err; - - xen_do_write_msr(msr, val, (u32)(val >> 32), xen_msr_safe ? &err : NULL); -} - /* This is called once we have the cpu_possible_mask */ void __init xen_setup_vcpu_info_placement(void) { @@ -1238,10 +1207,8 @@ static const typeof(pv_ops) xen_cpu_ops __initconst = { .write_cr4 = xen_write_cr4, .read_msr = xen_read_msr, - .write_msr = xen_write_msr, .read_msr_safe = xen_read_msr_safe, - .write_msr_safe = xen_write_msr_safe, .read_pmc = xen_read_pmc, diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S index 109af12f7647..e672632b1cc0 100644 --- a/arch/x86/xen/xen-asm.S +++ b/arch/x86/xen/xen-asm.S @@ -344,3 +344,58 @@ SYM_CODE_END(xen_entry_SYSENTER_compat) SYM_CODE_END(xen_entry_SYSCALL_compat) #endif /* CONFIG_IA32_EMULATION */ + +.macro XEN_SAVE_CALLEE_REGS_FOR_MSR + push %rcx + push %rdi + push %rsi + push %r8 + push %r9 + push %r10 + push %r11 +.endm + +.macro XEN_RESTORE_CALLEE_REGS_FOR_MSR + pop %r11 + pop %r10 + pop %r9 + pop %r8 + pop %rsi + pop %rdi + pop %rcx +.endm + +/* + * MSR number in %ecx, MSR value in %rax. + * + * %edx is set up to match %rax >> 32 like the native stub + * is expected to do + * + * Let xen_do_write_msr() return 'false' if the MSR access should + * be executed natively, IOW, 'true' means it has done the job. + * + * bool xen_do_write_msr(u32 msr, u64 value) + * + * If ZF=1 then this will fall down to the actual native WRMSR[NS] + * instruction. + * + * This also removes the need for Xen to maintain different safe and + * unsafe MSR routines, as the difference is handled by the same + * trap handler as is used natively. + */ + SYM_FUNC_START(asm_xen_write_msr) + ENDBR + FRAME_BEGIN + push %rax /* Save in case of native fallback */ + XEN_SAVE_CALLEE_REGS_FOR_MSR + mov %ecx, %edi /* MSR number */ + mov %rax, %rsi /* MSR data */ + call xen_do_write_msr + test %al, %al /* %al=1, i.e., ZF=0, means successfully done */ + XEN_RESTORE_CALLEE_REGS_FOR_MSR + mov 4(%rsp), %edx /* Set up %edx for native execution */ + pop %rax + FRAME_END + RET +SYM_FUNC_END(asm_xen_write_msr) +EXPORT_SYMBOL_GPL(asm_xen_write_msr) diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index 6545661010ce..fc3c55871037 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -146,6 +146,8 @@ __visible unsigned long xen_read_cr2_direct(void); /* These are not functions, and cannot be called normally */ __visible void xen_iret(void); +extern bool xen_do_write_msr(u32 msr, u64 val); + extern int xen_panic_handler_init(void); int xen_cpuhp_setup(int (*cpu_up_prepare_cb)(unsigned int), From patchwork Mon Mar 31 08:22:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877414 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA25E1DD889; Mon, 31 Mar 2025 08:24:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409456; cv=none; b=ZlbanS14LpovZnFu+skJ8b9iaERJ7dQ5c+zOqTNCB2HptwBGWqSEW0f8aceFuYnsisAv/dlTbBhYz2A+aeO9Hix7Bdf5ilDb4AmuYdx+q8XSILgoZAwWWLPzE9CLGPc8ztgEcdN9D+gyXrRknFgLyz5yYhQHVR4K3CJ+cAes/Do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409456; c=relaxed/simple; bh=qU9mZV/Ilpao3OzuBn2p54VPGVrfDdtWqmvb8dbK1/U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UohaNP3efj+4OsRBuJjlrGiBqfIaitP/UM73hFgvC50aajU5wIKxA55dHRvC6WfUmUqp3pfGWGO4jqpW6RnwAK9W3XJJ00ql97kZvusJJ6Flm3JR/21GWx5Ubm5eUqbLpdjfWzW15Rup7CGQ56vtXWTPq2iHEV1J4t6sUG10U68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=YZ0iWw1g; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="YZ0iWw1g" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0D3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:24 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0D3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409406; bh=MTNAHE0XoF/3VfQI20ic/94siCvLM4yHxZJbzupiGV4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YZ0iWw1g3Cln5AvXjilDQAKtaaSHfwJ+OM5wjPktUJyEqApfbUfhM09OIuU8QP2ZZ mYIqHTSj9WbWaRawl1ndfEI99/JZFReL4cHv2Bb1eaQsckThi+Yd7S/58DdR7MADp7 brFPhnzRo2b4ImK45TMad4AgaT1rzmLAJjkaQzPMD0yRdou+z3xLfMDvndeKh03uRp /+zHv3a4TGJZ32AoXQQXNfLWD3AqlBnWwlZqVKdNR8IVx0jW2LdzRYxxK+IYI7jo1X 8Jzz5y7bJekuTfQ2vMotkE1uZCAR+Ikj1SSqu1wAmeaafKH5SDC3fXFrxE045okmoi kvDiyWI69VGjg== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 13/15] x86/msr: Use the alternatives mechanism to read MSR Date: Mon, 31 Mar 2025 01:22:49 -0700 Message-ID: <20250331082251.3171276-14-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Also add support for the immediate form MSR read support. Suggested-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/msr.h | 274 ++++++++++++++++++++------ arch/x86/include/asm/paravirt.h | 40 ---- arch/x86/include/asm/paravirt_types.h | 9 - arch/x86/kernel/paravirt.c | 2 - arch/x86/xen/enlighten_pv.c | 45 ++--- arch/x86/xen/xen-asm.S | 34 ++++ arch/x86/xen/xen-ops.h | 7 + 7 files changed, 276 insertions(+), 135 deletions(-) diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index 066cde11254a..fc93c2601853 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -74,6 +74,8 @@ static inline void do_trace_rdpmc(unsigned int msr, u64 val, int failed) {} #endif #ifdef CONFIG_CC_IS_GCC +#define ASM_RDMSR_IMM \ + " .insn VEX.128.F2.M7.W0 0xf6 /0, %[msr]%{:u32}, %[val]\n\t" #define ASM_WRMSRNS_IMM \ " .insn VEX.128.F3.M7.W0 0xf6 /0, %[val], %[msr]%{:u32}\n\t" #endif @@ -85,10 +87,17 @@ static inline void do_trace_rdpmc(unsigned int msr, u64 val, int failed) {} * The register operand is encoded as %rax because all uses of the immediate * form MSR access instructions reference %rax as the register operand. */ +#define ASM_RDMSR_IMM \ + " .byte 0xc4,0xe7,0x7b,0xf6,0xc0; .long %c[msr]" #define ASM_WRMSRNS_IMM \ " .byte 0xc4,0xe7,0x7a,0xf6,0xc0; .long %c[msr]" #endif +#define RDMSR_AND_SAVE_RESULT \ + "rdmsr\n\t" \ + "shl $0x20, %%rdx\n\t" \ + "or %%rdx, %[val]\n\t" + #define PREPARE_RDX_FOR_WRMSR \ "mov %%rax, %%rdx\n\t" \ "shr $0x20, %%rdx\n\t" @@ -142,6 +151,7 @@ static __always_inline enum pv_msr_action get_pv_msr_action(const u32 msr) } } +extern void asm_xen_read_msr(void); extern void asm_xen_write_msr(void); #else static __always_inline enum pv_msr_action get_pv_msr_action(const u32 msr) @@ -150,35 +160,95 @@ static __always_inline enum pv_msr_action get_pv_msr_action(const u32 msr) } #endif -static __always_inline unsigned long long __rdmsr(unsigned int msr) +static __always_inline bool __native_rdmsr_variable(const u32 msr, u64 *val, const int type) { - DECLARE_ARGS(val, low, high); +#ifdef CONFIG_X86_64 + BUILD_BUG_ON(__builtin_constant_p(msr)); - asm volatile("1: rdmsr\n" - "2:\n" - _ASM_EXTABLE_TYPE(1b, 2b, EX_TYPE_RDMSR) - : EAX_EDX_RET(val, low, high) : "c" (msr)); + asm_inline volatile goto( + "1:\n" + RDMSR_AND_SAVE_RESULT + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For RDMSR */ - return EAX_EDX_VAL(val, low, high); + : [val] "=a" (*val) + : "c" (msr), [type] "i" (type) + : "memory", "rdx" + : badmsr); +#else + asm_inline volatile goto( + "1: rdmsr\n\t" + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For RDMSR */ + + : "=A" (*val) + : "c" (msr), [type] "i" (type) + : "memory" + : badmsr); +#endif + + return false; + +badmsr: + return true; +} + +#ifdef CONFIG_X86_64 +static __always_inline bool __native_rdmsr_constant(const u32 msr, u64 *val, const int type) +{ + BUILD_BUG_ON(!__builtin_constant_p(msr)); + + asm_inline volatile goto( + "1:\n" + ALTERNATIVE("mov %[msr], %%ecx\n\t" + "2:\n" + RDMSR_AND_SAVE_RESULT, + ASM_RDMSR_IMM, + X86_FEATURE_MSR_IMM) + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For RDMSR immediate */ + _ASM_EXTABLE_TYPE(2b, %l[badmsr], %c[type]) /* For RDMSR */ + + : [val] "=a" (*val) + : [msr] "i" (msr), [type] "i" (type) + : "memory", "ecx", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +} +#endif + +static __always_inline bool __native_rdmsr(const u32 msr, u64 *val, const int type) +{ +#ifdef CONFIG_X86_64 + if (__builtin_constant_p(msr)) + return __native_rdmsr_constant(msr, val, type); +#endif + + return __native_rdmsr_variable(msr, val, type); } -#define native_rdmsr(msr, val1, val2) \ +#define native_rdmsr(msr, low, high) \ do { \ - u64 __val = __rdmsr((msr)); \ - (void)((val1) = (u32)__val); \ - (void)((val2) = (u32)(__val >> 32)); \ + u64 __val = 0; \ + __native_rdmsr((msr), &__val, EX_TYPE_RDMSR); \ + (void)((low) = (u32)__val); \ + (void)((high) = (u32)(__val >> 32)); \ } while (0) static __always_inline u64 native_rdmsrl(const u32 msr) { - return __rdmsr(msr); + u64 val = 0; + + __native_rdmsr(msr, &val, EX_TYPE_RDMSR); + return val; } -static inline unsigned long long native_read_msr(unsigned int msr) +static inline u64 native_read_msr(const u32 msr) { - unsigned long long val; + u64 val = 0; - val = __rdmsr(msr); + __native_rdmsr(msr, &val, EX_TYPE_RDMSR); if (tracepoint_enabled(read_msr)) do_trace_read_msr(msr, val, 0); @@ -186,19 +256,139 @@ static inline unsigned long long native_read_msr(unsigned int msr) return val; } -static inline unsigned long long native_read_msr_safe(unsigned int msr, - int *err) +static inline u64 native_read_msr_safe(const u32 msr, int *err) { - DECLARE_ARGS(val, low, high); + u64 val = 0; + + *err = __native_rdmsr(msr, &val, EX_TYPE_RDMSR_SAFE) ? -EIO : 0; - asm volatile("1: rdmsr ; xor %[err],%[err]\n" - "2:\n\t" - _ASM_EXTABLE_TYPE_REG(1b, 2b, EX_TYPE_RDMSR_SAFE, %[err]) - : [err] "=r" (*err), EAX_EDX_RET(val, low, high) - : "c" (msr)); if (tracepoint_enabled(read_msr)) - do_trace_read_msr(msr, EAX_EDX_VAL(val, low, high), *err); - return EAX_EDX_VAL(val, low, high); + do_trace_read_msr(msr, val, *err); + + return val; +} + +static __always_inline bool __rdmsr_variable(const u32 msr, u64 *val, const int type) +{ +#ifdef CONFIG_X86_64 + BUILD_BUG_ON(__builtin_constant_p(msr)); + + asm_inline volatile goto( + "1:\n" + ALTERNATIVE(RDMSR_AND_SAVE_RESULT, + "call asm_xen_read_msr\n\t", + X86_FEATURE_XENPV) + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For RDMSR and CALL */ + + : [val] "=a" (*val), ASM_CALL_CONSTRAINT + : "c" (msr), [type] "i" (type) + : "memory", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +#else + return __native_rdmsr_variable(msr, val, type); +#endif +} + +static __always_inline bool __rdmsr_variable_all(const u32 msr, u64 *val, const int type) +{ + const enum pv_msr_action action = get_pv_msr_action(msr); + + if (action == PV_MSR_PV) { + return __rdmsr_variable(msr, val, type); + } else if (action == PV_MSR_IGNORE) { + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + } + + return __native_rdmsr_variable(msr, val, type); +} + +#ifdef CONFIG_X86_64 +static __always_inline bool __rdmsr_constant(const u32 msr, u64 *val, const int type) +{ + BUILD_BUG_ON(!__builtin_constant_p(msr)); + + asm_inline volatile goto( + "1:\n" + ALTERNATIVE_2("mov %[msr], %%ecx\n\t" + "2:\n" + RDMSR_AND_SAVE_RESULT, + ASM_RDMSR_IMM, + X86_FEATURE_MSR_IMM, + "mov %[msr], %%ecx\n\t" + "call asm_xen_read_msr\n\t", + X86_FEATURE_XENPV) + _ASM_EXTABLE_TYPE(1b, %l[badmsr], %c[type]) /* For RDMSR immediate */ + _ASM_EXTABLE_TYPE(2b, %l[badmsr], %c[type]) /* For RDMSR and CALL */ + + : [val] "=a" (*val), ASM_CALL_CONSTRAINT + : [msr] "i" (msr), [type] "i" (type) + : "memory", "ecx", "rdx" + : badmsr); + + return false; + +badmsr: + return true; +} + +static __always_inline bool __rdmsr_constant_all(const u32 msr, u64 *val, const int type) +{ + const enum pv_msr_action action = get_pv_msr_action(msr); + + if (action == PV_MSR_PV) { + return __rdmsr_constant(msr, val, type); + } else if (action == PV_MSR_IGNORE) { + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + } + + return __native_rdmsr_constant(msr, val, type); +} +#endif + +static __always_inline bool __rdmsr(const u32 msr, u64 *val, const int type) +{ +#ifdef CONFIG_X86_64 + if (__builtin_constant_p(msr)) + return __rdmsr_constant_all(msr, val, type); +#endif + + return __rdmsr_variable_all(msr, val, type); +} + +#define rdmsr(msr, low, high) \ +do { \ + u64 __val = 0; \ + __rdmsr((msr), &__val, EX_TYPE_RDMSR); \ + (void)((low) = (u32)__val); \ + (void)((high) = (u32)(__val >> 32)); \ +} while (0) + +#define rdmsrl(msr, val) \ +do { \ + u64 __val = 0; \ + __rdmsr((msr), &__val, EX_TYPE_RDMSR); \ + (val) = __val; \ +} while (0) + +#define rdmsr_safe(msr, low, high) \ +({ \ + u64 __val = 0; \ + int __err = __rdmsr((msr), &__val, EX_TYPE_RDMSR_SAFE) ? -EIO : 0; \ + (*low) = (u32)__val; \ + (*high) = (u32)(__val >> 32); \ + __err; \ +}) + +static __always_inline int rdmsrl_safe(const u32 msr, u64 *val) +{ + return __rdmsr(msr, val, EX_TYPE_RDMSR_SAFE) ? -EIO : 0; } /* @@ -503,40 +693,6 @@ static inline unsigned long long native_read_pmc(int counter) * Note: the rd* operations modify the parameters directly (without using * pointer indirection), this allows gcc to optimize better */ - -#define rdmsr(msr, low, high) \ -do { \ - u64 __val = native_read_msr((msr)); \ - (void)((low) = (u32)__val); \ - (void)((high) = (u32)(__val >> 32)); \ -} while (0) - -#define rdmsrl(msr, val) \ - ((val) = native_read_msr((msr))) - -static inline void wrmsrl(u32 msr, u64 val) -{ - native_write_msr(msr, val); -} - -/* rdmsr with exception handling */ -#define rdmsr_safe(msr, low, high) \ -({ \ - int __err; \ - u64 __val = native_read_msr_safe((msr), &__err); \ - (*low) = (u32)__val; \ - (*high) = (u32)(__val >> 32); \ - __err; \ -}) - -static inline int rdmsrl_safe(unsigned int msr, unsigned long long *p) -{ - int err; - - *p = native_read_msr_safe(msr, &err); - return err; -} - #define rdpmc(counter, low, high) \ do { \ u64 _l = native_read_pmc((counter)); \ diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 351feb890ab0..7ddb71ed9d0c 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -175,46 +175,6 @@ static inline void __write_cr4(unsigned long x) PVOP_VCALL1(cpu.write_cr4, x); } -static inline u64 paravirt_read_msr(unsigned msr) -{ - return PVOP_CALL1(u64, cpu.read_msr, msr); -} - -static inline u64 paravirt_read_msr_safe(unsigned msr, int *err) -{ - return PVOP_CALL2(u64, cpu.read_msr_safe, msr, err); -} - -#define rdmsr(msr, val1, val2) \ -do { \ - u64 _l = paravirt_read_msr(msr); \ - val1 = (u32)_l; \ - val2 = _l >> 32; \ -} while (0) - -#define rdmsrl(msr, val) \ -do { \ - val = paravirt_read_msr(msr); \ -} while (0) - -/* rdmsr with exception handling */ -#define rdmsr_safe(msr, a, b) \ -({ \ - int _err; \ - u64 _l = paravirt_read_msr_safe(msr, &_err); \ - (*a) = (u32)_l; \ - (*b) = _l >> 32; \ - _err; \ -}) - -static inline int rdmsrl_safe(unsigned msr, unsigned long long *p) -{ - int err; - - *p = paravirt_read_msr_safe(msr, &err); - return err; -} - static inline unsigned long long paravirt_read_pmc(int counter) { return PVOP_CALL1(u64, cpu.read_pmc, counter); diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 8a563576d70e..5c57e6e4115f 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -90,15 +90,6 @@ struct pv_cpu_ops { void (*cpuid)(unsigned int *eax, unsigned int *ebx, unsigned int *ecx, unsigned int *edx); - /* Unsafe MSR operations. These will warn or panic on failure. */ - u64 (*read_msr)(unsigned int msr); - - /* - * Safe MSR operations. - * read sets err to 0 or -EIO. write returns 0 or -EIO. - */ - u64 (*read_msr_safe)(unsigned int msr, int *err); - u64 (*read_pmc)(int counter); void (*start_context_switch)(struct task_struct *prev); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index ffb04445f97e..e3d4f9869779 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -128,8 +128,6 @@ struct paravirt_patch_template pv_ops = { .cpu.read_cr0 = native_read_cr0, .cpu.write_cr0 = native_write_cr0, .cpu.write_cr4 = native_write_cr4, - .cpu.read_msr = native_read_msr, - .cpu.read_msr_safe = native_read_msr_safe, .cpu.read_pmc = native_read_pmc, .cpu.load_tr_desc = native_load_tr_desc, .cpu.set_ldt = native_set_ldt, diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index d02f55bfa869..e49f16278487 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1087,19 +1087,26 @@ static void xen_write_cr4(unsigned long cr4) native_write_cr4(cr4); } -static u64 xen_do_read_msr(unsigned int msr, int *err) +/* + * Return true in xen_rdmsr_ret_type to indicate the requested MSR read has + * been done successfully. + */ +struct xen_rdmsr_ret_type xen_do_read_msr(u32 msr) { - u64 val = 0; /* Avoid uninitialized value for safe variant. */ - bool emulated; + struct xen_rdmsr_ret_type ret; - if (pmu_msr_chk_emulated(msr, &val, true, &emulated) && emulated) - return val; + ret.done = true; - if (err) - val = native_read_msr_safe(msr, err); - else - val = native_read_msr(msr); + if (pmu_msr_chk_emulated(msr, &ret.val, true, &ret.done) && ret.done) + return ret; + + ret.val = 0; + ret.done = false; + return ret; +} +u64 xen_do_read_msr_fixup(u32 msr, u64 val) +{ switch (msr) { case MSR_IA32_APICBASE: val &= ~X2APIC_ENABLE; @@ -1108,7 +1115,11 @@ static u64 xen_do_read_msr(unsigned int msr, int *err) else val &= ~MSR_IA32_APICBASE_BSP; break; + + default: + break; } + return val; } @@ -1160,18 +1171,6 @@ bool xen_do_write_msr(u32 msr, u64 val) } } -static u64 xen_read_msr_safe(unsigned int msr, int *err) -{ - return xen_do_read_msr(msr, err); -} - -static u64 xen_read_msr(unsigned int msr) -{ - int err; - - return xen_do_read_msr(msr, xen_msr_safe ? &err : NULL); -} - /* This is called once we have the cpu_possible_mask */ void __init xen_setup_vcpu_info_placement(void) { @@ -1206,10 +1205,6 @@ static const typeof(pv_ops) xen_cpu_ops __initconst = { .write_cr4 = xen_write_cr4, - .read_msr = xen_read_msr, - - .read_msr_safe = xen_read_msr_safe, - .read_pmc = xen_read_pmc, .load_tr_desc = paravirt_nop, diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S index e672632b1cc0..6e7a9daa03d4 100644 --- a/arch/x86/xen/xen-asm.S +++ b/arch/x86/xen/xen-asm.S @@ -399,3 +399,37 @@ SYM_CODE_END(xen_entry_SYSCALL_compat) RET SYM_FUNC_END(asm_xen_write_msr) EXPORT_SYMBOL_GPL(asm_xen_write_msr) + +/* + * The prototype of the Xen C code: + * struct { u64 val, bool done } xen_do_read_msr(u32 msr) + */ +SYM_FUNC_START(asm_xen_read_msr) + ENDBR + FRAME_BEGIN + XEN_SAVE_CALLEE_REGS_FOR_MSR + mov %ecx, %edi /* MSR number */ + call xen_do_read_msr + test %dl, %dl /* %dl=1, i.e., ZF=0, meaning successfully done */ + XEN_RESTORE_CALLEE_REGS_FOR_MSR + jnz 2f + +1: rdmsr + _ASM_EXTABLE_FUNC_REWIND(1b, -5, FRAME_OFFSET / (BITS_PER_LONG / 8)) + shl $0x20, %rdx + or %rdx, %rax + /* + * The top of the stack points directly at the return address; + * back up by 5 bytes from the return address. + */ + + XEN_SAVE_CALLEE_REGS_FOR_MSR + mov %ecx, %edi + mov %rax, %rsi + call xen_do_read_msr_fixup + XEN_RESTORE_CALLEE_REGS_FOR_MSR + +2: FRAME_END + RET +SYM_FUNC_END(asm_xen_read_msr) +EXPORT_SYMBOL_GPL(asm_xen_read_msr) diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index fc3c55871037..46efeaa4bbd3 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -146,7 +146,14 @@ __visible unsigned long xen_read_cr2_direct(void); /* These are not functions, and cannot be called normally */ __visible void xen_iret(void); +struct xen_rdmsr_ret_type { + u64 val; + bool done; +}; + extern bool xen_do_write_msr(u32 msr, u64 val); +extern struct xen_rdmsr_ret_type xen_do_read_msr(u32 msr); +extern u64 xen_do_read_msr_fixup(u32 msr, u64 val); extern int xen_panic_handler_init(void); From patchwork Mon Mar 31 08:22:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877415 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39F9F1D88D7; Mon, 31 Mar 2025 08:24:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409454; cv=none; b=RpfdXxbc4bPh4Hcsm12patcrihjZ4hfyT9nWIy83sdRDnPpkHBBvMFt3ZTopZbazJJv5ChW3kE7UGEQfJ2jBJzFDdQOjbcwoFlVOLnWMA1qpFPBBI8hatGj0X4Lkf7L6KNckDS1xGBOQXUz4jCdnRFJqLcKxhF0dpk/1+O49pbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409454; c=relaxed/simple; bh=gbh0m3B09/512sJ59MDydTRVOSmVcP06GMLUAWGZ7us=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hVV/xFBIgjkbBEuOQMPu5tN8CcrSx1LZPuT9Pf4Y5ZGjeMLglD/L2DVrhYCozktoEAKDtIJn4BQoedasytke4B4fpA+FMI/veDilGuuZGXjE9YXbMYEnyUGQ65yRYvnpAGpTQocOv3AIH6ZpZD58rH+ncAdfh5vRY5Syparxses= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=D7JCbMeX; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="D7JCbMeX" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0E3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:26 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0E3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409408; bh=tIAU2yy9AOedAvkyMYNrk111yRXguMSruNF4FMMzeHg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D7JCbMeXqweNh4pJPPw3+DgPKkMbuFPHUf0gRRzqUD2N/ecx68NbyfGuhSjPaQoWY mPmtnI1wTz8QnIJupt2R52o2elidYMFxQqTBQit0V10L9BDFmClZT/ppYUt5in2qs4 e9j2ZDUURdepMNCTtW+l4EcDMhlQH5vFCF5V7jlR9s/o3tpqLx+Hqmm4jgFgtPgYf1 eoMDOqWoJNnyPqToiT5yyUynx2AtKyvNZcR+n2ZUNik3Ln6pveOKPIJCouqGM6x74X 1pVyDGySW5ixXeseTIbyEzgsxuHGPcgoTg9cznUx900NH2wgPPUTESijAYougzG4f5 SiD+uPwm/kYOQ== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 14/15] x86/extable: Add support for the immediate form MSR instructions Date: Mon, 31 Mar 2025 01:22:50 -0700 Message-ID: <20250331082251.3171276-15-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Xin Li (Intel) --- arch/x86/mm/extable.c | 59 ++++++++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index eb9331240a88..56138c0762b7 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -164,31 +164,54 @@ static bool ex_handler_uaccess(const struct exception_table_entry *fixup, return ex_handler_default(fixup, regs); } +#ifdef CONFIG_X86_64 +static const u8 msr_imm_insn_prefix[] = { 0xc4, 0xe7 }; +#endif + static bool ex_handler_msr(const struct exception_table_entry *fixup, - struct pt_regs *regs, bool wrmsr, bool safe, int reg) + struct pt_regs *regs, bool wrmsr, bool safe) { + /* + * To ensure consistency with the existing RDMSR and WRMSR(NS), the register + * operand of the immediate form MSR access instructions is ALWAYS encoded as + * RAX in for the MSR value to be written or read. + * + * Full decoder for the immediate form MSR access instructions looks overkill. + */ + bool is_imm_insn; + u32 msr; + u64 msr_val; + +#ifdef CONFIG_X86_64 + is_imm_insn = !memcmp((void *)regs->ip, msr_imm_insn_prefix, sizeof(msr_imm_insn_prefix)); +#else + is_imm_insn = false; +#endif + + if (is_imm_insn) { + u8 *insn = (u8 *)regs->ip; + + msr = insn[5] | (insn[6] << 8) | (insn[7] << 16) | (insn[8] << 24); + } else { + msr = (u32)regs->cx; + } + if (__ONCE_LITE_IF(!safe && wrmsr)) { - pr_warn("unchecked MSR access error: WRMSR to 0x%x (tried to write 0x%08x%08x) at rIP: 0x%lx (%pS)\n", - (unsigned int)regs->cx, (unsigned int)regs->dx, - (unsigned int)regs->ax, regs->ip, (void *)regs->ip); + msr_val = regs->ax; + if (!is_imm_insn) + msr_val |= (u64)regs->dx << 32; + + pr_warn("unchecked MSR access error: WRMSR to 0x%x (tried to write 0x%016llx) at rIP: 0x%lx (%pS)\n", + msr, msr_val, regs->ip, (void *)regs->ip); show_stack_regs(regs); } if (__ONCE_LITE_IF(!safe && !wrmsr)) { pr_warn("unchecked MSR access error: RDMSR from 0x%x at rIP: 0x%lx (%pS)\n", - (unsigned int)regs->cx, regs->ip, (void *)regs->ip); + msr, regs->ip, (void *)regs->ip); show_stack_regs(regs); } - if (!wrmsr) { - /* Pretend that the read succeeded and returned 0. */ - regs->ax = 0; - regs->dx = 0; - } - - if (safe) - *pt_regs_nr(regs, reg) = -EIO; - return ex_handler_default(fixup, regs); } @@ -366,13 +389,13 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, case EX_TYPE_BPF: return ex_handler_bpf(e, regs); case EX_TYPE_WRMSR: - return ex_handler_msr(e, regs, true, false, reg); + return ex_handler_msr(e, regs, true, false); case EX_TYPE_RDMSR: - return ex_handler_msr(e, regs, false, false, reg); + return ex_handler_msr(e, regs, false, false); case EX_TYPE_WRMSR_SAFE: - return ex_handler_msr(e, regs, true, true, reg); + return ex_handler_msr(e, regs, true, true); case EX_TYPE_RDMSR_SAFE: - return ex_handler_msr(e, regs, false, true, reg); + return ex_handler_msr(e, regs, false, true); case EX_TYPE_WRMSR_IN_MCE: ex_handler_msr_mce(regs, true); break; From patchwork Mon Mar 31 08:22:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xin Li X-Patchwork-Id: 877615 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 346F91E7C16; Mon, 31 Mar 2025 08:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; cv=none; b=kS3+kgb4e9rIl8E85ehpa213F4hDwOgcSWdlct77y3Qu9eIFnNbGcur+hyWu7m/QiW3zEWkzFjOFjSsd7XAUE9P3BkwGSYc7on/uDhSKU/QB1q0hKLxVj2Z3+m8jMLyw5s7ZHDw+cWivwh+YtH0nmfcdzIdgJnAccrwCCbLfmYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743409462; c=relaxed/simple; bh=ABSKvHyBIrh6QHK4hlgn4QnqvAzBsraSRqbIPjJ35nM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t0itzJLIqPitCu+aBdlGbHZizteGkkwk69AHy9FPCx8YgrKEhbpIvL0p5OB5ARcVAaFgGib6xfsZESSOkyCQiW014ioFrFY4pO/OgQKxGcyvUW2D+XoEnt9vyz31x7gfU0puM45WWmvPeQi7irdE5vg0TXro5TAdTZHzGGiWme4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=SuHDQIJy; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="SuHDQIJy" Received: from terminus.zytor.com (terminus.zytor.com [IPv6:2607:7c80:54:3:0:0:0:136]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 52V8Mp0F3171319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 31 Mar 2025 01:23:28 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 52V8Mp0F3171319 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2025032001; t=1743409410; bh=cMFNgaQ/yU5pBBnUjuHrZWgQ7p+gEOsEGfeLMhV/cHU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SuHDQIJy8AzSLC0DrXz1UGsHXHhDsnDQpCgFuieur9l29e7EM1ArYQl7rGjnX8wD+ WkraR40jpmWTpADYisOZ7nbeYq+ByLQ5TLLYH8kKM7DCg5buwGVA5GHhHZVrggSeZ4 p9nifPZWSuSDm1534yUVgXXqF/kvTz/WBhHohMexeK1jikiHydt9HM2bVqx8MyrHLR c8M6luTBzpzY8I0f63iDe6i8yCHSJXzCnLx3cOikN4ytSEhwdKsNU/oWb2NhlDUBla mlZbjuvfxWirBhjjqog8XtnbNx9xitJDGMf/rsrr0eINZ0q1eEmyIbBAiN8ijpnMcH lJXB3Tmz/tP0A== From: "Xin Li (Intel)" To: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux.dev, linux-edac@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-ide@vger.kernel.org, linux-pm@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, jgross@suse.com, andrew.cooper3@citrix.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, wei.liu@kernel.org, ajay.kaher@broadcom.com, alexey.amakhalov@broadcom.com, bcm-kernel-feedback-list@broadcom.com, tony.luck@intel.com, pbonzini@redhat.com, vkuznets@redhat.com, seanjc@google.com, luto@kernel.org, boris.ostrovsky@oracle.com, kys@microsoft.com, haiyangz@microsoft.com, decui@microsoft.com Subject: [RFC PATCH v1 15/15] x86/msr: Move the ARGS macros after the MSR read/write APIs Date: Mon, 31 Mar 2025 01:22:51 -0700 Message-ID: <20250331082251.3171276-16-xin@zytor.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250331082251.3171276-1-xin@zytor.com> References: <20250331082251.3171276-1-xin@zytor.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since the ARGS macros are no longer used in the MSR read/write API implementation, move them after their definitions. Signed-off-by: Xin Li (Intel) --- arch/x86/include/asm/msr.h | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/msr.h b/arch/x86/include/asm/msr.h index fc93c2601853..9b109d1d92aa 100644 --- a/arch/x86/include/asm/msr.h +++ b/arch/x86/include/asm/msr.h @@ -37,23 +37,6 @@ struct saved_msrs { struct saved_msr *array; }; -/* - * both i386 and x86_64 returns 64-bit value in edx:eax, but gcc's "A" - * constraint has different meanings. For i386, "A" means exactly - * edx:eax, while for x86_64 it doesn't mean rdx:rax or edx:eax. Instead, - * it means rax *or* rdx. - */ -#ifdef CONFIG_X86_64 -/* Using 64-bit values saves one instruction clearing the high half of low */ -#define DECLARE_ARGS(val, low, high) unsigned long low, high -#define EAX_EDX_VAL(val, low, high) ((low) | (high) << 32) -#define EAX_EDX_RET(val, low, high) "=a" (low), "=d" (high) -#else -#define DECLARE_ARGS(val, low, high) unsigned long long val -#define EAX_EDX_VAL(val, low, high) (val) -#define EAX_EDX_RET(val, low, high) "=A" (val) -#endif - /* * Be very careful with includes. This header is prone to include loops. */ @@ -620,6 +603,23 @@ static __always_inline int wrmsrl_safe(const u32 msr, const u64 val) extern int rdmsr_safe_regs(u32 regs[8]); extern int wrmsr_safe_regs(u32 regs[8]); +/* + * both i386 and x86_64 returns 64-bit value in edx:eax, but gcc's "A" + * constraint has different meanings. For i386, "A" means exactly + * edx:eax, while for x86_64 it doesn't mean rdx:rax or edx:eax. Instead, + * it means rax *or* rdx. + */ +#ifdef CONFIG_X86_64 +/* Using 64-bit values saves one instruction clearing the high half of low */ +#define DECLARE_ARGS(val, low, high) unsigned long low, high +#define EAX_EDX_VAL(val, low, high) ((low) | (high) << 32) +#define EAX_EDX_RET(val, low, high) "=a" (low), "=d" (high) +#else +#define DECLARE_ARGS(val, low, high) unsigned long long val +#define EAX_EDX_VAL(val, low, high) (val) +#define EAX_EDX_RET(val, low, high) "=A" (val) +#endif + /** * rdtsc() - returns the current TSC without ordering constraints *