From patchwork Wed Apr 30 00:16:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886837 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF4D42139D1 for ; Wed, 30 Apr 2025 00:16:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972191; cv=none; b=WvpLfOU9CeGXkE9nYYzQZXVS95JaQ8zM2r3lRrvKW+hX6wkpZGXf6jItNKDQ2Ed97HLn9DMuOdAxOUdH+wUexVCBmgOIOa5wTKHK/7hvuyK2+XJ/Sa924injfaUJ3u9NI+6747Dq0SKm6x8K9vqUdKfjvDkJlluFIdUUSlxVPRM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972191; c=relaxed/simple; bh=07JIHRwK+fbY/KS3WXdAQFowggDed4ZKoyBK3fLGucQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UkylzEMVW10vPniTaYIB2CZ4aPLKNkeuMiBgYV8GBklUAoZQoGYhIjWHSZ90Z39HxDHu5p3M3Ah/h5bXal4LVxb9JbQUnt1p93VrHERELb0Vo0gQw+aRdbiobh7InzJgZfxW1GP7ky5uobgMFMgUuo2AogpicvU+IcaoLzzkRts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=Nzv7/mko; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="Nzv7/mko" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2243803b776so111464815ad.0 for ; Tue, 29 Apr 2025 17:16:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972189; x=1746576989; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=SkVK3bviqHN/t8JmfUhzYvFjjtQu0YyYqPmZLVGI/TA=; b=Nzv7/mkobhS1qwl3caSpbMi+f0htEk0T/vf7Qe1W5wYEr1P/pip/onTpADOa77EWPL VC5yH3qrd1yerG80gpoTNpBqBzsH/YKyr9yHicinym+UF5AFlA6btYIFcdbe01ZGQix+ ur4iGwktyjesd2cnorq6wyIqROMMxpePJ2m2iRxx0h+IKFy7xzT6Q+pjxaqsS9kzfjl0 CVNF0FtZtkP9ELctLcok37XoHeUgnpXyzSOMd7xGf21NX5v/MrHpwAEIWQWTXaRrD+ln XdX8GKmTUrslyWATKwGCsPw/EJqmlqpDP1ID9Fp5dqsXEG6b30GfgNlTohfxIHvz7Ja/ yMRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972189; x=1746576989; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SkVK3bviqHN/t8JmfUhzYvFjjtQu0YyYqPmZLVGI/TA=; b=NQ17QBXw2I1vcPKBUbhHDqp9kw3P0dST+iOUuPKq2uVAkFS7ooTwqeIjTehd/SDv+7 oQ6XSIPEf6yQIu2EpMvMYaUYkwj30GUDg+BXY0cYHFvJelNHGCKhvkkGJUFK/pSLrjh/ gIjY4OnVkiNY6SX9HD5cgrsqeGAx37oQF1GbGNUhJoPqXOZsxCPS307EMGUXvZnoF4Xh MNKIvzqDE0PVDd7bySPhtp5vIEfEafPZtNuxC2Jz4g5XwWhdCU6b55cK4SBL7oWnSOi8 Y1M0SOf598quLXHlt+gUpep3hbo0UwcRWf1ITIRn0BucKOnSuFIOBW7lD+29uDGi1Ahx Rerg== X-Forwarded-Encrypted: i=1; AJvYcCViVyJAbtUQavzLH0oKNj133APwjmLDvApGyB+VCZeiDZcTiasIDu3Q+0dhFS1os/HG1QmoleGMEMZRcdDq8m8=@vger.kernel.org X-Gm-Message-State: AOJu0Yx6affCRkuVCFTTcrZRIQM/1WeC+Rzc9Mh+7qRNpdka/bCZLKuD 71FwikHI3mge+nBN4+vrvFRnBDj05Xo2D3S8jo/rtsYbnSPlQSOyzEfJO4z5xCE= X-Gm-Gg: ASbGncsmuMMBlXUY6op5mMIz87zJNE+kas6WB0FmVjOEGpVXFlOF8N00T9eStnJNQNO W5phlitJiV0IMDACMauiaUcLyP15Nkv1b9lA9HuEJFDYBKN2BLEuTES0WIHF+nUGi654nhsnTvD YLgZZ1lwjTpL6YNw292cnnvekeqN/JaKLyfHR3naUmT5O7xCjFzE1kuF5+avOob8NeY9izQ1Kdg RBvLBcUyHZSRRn/4hITTUAxylmwS7NE2EZzpSgdAq8+5RtF69ZCF496WGoXVEhybVT80sj0r2MV PnyDJH/rCufDJEG3F/j1JStoz3xhGmIzFYH5dQADnn4KWV95dxQ= X-Google-Smtp-Source: AGHT+IEaCOo3UdLjB/YDRvV5wqIWEkwy6tQXnBD9FT8T9fOLPTePyGUVcVc/ygroOS4hwPE4lcSyLA== X-Received: by 2002:a17:903:1983:b0:216:644f:bc0e with SMTP id d9443c01a7336-22df35192a3mr22839375ad.24.1745972189106; Tue, 29 Apr 2025 17:16:29 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:16:28 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:18 -0700 Subject: [PATCH v14 01/27] mm: VM_SHADOW_STACK definition for riscv Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-1-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 VM_HIGH_ARCH_5 is used for riscv Reviewed-by: Zong Li Reviewed-by: Alexandre Ghiti Signed-off-by: Deepak Gupta --- include/linux/mm.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index b7f13f087954..3487f28fa0bf 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -352,6 +352,13 @@ extern unsigned int kobjsize(const void *objp); # define VM_SHADOW_STACK VM_HIGH_ARCH_6 #endif +#if defined(CONFIG_RISCV_USER_CFI) +/* + * Following x86 and picking up the same bitpos. + */ +# define VM_SHADOW_STACK VM_HIGH_ARCH_5 +#endif + #ifndef VM_SHADOW_STACK # define VM_SHADOW_STACK VM_NONE #endif From patchwork Wed Apr 30 00:16:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886836 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12A3821C9E7 for ; Wed, 30 Apr 2025 00:16:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972198; cv=none; b=CRGmFtJj4IYgyfF1+cE2EJBEaeegJyF3oT2rh40mR6YiDMh/070YPbO1GMgA5drDY2JuAQA98MjBpCShfkO/JkbPU3sE1mdRincnrMHK+Enj54P6uhd9fcXQo5WzlP3KiSwI7Y6igyZs9aMcsiYf/V7lpw0sqRX2fVuD+jTc/ko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972198; c=relaxed/simple; bh=05039bUJ0bhZsU9ePl+ot7OaszQahvBHRlGY7m/JZd0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=B8f+8QiAYFqVYKfeyVkMZDkvOmGcBSH7/xC/xNTLF7ojm/SlWyB0U1ajpXOxvG1xmogk01LdbNpK6wNdMNBpoztOjkfoyY17WjxCETtYSx9wY5zOr/4y25mV8sUiO8gVKCe3hueJz3zJQ/zdn5XZi+MXcEsKoo/rT/F7qNpmLok= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=u7oAT4v4; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="u7oAT4v4" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-224019ad9edso100154845ad.1 for ; Tue, 29 Apr 2025 17:16:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972195; x=1746576995; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Kv/IZrFWN34T6LwOeKmoAZHYNOr2b+SAEf5j5mwxEcc=; b=u7oAT4v4q9V310c5meUGbAnUOLmZ3h9nKKrNiMomIEIVPe9sXJoizXT4B7jWVwD5n/ QemkJZsesQ3qHq/0hqaUgvzfTvb0O3Hptlnaa/VTiRKmk4ldHohtOi8wtSy97Z7SkM8M Mgk2bWmU2CeDsh3/F6l9CSXQHh/t+WIQ++X9S4z7DN4EleE2J42pGZa3LYKyKsZSiF51 RFeBROUaJpE4KA3Vzac58F65wUFUQj6nTZWArrXYEmVwda44txVdyf3Q75a5HRoFVLGQ z3fwlEDHsfo/5SuxECijqNUC6x9q7cfLGXqRWcWxPdqoG9sr6sQWHvrQKgGcJsEoXUb1 Q/Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972195; x=1746576995; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kv/IZrFWN34T6LwOeKmoAZHYNOr2b+SAEf5j5mwxEcc=; b=kwPN47oiZ9fZQpZIZqePUFESHfOy66LriOf19GZAdp5wfB4T3SRZ8TUZn8KXlyknHc HJ39+3JJPSBhtQd/ZFuGCXYchkqcb6C7SnLzUcRntyyQ4NRnMWRNJUsbYQ58cRm1qFf3 8zCkF/m/bDkS4dgBLsiq/Hi41zl48q95W3Y7fabvO4jgsvadjwMhcXHC3NkrIRaex/mt H3Ete1q/+9ffLGf5IIemPpcuNOuevKmT3C+GZj2W2gBh+3r+x0BrdOldGksr3WiKbsim 3ce92t9nOu2Zk2q4mTh/NqUe6TOTZIAzGpXajn9Vgu2+isttsR9dNhUOUBc88Zvm+wvk 3Dig== X-Forwarded-Encrypted: i=1; AJvYcCVyoe8K2mlxFY4VNIeHS2p+3rXmdNaejOx4LwfKkRmDD/WA54csE3+UXEYWhYdHiO815447etGTe6DgmaIgDEA=@vger.kernel.org X-Gm-Message-State: AOJu0YymAGaJpGt9Iif5kts14qrbGvCHNS+tTKWQISPNrrppSeN+K5Pt COch0nn/VtBni49Nd6fbw06B8TMsaHts8CQGYlavFM3abmcSZlvwm9mAuLuyJAs= X-Gm-Gg: ASbGncuXarA7dEkmQAyG+nq8s/dFh5P1KPSJyQPKDBMh6KSVMWFijEb8XvDRH8xFZCY cQs2ozRqgrbwdY7uoG1im0XT4oUqR7QW2WP9Nu2qU1ztPGVzeZcLjlOXRJt7nlVohBRfji0ufCx 3/wVyvSuE40ukmT22h1QH7cgUUPecai2we1gj5wXVuprGKWIXqDhMkMof6mcQjDpzqUSHVAtgji UZDr6elffIgzUHA4bqoBEFITDtkTjnCQ0+8/ceyA0EEBQTSRbA5pkDjLidtoDrOaHhBGDUEVSyM H3ieep7UleRmPBvjhxNDQooa3rwpSKAXNM4HFlg3/wLM9QuIQzfforg42IQPjg== X-Google-Smtp-Source: AGHT+IElTpm1m59BCiMYV66Qwl1bIP6MumvDjXe79sV2QfTXj47VxQa3ZP8qKREhDRx3opT6xCz9bg== X-Received: by 2002:a17:903:1b0e:b0:223:6744:bfb9 with SMTP id d9443c01a7336-22df359c764mr20042105ad.41.1745972195360; Tue, 29 Apr 2025 17:16:35 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:16:35 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:20 -0700 Subject: [PATCH v14 03/27] riscv: zicfiss / zicfilp enumeration Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-3-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 This patch adds support for detecting zicfiss and zicfilp. zicfiss and zicfilp stands for unprivleged integer spec extension for shadow stack and branch tracking on indirect branches, respectively. This patch looks for zicfiss and zicfilp in device tree and accordinlgy lights up bit in cpu feature bitmap. Furthermore this patch adds detection utility functions to return whether shadow stack or landing pads are supported by cpu. Reviewed-by: Zong Li Reviewed-by: Alexandre Ghiti Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/cpufeature.h | 12 ++++++++++++ arch/riscv/include/asm/hwcap.h | 2 ++ arch/riscv/include/asm/processor.h | 1 + arch/riscv/kernel/cpufeature.c | 13 +++++++++++++ 4 files changed, 28 insertions(+) diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 3a87f612035c..100f4b53ba5d 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -146,4 +146,16 @@ static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsi return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); } +static inline bool cpu_supports_shadow_stack(void) +{ + return (IS_ENABLED(CONFIG_RISCV_USER_CFI) && + riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICFISS)); +} + +static inline bool cpu_supports_indirect_br_lp_instr(void) +{ + return (IS_ENABLED(CONFIG_RISCV_USER_CFI) && + riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICFILP)); +} + #endif diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index e3cbf203cdde..abc33ca1302e 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -105,6 +105,8 @@ #define RISCV_ISA_EXT_ZVFBFWMA 96 #define RISCV_ISA_EXT_ZAAMO 97 #define RISCV_ISA_EXT_ZALRSC 98 +#define RISCV_ISA_EXT_ZICFILP 99 +#define RISCV_ISA_EXT_ZICFISS 100 #define RISCV_ISA_EXT_XLINUXENVCFG 127 diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 5f56eb9d114a..e3aba3336e63 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -13,6 +13,7 @@ #include #include +#include #define arch_get_mmap_end(addr, len, flags) \ ({ \ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 2054f6c4b0ae..98e55467557b 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -258,6 +258,15 @@ static int riscv_ext_svadu_validate(const struct riscv_isa_ext_data *data, return 0; } +static int riscv_cfi_validate(const struct riscv_isa_ext_data *data, + const unsigned long *isa_bitmap) +{ + if (!IS_ENABLED(CONFIG_RISCV_USER_CFI)) + return -EINVAL; + + return 0; +} + static const unsigned int riscv_a_exts[] = { RISCV_ISA_EXT_ZAAMO, RISCV_ISA_EXT_ZALRSC, @@ -444,6 +453,10 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicbom, RISCV_ISA_EXT_ZICBOM, riscv_xlinuxenvcfg_exts, riscv_ext_zicbom_validate), __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicboz, RISCV_ISA_EXT_ZICBOZ, riscv_xlinuxenvcfg_exts, riscv_ext_zicboz_validate), __RISCV_ISA_EXT_DATA(ziccrse, RISCV_ISA_EXT_ZICCRSE), + __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicfilp, RISCV_ISA_EXT_ZICFILP, riscv_xlinuxenvcfg_exts, + riscv_cfi_validate), + __RISCV_ISA_EXT_SUPERSET_VALIDATE(zicfiss, RISCV_ISA_EXT_ZICFISS, riscv_xlinuxenvcfg_exts, + riscv_cfi_validate), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), __RISCV_ISA_EXT_DATA(zicond, RISCV_ISA_EXT_ZICOND), __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR), From patchwork Wed Apr 30 00:16:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886835 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D2D7231828 for ; Wed, 30 Apr 2025 00:16:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972204; cv=none; b=Ly5GJTSnYHaJlHTbYx2emOpKwMCTFPxzo4reVxTdPl4ne7Xlf+yazRW9DMhPM9qGe9TArj4VPxQuYYhZj07X0BFC8gXKc8f5T6dYbcd+ij+zR9DbiG18NobmLqZtI/tDnEGgk6h7UFo8pPowYp/qporBA8zRHLjqKXTfUSb65go= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972204; c=relaxed/simple; bh=IVp1L32E8a9jVUlfHVcjufgvP6ZGHAk+tTWTIvB9zVo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Qt4pQp/XrnvegM60f1BoG0QiPWWDuk0nr4qWSoEf4e0OaEckfHLVhWN5jNvJgotC18JoAVNe3jqR13F7KsHOGk6l3/2pJUcSvVngSCDiQRMwYKVvJ/SgT0BUDGMqapD8FipnzW8mJViWm7VBwpg9uQJPaQh7OGrjnK6+AFquayY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=gieGdr4M; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="gieGdr4M" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2240b4de12bso96997795ad.2 for ; Tue, 29 Apr 2025 17:16:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972202; x=1746577002; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=eyCgE0sNHOlyx9zGD2XlqTc9W1j01DyA5vw9LyX8zMs=; b=gieGdr4MkSkc8qNxieYflSjPcHG/loLIwrlNDbzN8jzjLqCcCzmiKUY4Qg2t4fAkzh LXA7M+609NwSPvemoIdHDKlZ7jgtN2lgJafzYBE1tc4moaBZBPEbfFVugPEqMim2gnph Y5m9iC40j2NuqEV9tl/a51w0gAOZyr4dxOdi3eyaoLoZ0ZG6UMvGGofgUYUKmnUvGHyH ORKVSmQd2C7JEjs3Ei5z1JSbN2XSR1LJewuYh+FlLv3eazFkZGA8+pJkgLpZS7Y7gTKi myMf28BfjAQc5rYPItftqmK1xPzS85CKNss/UHWx8xpcMbVwifMHAiw7KyeeGRDixAwt Saig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972202; x=1746577002; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eyCgE0sNHOlyx9zGD2XlqTc9W1j01DyA5vw9LyX8zMs=; b=PeBAoc+JunV/G2rhTWdPYTzmBrlW1aS23IlOKNcjFInholoydfNA6h8wH5jncuukLF U1KDK5Y0tHZHYCAoaI6ZvSZLs62cDu8If4dIRmk9Cart9+NVQUTZYc42kYfRIXJBCFPC BCLUO9ZJXyk9rwOuiJFoaJVePzP6UmdS1iSCT/IpHSsDwqYV6Jo57FLrkYo5pCUKyeH0 RaPfhnzsWc4raa2ZjcMYD1x60L1xc+myOUxhAmEarD00u6og69iV+PVNlI3o+M7z5mHZ Yhhc8zcHfvU5hstRdfidoTpx07uuiqS/oLnjLVcpP3WqKArCIlQWjzaSGklwVpTeVTWh Ri+g== X-Forwarded-Encrypted: i=1; AJvYcCWoHH5j4RW1kHoCsHjlCtkbNw0tPK33sEVc7yF3cQ9woNt6mJlK/DOAMbtmVSWhw3C62GUE9J4bCvjJLhQLBCM=@vger.kernel.org X-Gm-Message-State: AOJu0Yy952Iq6hIdMAVTm38TgGroLAN3SuSEmjknV4IuTKwp+WrfwOXb B8QFESDSK4qQGLTQGSQkxXHhMiRBgv0BEDDb6mImPi0BFK0ZcNaR2Gv+n95v6Fk= X-Gm-Gg: ASbGncsPuZTgreYsoWYhxjF009uM+HfsebUpwQps+YVMG9Y/MuW844cgxNbBGTQHUa3 QuVnLAR789q9m/CYsI+tRHSgrI5nah6zyLB/quDMt6znkwL55LUmAzDeQIzYH7bbOtKCM7qsmCm 3bjhxBgmIY1snNpS7g9tVqiUiyB8IGM9aCX2FkSXnaMuzeL6mzkth7+ePIbXUag2zZSKQiq8pe8 KYhlDZ3UvI8V7K2YtvTqknzedTjEOrleiLL8IflBUP/kl82wN3UWYSoFdCVBzXejDB7Ay2b/fX2 1DNLCzg/LmQx6lcE97knkdnDEcOGpLR7dvNcq20/TBvs8t0EVso= X-Google-Smtp-Source: AGHT+IHe4GmqH3A4bkQzoQX0kbX9VQ40AhHjKMv6yVTwaVC4Z0lHoaKG8FLcVA+eipUEtZcj/tZgcw== X-Received: by 2002:a17:902:e788:b0:223:5379:5e4e with SMTP id d9443c01a7336-22df5764436mr8015695ad.10.1745972201688; Tue, 29 Apr 2025 17:16:41 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:16:41 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:22 -0700 Subject: [PATCH v14 05/27] riscv: usercfi state for task and save/restore of CSR_SSP on trap entry/exit Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-5-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 Carves out space in arch specific thread struct for cfi status and shadow stack in usermode on riscv. This patch does following - defines a new structure cfi_status with status bit for cfi feature - defines shadow stack pointer, base and size in cfi_status structure - defines offsets to new member fields in thread in asm-offsets.c - Saves and restore shadow stack pointer on trap entry (U --> S) and exit (S --> U) Shadow stack save/restore is gated on feature availiblity and implemented using alternative. CSR can be context switched in `switch_to` as well but soon as kernel shadow stack support gets rolled in, shadow stack pointer will need to be switched at trap entry/exit point (much like `sp`). It can be argued that kernel using shadow stack deployment scenario may not be as prevalant as user mode using this feature. But even if there is some minimal deployment of kernel shadow stack, that means that it needs to be supported. And thus save/restore of shadow stack pointer in entry.S instead of in `switch_to.h`. Reviewed-by: Charlie Jenkins Reviewed-by: Zong Li Reviewed-by: Alexandre Ghiti Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/processor.h | 1 + arch/riscv/include/asm/thread_info.h | 3 +++ arch/riscv/include/asm/usercfi.h | 23 +++++++++++++++++++++++ arch/riscv/kernel/asm-offsets.c | 4 ++++ arch/riscv/kernel/entry.S | 28 ++++++++++++++++++++++++++++ 5 files changed, 59 insertions(+) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index e3aba3336e63..d851bb5c6da0 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -14,6 +14,7 @@ #include #include +#include #define arch_get_mmap_end(addr, len, flags) \ ({ \ diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h index f5916a70879a..e066f41176ca 100644 --- a/arch/riscv/include/asm/thread_info.h +++ b/arch/riscv/include/asm/thread_info.h @@ -73,6 +73,9 @@ struct thread_info { */ unsigned long a0, a1, a2; #endif +#ifdef CONFIG_RISCV_USER_CFI + struct cfi_state user_cfi_state; +#endif }; #ifdef CONFIG_SHADOW_CALL_STACK diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h new file mode 100644 index 000000000000..94b214c295c0 --- /dev/null +++ b/arch/riscv/include/asm/usercfi.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 + * Copyright (C) 2024 Rivos, Inc. + * Deepak Gupta + */ +#ifndef _ASM_RISCV_USERCFI_H +#define _ASM_RISCV_USERCFI_H + +#ifndef __ASSEMBLY__ +#include + +#ifdef CONFIG_RISCV_USER_CFI +struct cfi_state { + unsigned long ubcfi_en : 1; /* Enable for backward cfi. */ + unsigned long user_shdw_stk; /* Current user shadow stack pointer */ + unsigned long shdw_stk_base; /* Base address of shadow stack */ + unsigned long shdw_stk_size; /* size of shadow stack */ +}; + +#endif /* CONFIG_RISCV_USER_CFI */ + +#endif /* __ASSEMBLY__ */ + +#endif /* _ASM_RISCV_USERCFI_H */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index 16490755304e..f33945432f8f 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -49,6 +49,10 @@ void asm_offsets(void) #endif OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu); +#ifdef CONFIG_RISCV_USER_CFI + OFFSET(TASK_TI_CFI_STATE, task_struct, thread_info.user_cfi_state); + OFFSET(TASK_TI_USER_SSP, task_struct, thread_info.user_cfi_state.user_shdw_stk); +#endif OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]); OFFSET(TASK_THREAD_F1, task_struct, thread.fstate.f[1]); OFFSET(TASK_THREAD_F2, task_struct, thread.fstate.f[2]); diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 33a5a9f2a0d4..c4bfe2085c41 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -91,6 +91,32 @@ REG_L a0, TASK_TI_A0(tp) .endm +/* + * If previous mode was U, capture shadow stack pointer and save it away + * Zero CSR_SSP at the same time for sanitization. + */ +.macro save_userssp tmp, status + ALTERNATIVE("nops(4)", + __stringify( \ + andi \tmp, \status, SR_SPP; \ + bnez \tmp, skip_ssp_save; \ + csrrw \tmp, CSR_SSP, x0; \ + REG_S \tmp, TASK_TI_USER_SSP(tp); \ + skip_ssp_save:), + 0, + RISCV_ISA_EXT_ZICFISS, + CONFIG_RISCV_USER_CFI) +.endm + +.macro restore_userssp tmp + ALTERNATIVE("nops(2)", + __stringify( \ + REG_L \tmp, TASK_TI_USER_SSP(tp); \ + csrw CSR_SSP, \tmp), + 0, + RISCV_ISA_EXT_ZICFISS, + CONFIG_RISCV_USER_CFI) +.endm SYM_CODE_START(handle_exception) /* @@ -147,6 +173,7 @@ SYM_CODE_START(handle_exception) REG_L s0, TASK_TI_USER_SP(tp) csrrc s1, CSR_STATUS, t0 + save_userssp s2, s1 csrr s2, CSR_EPC csrr s3, CSR_TVAL csrr s4, CSR_CAUSE @@ -236,6 +263,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception) * structures again. */ csrw CSR_SCRATCH, tp + restore_userssp s3 1: #ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE move a0, sp From patchwork Wed Apr 30 00:16:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886834 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBB5623C51D for ; Wed, 30 Apr 2025 00:16:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972210; cv=none; b=HtHB+ZdNvQTjeg/5Tthmzun6AdPYIMzRLCpSRnG9Zlm3nD5SCcaw0Nk2y8Mn9T3aoIV3L3G1x9Cvi24U0/McLn5rk+yQ+ifECX1w3wGOvt4XtG34IRShBh7fkIgXqZ2U1BFO+PKgGhbt+JYxollvsi7Mhyb4JB0uEgZkb9p/GjI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972210; c=relaxed/simple; bh=5l4eYXK/UGIl+TXQnZwANU66ClnBTNssKdiHYV/yMZ4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ANvwaswbBdlGjxzSuDxsa0K53xWcUv84fRIgwgrwEn6XbocU7pz0cV3ZVJo/y0DjSqR4640PVSJ2piYF4DgcvD0D4b4DY8EXsX1OtLlX0uMWAw87iTkhfPNTNfy8cqHhmAT9R4BaAYXFoFfrkMTjxo6jHBASCXKcGdSQci84RDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=DE9+7HPz; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="DE9+7HPz" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2279915e06eso76248605ad.1 for ; Tue, 29 Apr 2025 17:16:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972208; x=1746577008; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2tvKQDIIBJb26wPDVIJoD2xzubcY8uay1NjKLxC4PFw=; b=DE9+7HPzxPOEaibFgUkjhUnhAkT0ySzDmikR/k1rm2Y6ZTisTHjfTdTfZZjcXdRtOx O/TU7Vc28+25rPUdh2JT2ntq4cnVyXd5PL3TugXpxlqxVhK737/bV4P5Lwwq0bNCgac8 DUu+5LsVEhzk3fSSRRqwR68lHFSonSOTa4wnIPOTrI8JxTS1NU5aGKOPgluBUMUimWlX +zGLtXRbzjVEhK2ONU1obN6s5r8NeN7Pig/PUgztyGCxjLlSAo+MF3PkayfZ5fpSWOWP MM/fftKggRC86X5Ueb11qnIdXXefbQL+GxzJUw77ECo9+1KEyyPsX2LWX8ynnAsE9hG/ Kkxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972208; x=1746577008; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2tvKQDIIBJb26wPDVIJoD2xzubcY8uay1NjKLxC4PFw=; b=X/1UgVnQTbP1VsscOSPvXUC8ovqkmzukc1kQyEOAJ9OFh1DTzGBrrLOToMrxviUYQs ji9QzlJOyLqhfkw8PVBksGCURdCYie1puzfSEj4rkpKN/FiZwAvSA1g8MsmKO5MDe5S5 7Z7lOrbJylO9BqbJerQHQMHbvgXM2o26OVpKOtzZy0gDAmLLsyjMlUETpWsO+rmF0XL1 kRaqG7+tgydUxapqQ7XvBz+2TRvvXTCfmXzLVgY5/Hb6dmACV2HNjh9srEVRqQvp/mWU sOf8gkh4gsmvHV3PlC9wA/21q65kcRYMchLfsXidd0sXg9dPF2t/XI8OZ8FZw6ILisyH cEbw== X-Forwarded-Encrypted: i=1; AJvYcCUkMdRNuD9PveMDi4NYn+/DRjujkLbLQPrzGuoE/SG/Lfv/4ZR0MZjKzp+NZPiTGIaxD0HGpXxi4GAwxGK3+RM=@vger.kernel.org X-Gm-Message-State: AOJu0YwgC79zvS39x9zqAVxtbXGE+cZ2Bb3QZXytemw9tXnQmrmiSQS7 OB3+7PyquAjh+IRCbV7JMwvwxTzZxPCDs3aojLJm5ajUD5u/Ij75hEvW5FmcYo8= X-Gm-Gg: ASbGncse/mY2LlWsbRjk60d0TYSYlSx9aNEvb7ATt2c301As5EI4C3Lg8xje0A24IXY gw9/W2OPzWQSuzwuyx6DlzSkJWr1tZGy8fIJJe40FF0ZrfwPO3AynB2Bw3SgadIP9ZD+PTCfll3 tWxsTpkZZ80tLY5LI1hjYCSUMzdlnkJHsuo4uBj67+X9TVbZVjxx7nfsNPucF08QbVKnVziFoUp q8mQczoQO0enYYFPx/vKo0MuP1arUTZkorCbozZU5tuUsoX0URCngvOI7mBtfguWe/pYBttSY73 JZzVAtZFHLHaR7t7rLuVp7Shby26tnbtzVvwDp3RTyjbWmEboG4= X-Google-Smtp-Source: AGHT+IHF0KduTnYGQrePlJEI6tXkimvw7nrR1QkHLBA+cw8lQBj1dgbc3Ah+yAI0hO6Ky9wQoxTVRg== X-Received: by 2002:a17:902:d4d1:b0:223:67ac:8929 with SMTP id d9443c01a7336-22df56bf231mr8352805ad.0.1745972208061; Tue, 29 Apr 2025 17:16:48 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:16:47 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:24 -0700 Subject: [PATCH v14 07/27] riscv mm: manufacture shadow stack pte Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-7-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 This patch implements creating shadow stack pte (on riscv). Creating shadow stack PTE on riscv means that clearing RWX and then setting W=1. Reviewed-by: Alexandre Ghiti Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/pgtable.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index dba257cc4e2d..f21c888f59eb 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -433,6 +433,11 @@ static inline pte_t pte_mkwrite_novma(pte_t pte) return __pte(pte_val(pte) | _PAGE_WRITE); } +static inline pte_t pte_mkwrite_shstk(pte_t pte) +{ + return __pte((pte_val(pte) & ~(_PAGE_LEAF)) | _PAGE_WRITE); +} + /* static inline pte_t pte_mkexec(pte_t pte) */ static inline pte_t pte_mkdirty(pte_t pte) @@ -778,6 +783,11 @@ static inline pmd_t pmd_mkwrite_novma(pmd_t pmd) return pte_pmd(pte_mkwrite_novma(pmd_pte(pmd))); } +static inline pmd_t pmd_mkwrite_shstk(pmd_t pte) +{ + return __pmd((pmd_val(pte) & ~(_PAGE_LEAF)) | _PAGE_WRITE); +} + static inline pmd_t pmd_wrprotect(pmd_t pmd) { return pte_pmd(pte_wrprotect(pmd_pte(pmd))); From patchwork Wed Apr 30 00:16:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886833 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54ADF2566D1 for ; Wed, 30 Apr 2025 00:16:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972217; cv=none; b=PZa4tOANelv9Em8hMY8ighZWsog4QCGSHuKTIZg+pQIuUlr8W00I4Py1IQisgkw5Kp64XQJVGsr1IJIAhvxsu/PS6shHcLnTX1jl+NfLsRtJ3aXw828mpsUIxVhh/eZJixCAAnCxR+WnVlmblJMlsKo19ii6grtb5GaXYi+uw0Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972217; c=relaxed/simple; bh=6epe1IklbdE30nYdZrV0ZHwki5ITawyBnbVBknH019E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bXkN5wkVFQ0aiOyJWPQWN+JyhuT6PodotEw2yVWevXV9LiI1G7AV6QP8IawFQa7T634ibLHgTxntpZKm7H6kMDkVtqaecW71X/T2zGZidgMk4ezXQMj3z5HxJxgVZoH3lLYKJrKMxj4ofTvXhngbbk/sLDh2w/MrfTEern3Jz4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=L22v4ol8; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="L22v4ol8" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-22d95f0dda4so96380545ad.2 for ; Tue, 29 Apr 2025 17:16:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972214; x=1746577014; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=DsODLDfRcPuDSErfbs+tzfaJx/MFrIF5cCrvvQql1Yk=; b=L22v4ol8y8KR7CzqC463COkzA8qSb4dl9YBn/Ju48YLVisEsN4Wk8NzMdYkiQH9wHM Z3lcFxqN6MUHZPc/cvJgI/a/xAbbJqN6oDl2Nq3FR05E/WbSbczHAFcYRhifIQa2u5ii NXtLfSkO5ocEwxI8ZciO+kb6otd++pV/hax+sCmk690uyCTvoUaXj26L2bjp4kuVemCh 1pdHrSdtblvTVgL2Ij84crtTV3vmuM3Lv0iRiGJfIPBnSNCGzJU4e8B30tf+hUC/+Vbm JadZWuirlsBfKoP+wV6bSp0eihzJD0UTCh76ECgDxmD74TUKgz0KFT4JVLccE6TNg3go w2dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972214; x=1746577014; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DsODLDfRcPuDSErfbs+tzfaJx/MFrIF5cCrvvQql1Yk=; b=GLRMjOb2ess3WYcNcjcakijUZwrrIDgftKEAIWLvJ7umHOKHwbYuWdSfN6uJCbYbIC GteBoOEYGTqEEYxh18wCC43SF/wTS+fgy7kL3hZqzGu2phjOhmK2FvhUAY5ydtMgTobZ QbHat0g5Ypw0gAHZmKUguQQH6bu23Jlbde/Y9oNvRCE2Je4qsuFbKKYH45UOk0UXYFJF 7PH8qF58e33+8EcYWFOHdGmwUlnUpvi5ObL4itzYKLI9jJdTfqMIslq6JbtGFgl04JSl H7JjTWyNf25LUWAkd81deeYqv/cCdY03ijZ3ixgFeP55MpcNvxK9lSMXD+yTha/Wuyi6 8oug== X-Forwarded-Encrypted: i=1; AJvYcCXViksNNoH+ZKM3REW7nkHtBeLbiGpzj7yHhvhQSAFfNWdv3psVnl6IO6bdHudLqI0VABh8CipkBGaeD4xBZ+0=@vger.kernel.org X-Gm-Message-State: AOJu0YzZSqK44uj7l7vgLS5hutwU68wh0DBHf0890TRIEFBmeVUN1ikI 8QAsLkIA3uycfXmPUmyEUTDAzM/5CVYA0PrQ0NRO2hSuyGcnc1D09mu+VDBSweA= X-Gm-Gg: ASbGncsEXQHm5T6OFY5FNlVG37UVyZf1DDJgpbgrdTAcFnLTuE0/cl9wR+Wuv26X2KF egTFkV7CPzY7r3lG63aDOw+uMfR6goHtivEsdRSjCyRgL87DbPMvmhA07fzCVfuvkXaA0NBkvF0 Nq2at2i8pQxRnb432Ients7uqJKPjDL8oSA3mnkvB48z2cj0ND+ZP02szA59NvjwfeR9U852RJH LerS8ciiwQReL93S+ERvOMDCTBTtv7zdN6zLn1qzfhroxeggRqBEJUx6M6SkW7drpyZ4PNzBkF9 wl3IBcz56Yuoy1lCxJxL+VhJcJotkSTdG91peTnMoFnCnfwZVTc= X-Google-Smtp-Source: AGHT+IFJTmgyX5+R2cHG56gTAhOoSWDR7FtiznejYy+qmKOCH5eDKH+EG+01nS/OcUb7Pgf8hhzsSQ== X-Received: by 2002:a17:902:fc47:b0:21f:dbb:20a6 with SMTP id d9443c01a7336-22df35a39b4mr20572645ad.33.1745972214477; Tue, 29 Apr 2025 17:16:54 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:16:54 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:26 -0700 Subject: [PATCH v14 09/27] riscv mmu: write protect and shadow stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-9-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 `fork` implements copy on write (COW) by making pages readonly in child and parent both. ptep_set_wrprotect and pte_wrprotect clears _PAGE_WRITE in PTE. Assumption is that page is readable and on fault copy on write happens. To implement COW on shadow stack pages, clearing up W bit makes them XWR = 000. This will result in wrong PTE setting which says no perms but V=1 and PFN field pointing to final page. Instead desired behavior is to turn it into a readable page, take an access (load/store) fault on sspush/sspop (shadow stack) and then perform COW on such pages. This way regular reads would still be allowed and not lead to COW maintaining current behavior of COW on non-shadow stack but writeable memory. On the other hand it doesn't interfere with existing COW for read-write memory. Assumption is always that _PAGE_READ must have been set and thus setting _PAGE_READ is harmless. Reviewed-by: Alexandre Ghiti Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/pgtable.h | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 60d4821627d2..4e3431ccf634 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -423,7 +423,7 @@ static inline int pte_devmap(pte_t pte) static inline pte_t pte_wrprotect(pte_t pte) { - return __pte(pte_val(pte) & ~(_PAGE_WRITE)); + return __pte((pte_val(pte) & ~(_PAGE_WRITE)) | (_PAGE_READ)); } /* static inline pte_t pte_mkread(pte_t pte) */ @@ -624,7 +624,15 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep) { - atomic_long_and(~(unsigned long)_PAGE_WRITE, (atomic_long_t *)ptep); + pte_t read_pte = READ_ONCE(*ptep); + /* + * ptep_set_wrprotect can be called for shadow stack ranges too. + * shadow stack memory is XWR = 010 and thus clearing _PAGE_WRITE will lead to + * encoding 000b which is wrong encoding with V = 1. This should lead to page fault + * but we dont want this wrong configuration to be set in page tables. + */ + atomic_long_set((atomic_long_t *)ptep, + ((pte_val(read_pte) & ~(unsigned long)_PAGE_WRITE) | _PAGE_READ)); } #define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH From patchwork Wed Apr 30 00:16:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886832 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB397257455 for ; Wed, 30 Apr 2025 00:17:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972225; cv=none; b=o0rsHFTYu/gND8DboUVgmT/nOXFQajc1efptaiTCpxJDzo2n4xhoPGvjOIX3sZx7iBfQj/zay56VVHrTODKMfhE7YyO+JD1r6fk54STtPQdfr9P9CfdS+W4Psf40ajEbWTWd6GsfOXDiku6v42erQaH8xrSxWwG5iVlGog/IoSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972225; c=relaxed/simple; bh=Meuj+MngqT5Zko8mx51h2YgkzW9CZE8rsTJNoWsCvP8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=O5+kAnFmwRfXGMZmv7Qm7xJEgprxFGZZ/wA3e7igf+RB+LQfCKn/DFg9yrP+zf9D7X2H3kP2jITpl/aXOkbn/9cQVwcT2TFvNvoNyEJjrPBRAughg4LCRICZmwHLUc2U7wZVK5p3hRo0/fzexS7dBZ4NBsnfLDY2ZAZU2s4HEzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=zQkNhvSY; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="zQkNhvSY" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-227cf12df27so4226355ad.0 for ; Tue, 29 Apr 2025 17:17:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972221; x=1746577021; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=HZxLHlD3l8KQyqfBm0pNowD5Eu+jDmaJ7iNXDIvukjQ=; b=zQkNhvSYOEkYC+dNOZ26xRRjrcP7diEcxClcHS/+llkbgJuxVjSraodtLPeZCogo6f EDKR/K0+3Z7CnDIHedU79RGwq8cRL6w181UGR4LP9cfQnsgBL5qatL8+e1zFVBfVrToM n4H4x/NFU38agkwyJ5wWqlsghXoVI3HKnmUEF8fMIG2V0j/owR/F1IjCLG0/Yqdm/icW LGoYmuSZpDJR2nCZ7bvnYb0gfBr6VNLf1Lejm3rVP4L7Ne5uhOxlL9pNT7qAO1gC9Xo7 q77K3toHc9VWWNaLDtM26UI0uz17vt2wwfYowT+9GY+i0I7rHtcF09T/BXp/CBRUafWZ 3Fzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972221; x=1746577021; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HZxLHlD3l8KQyqfBm0pNowD5Eu+jDmaJ7iNXDIvukjQ=; b=DkjpvterHX1ZME9nNBXCuYXRiB4FI0y1bPsiNoPvsqVVwG5w5jBJwJpD7jwW1lJFW+ qVJeYKo7MInZieKWxnS3mvaRteh4Vp6xLp76T9UmNqYvPqc+rY2ODC22sKNlaA0MBrnU ZDcSzei2aWCEFht64p25LWWNzV/C3ba/4ZWoDid2WRgeGNHGuJ3LT285JbrbiNbBZ+OD uStr+Fym8csOnL2h0zFGhoZpCsHPid7Ixn6nKIHN5AI/m21tVcxdhWkNCIQa8GCov/+R ZcFYZe0XChKppHQPY3xn1JSR4wBm2V8JVfwhxWQKv/LvbhEhZnbEFHAH343vfS/wHKvY DWbg== X-Forwarded-Encrypted: i=1; AJvYcCXTg1POIeEHb1S+QsehamsNPzpIdv1DOHnIKzOO8qDOCZ81y18loUyZtQ8NcH7MVTSkWRtcoHKtH2hdZLnbGWU=@vger.kernel.org X-Gm-Message-State: AOJu0YyUFDlwGm06r4BgJ/jMay90GpRlG94F5SWAJzP7D52dZwOJ1x0t 0tiJpwUMs3aM19R49x3WRok8XSe0ZLxXVlKbfrNMLyEwhtK7MUJU6ggXz7C+8sk= X-Gm-Gg: ASbGncu29BIZBDgsE5TF3/96mpyzprq9GE30HtF95forrr4jl0c+b1cEg+SecSJvZsw M10uAUmMlsTcnfaAGW9G2cmq5i92H+HYF17P0QijLUMdB+EADxjAGiAw+/Ox8LiC8T3dF5oOU2D G2Ti4Mobn+h3Sy4exvIEcHcehRd49ZxN+DVWiH/GeDj71AyeCdUj/YVSUjTimq1+mu0roFTyY0i IfKVFlAktdaxaRvJ2YA7dy7I55anKZiLtUhGf1GU/DtuH5aXX7d//CUwgGovFym527ZgJ5hK14X YoNqfXZeik9rmFTXGcJjZIizKOOpPGnulku38g71Kfwv/zsLFCM= X-Google-Smtp-Source: AGHT+IFHUa1gkfbu4YATBNxYYRy+IpJ9BfKSEyfxBEJdg/REwdO6LsoaUDr+6BDe+fF4LnfflTj+RQ== X-Received: by 2002:a17:903:1a2c:b0:215:a303:24e9 with SMTP id d9443c01a7336-22df4748c43mr10916095ad.3.1745972220890; Tue, 29 Apr 2025 17:17:00 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.16.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:00 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:28 -0700 Subject: [PATCH v14 11/27] riscv/shstk: If needed allocate a new shadow stack on clone Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-11-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 Userspace specifies CLONE_VM to share address space and spawn new thread. `clone` allow userspace to specify a new stack for new thread. However there is no way to specify new shadow stack base address without changing API. This patch allocates a new shadow stack whenever CLONE_VM is given. In case of CLONE_VFORK, parent is suspended until child finishes and thus can child use parent shadow stack. In case of !CLONE_VM, COW kicks in because entire address space is copied from parent to child. `clone3` is extensible and can provide mechanisms using which shadow stack as an input parameter can be provided. This is not settled yet and being extensively discussed on mailing list. Once that's settled, this commit will adapt to that. Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/mmu_context.h | 7 ++ arch/riscv/include/asm/usercfi.h | 25 ++++++++ arch/riscv/kernel/process.c | 9 +++ arch/riscv/kernel/usercfi.c | 120 +++++++++++++++++++++++++++++++++++ 4 files changed, 161 insertions(+) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index 8c4bc49a3a0f..dbf27a78df6c 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -48,6 +48,13 @@ static inline unsigned long mm_untag_mask(struct mm_struct *mm) } #endif +#define deactivate_mm deactivate_mm +static inline void deactivate_mm(struct task_struct *tsk, + struct mm_struct *mm) +{ + shstk_release(tsk); +} + #include #endif /* _ASM_RISCV_MMU_CONTEXT_H */ diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h index 94b214c295c0..0581587490c2 100644 --- a/arch/riscv/include/asm/usercfi.h +++ b/arch/riscv/include/asm/usercfi.h @@ -8,6 +8,9 @@ #ifndef __ASSEMBLY__ #include +struct task_struct; +struct kernel_clone_args; + #ifdef CONFIG_RISCV_USER_CFI struct cfi_state { unsigned long ubcfi_en : 1; /* Enable for backward cfi. */ @@ -16,6 +19,28 @@ struct cfi_state { unsigned long shdw_stk_size; /* size of shadow stack */ }; +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); +void shstk_release(struct task_struct *tsk); +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size); +unsigned long get_shstk_base(struct task_struct *task, unsigned long *size); +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr); +bool is_shstk_enabled(struct task_struct *task); + +#else + +#define shstk_alloc_thread_stack(tsk, args) 0 + +#define shstk_release(tsk) + +#define get_shstk_base(task, size) 0UL + +#define set_shstk_base(task, shstk_addr, size) + +#define set_active_shstk(task, shstk_addr) + +#define is_shstk_enabled(task) false + #endif /* CONFIG_RISCV_USER_CFI */ #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index 7c244de77180..99acb6342a37 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -29,6 +29,7 @@ #include #include #include +#include #if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_STACKPROTECTOR_PER_TASK) #include @@ -211,6 +212,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long clone_flags = args->flags; unsigned long usp = args->stack; unsigned long tls = args->tls; + unsigned long ssp = 0; struct pt_regs *childregs = task_pt_regs(p); /* Ensure all threads in this mm have the same pointer masking mode. */ @@ -229,11 +231,18 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.s[0] = (unsigned long)args->fn; p->thread.s[1] = (unsigned long)args->fn_arg; } else { + /* allocate new shadow stack if needed. In case of CLONE_VM we have to */ + ssp = shstk_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(ssp)) + return PTR_ERR((void *)ssp); + *childregs = *(current_pt_regs()); /* Turn off status.VS */ riscv_v_vstate_off(childregs); if (usp) /* User fork */ childregs->sp = usp; + /* if needed, set new ssp */ + ssp ? set_active_shstk(p, ssp) : 0; if (clone_flags & CLONE_SETTLS) childregs->tp = tls; childregs->a0 = 0; /* Return value of fork() */ diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c index 0b3bbb41490a..ec3d78efd6f3 100644 --- a/arch/riscv/kernel/usercfi.c +++ b/arch/riscv/kernel/usercfi.c @@ -19,6 +19,41 @@ #define SHSTK_ENTRY_SIZE sizeof(void *) +bool is_shstk_enabled(struct task_struct *task) +{ + return task->thread_info.user_cfi_state.ubcfi_en; +} + +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size) +{ + task->thread_info.user_cfi_state.shdw_stk_base = shstk_addr; + task->thread_info.user_cfi_state.shdw_stk_size = size; +} + +unsigned long get_shstk_base(struct task_struct *task, unsigned long *size) +{ + if (size) + *size = task->thread_info.user_cfi_state.shdw_stk_size; + return task->thread_info.user_cfi_state.shdw_stk_base; +} + +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr) +{ + task->thread_info.user_cfi_state.user_shdw_stk = shstk_addr; +} + +/* + * If size is 0, then to be compatible with regular stack we want it to be as big as + * regular stack. Else PAGE_ALIGN it and return back + */ +static unsigned long calc_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + /* * Writes on shadow stack can either be `sspush` or `ssamoswap`. `sspush` can happen * implicitly on current shadow stack pointed to by CSR_SSP. `ssamoswap` takes pointer to @@ -141,3 +176,88 @@ SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsi return allocate_shadow_stack(addr, aligned_size, size, set_tok); } + +/* + * This gets called during clone/clone3/fork. And is needed to allocate a shadow stack for + * cases where CLONE_VM is specified and thus a different stack is specified by user. We + * thus need a separate shadow stack too. How does separate shadow stack is specified by + * user is still being debated. Once that's settled, remove this part of the comment. + * This function simply returns 0 if shadow stack are not supported or if separate shadow + * stack allocation is not needed (like in case of !CLONE_VM) + */ +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + /* If shadow stack is not supported, return 0 */ + if (!cpu_supports_shadow_stack()) + return 0; + + /* + * If shadow stack is not enabled on the new thread, skip any + * switch to a new shadow stack. + */ + if (!is_shstk_enabled(tsk)) + return 0; + + /* + * For CLONE_VFORK the child will share the parents shadow stack. + * Set base = 0 and size = 0, this is special means to track this state + * so the freeing logic run for child knows to leave it alone. + */ + if (args->flags & CLONE_VFORK) { + set_shstk_base(tsk, 0, 0); + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow + * stack. + */ + if (!(args->flags & CLONE_VM)) + return 0; + + /* + * reaching here means, CLONE_VM was specified and thus a separate shadow + * stack is needed for new cloned thread. Note: below allocation is happening + * using current mm. + */ + size = calc_shstk_size(args->stack_size); + addr = allocate_shadow_stack(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + set_shstk_base(tsk, addr, size); + + return addr + size; +} + +void shstk_release(struct task_struct *tsk) +{ + unsigned long base = 0, size = 0; + /* If shadow stack is not supported or not enabled, nothing to release */ + if (!cpu_supports_shadow_stack() || !is_shstk_enabled(tsk)) + return; + + /* + * When fork() with CLONE_VM fails, the child (tsk) already has a + * shadow stack allocated, and exit_thread() calls this function to + * free it. In this case the parent (current) and the child share + * the same mm struct. Move forward only when they're same. + */ + if (!tsk->mm || tsk->mm != current->mm) + return; + + /* + * We know shadow stack is enabled but if base is NULL, then + * this task is not managing its own shadow stack (CLONE_VFORK). So + * skip freeing it. + */ + base = get_shstk_base(tsk, &size); + if (!base) + return; + + vm_munmap(base, size); + set_shstk_base(tsk, 0, 0); +} From patchwork Wed Apr 30 00:16:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886831 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 225602609CD for ; Wed, 30 Apr 2025 00:17:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972230; cv=none; b=J2Ut/Q2GuxXeYfl5e+KddQHE8v0dWy8l9/4UkpBAtQ3oJMqmXWEVbG0BiNLpyoJnUCWN2OGcXSECEWn24dbQ14IQyQFKJ/UNJMgQgAeIwdx/G7UvupaWiecCsMAxiGtlQVqvRLO/Cd7pXMysEdt8al78EpdFWjjJJHaapnvSAXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972230; c=relaxed/simple; bh=4V/vc+IGuEXLJLSMGAlz1mzVikhLjzrFyJ0kt0SiHdA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Fg2jihars5dQa9HMCgNk8UkyBgrA83xajxBJZqY0I/GeqnYvBCm9BIbNYGRiia+A+DNQEFElHeFCrEd5jYLLbU1fig6M5UtoIH0J+Bw2DarDMwn1MmiLoMBfYUeRTzDyY4Bv+kGeAsC9FUyb2pI58jb648YC1tGveseCY1NFhvo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=xOAnbh4L; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="xOAnbh4L" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-22c33ac23edso61521765ad.0 for ; Tue, 29 Apr 2025 17:17:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972227; x=1746577027; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Wqa+EnEmqD/WcxsFXdACBzGsJGhuUJWdKCVGNJwyc08=; b=xOAnbh4L4jFlBiCPu+/FlZJCwc0JLiVSzF3TxIHj2ISzRc8R/xaMq2jWRjWxyBSIvJ l+3Ru1kihn3EtzzDpTFHE4aDzaHxFzLOgdVaHm/f5CvpODglhPOBLOdPxFrItPKVg/9D 6vzZN/Oh7OWzeaXGAgWjsWgxwEg2X05zCoFxhQnJMhMYWaHyR0jLiCo2iC4WzxHMK8YI BHs/FuYDA2WGIb3/65olA4YnZGyrHOdmMFntuOznGKalYEly2uVXd+M5vZpnuf9cTN08 kMPXyvkcdb5TVQjZJFrlLJPtz26JIZ4H9kkKcvXBlqIFI3AV3yaZ7sqrEZJhrYWKl5tV 3/5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972227; x=1746577027; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wqa+EnEmqD/WcxsFXdACBzGsJGhuUJWdKCVGNJwyc08=; b=UJCahaAoVaQDKrUE/vhZoGBxHNXUCsNdSG+MtIpeqyDhc+pBdlJobbdJYKnO4zwT8T mXj9MtDwK57rDF7F3bH2p71Vmstrg5Nx5Ywo72t0kb7i0+HeRN73qoNbau3R9cwEJy2K oWuiqONlFL3D8o/JPQZejbKKIVzQnPbcJjJPZjt3rI9e63NPcDliTZIaguZAWqtU+Lsn 8r2dfBWpPeVmoxoMN5p6cRBSgR2PT8bA/WXm03+G20ereMSuWMnlrC1y56jgKPKrvegh h5Ihygmtj33ABLdQg2Jwf0mBQtMu1tvqAXyqe0LxfU1lPAjkOPYCt4+BHzCQFM/Ok6ls 5o4A== X-Forwarded-Encrypted: i=1; AJvYcCVCHfIMUDNucQrywcUzRaDixMct7rkNNfIQpscMgw3GpUOHtxa5L4QnorSvMnPa3CjnllsVdIYYf2REFUkKF9Y=@vger.kernel.org X-Gm-Message-State: AOJu0YzJ0hRuiV/By5ZvJmLKHcJ6ZaeyQagTwMtt7akz509IUfHEz+2d xmmx8pGJRUKRrNhuWxXp5y1gcxahmGbrAicvzvn5N7hEk69POb8djTalI/52LYY= X-Gm-Gg: ASbGncuEgZVMq5x9CERS3twng/vzG+e0E/6PG/EDgC8/gbykk8XEhL8CZopDrQoqU/H yyCZJbbcGWXP2lsobSqXzJLZ9FZAVk/d3ncxq/N/7yQog5ygg9/MxWpGif8UAqfirlsYLx7K5WG t7kh3s4vswJk9p8QOtg9X2l/B2mCkdsTQxGaRgH4DvBYZcq3bvnCesMmzk17x73wQF0K6QNynoZ +S+uVEqcQrvyBEOSXsDj0tQNKMgqBTJ1HfbXvmemjzdJXBGUZA1RTLJjBpVRkNnOE0v6GYPCnQf fY8kWxK0de9g5cBDOPS0g1PXb5Z5KFekJALpKE45i+HgnZQwqXk= X-Google-Smtp-Source: AGHT+IHQiHW6ihtjUTT0DTqmkuWcAmVo/rllEq5v8YlabaxyEntdh+cJYjoV0mkSuSPow8vsrYIpLg== X-Received: by 2002:a17:902:d4d1:b0:223:669f:ca2d with SMTP id d9443c01a7336-22df356d961mr20640625ad.35.1745972227361; Tue, 29 Apr 2025 17:17:07 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:07 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:30 -0700 Subject: [PATCH v14 13/27] prctl: arch-agnostic prctl for indirect branch tracking Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-13-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 Three architectures (x86, aarch64, riscv) have support for indirect branch tracking feature in a very similar fashion. On a very high level, indirect branch tracking is a CPU feature where CPU tracks branches which uses memory operand to perform control transfer in program. As part of this tracking on indirect branches, CPU goes in a state where it expects a landing pad instr on target and if not found then CPU raises some fault (architecture dependent) x86 landing pad instr - `ENDBRANCH` arch64 landing pad instr - `BTI` riscv landing instr - `lpad` Given that three major arches have support for indirect branch tracking, This patch makes `prctl` for indirect branch tracking arch agnostic. To allow userspace to enable this feature for itself, following prtcls are defined: - PR_GET_INDIR_BR_LP_STATUS: Gets current configured status for indirect branch tracking. - PR_SET_INDIR_BR_LP_STATUS: Sets a configuration for indirect branch tracking. Following status options are allowed - PR_INDIR_BR_LP_ENABLE: Enables indirect branch tracking on user thread. - PR_INDIR_BR_LP_DISABLE; Disables indirect branch tracking on user thread. - PR_LOCK_INDIR_BR_LP_STATUS: Locks configured status for indirect branch tracking for user thread. Reviewed-by: Mark Brown Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- include/linux/cpu.h | 4 ++++ include/uapi/linux/prctl.h | 27 +++++++++++++++++++++++++++ kernel/sys.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+) diff --git a/include/linux/cpu.h b/include/linux/cpu.h index e3049543008b..ea8b351b5bc5 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -204,4 +204,8 @@ static inline bool cpu_mitigations_auto_nosmt(void) } #endif +int arch_get_indir_br_lp_status(struct task_struct *t, unsigned long __user *status); +int arch_set_indir_br_lp_status(struct task_struct *t, unsigned long status); +int arch_lock_indir_br_lp_status(struct task_struct *t, unsigned long status); + #endif /* _LINUX_CPU_H_ */ diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 15c18ef4eb11..2e09b19317a3 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -364,4 +364,31 @@ struct prctl_mm_map { # define PR_TIMER_CREATE_RESTORE_IDS_ON 1 # define PR_TIMER_CREATE_RESTORE_IDS_GET 2 +/* + * Get the current indirect branch tracking configuration for the current + * thread, this will be the value configured via PR_SET_INDIR_BR_LP_STATUS. + */ +#define PR_GET_INDIR_BR_LP_STATUS 78 + +/* + * Set the indirect branch tracking configuration. PR_INDIR_BR_LP_ENABLE will + * enable cpu feature for user thread, to track all indirect branches and ensure + * they land on arch defined landing pad instruction. + * x86 - If enabled, an indirect branch must land on `ENDBRANCH` instruction. + * arch64 - If enabled, an indirect branch must land on `BTI` instruction. + * riscv - If enabled, an indirect branch must land on `lpad` instruction. + * PR_INDIR_BR_LP_DISABLE will disable feature for user thread and indirect + * branches will no more be tracked by cpu to land on arch defined landing pad + * instruction. + */ +#define PR_SET_INDIR_BR_LP_STATUS 79 +# define PR_INDIR_BR_LP_ENABLE (1UL << 0) + +/* + * Prevent further changes to the specified indirect branch tracking + * configuration. All bits may be locked via this call, including + * undefined bits. + */ +#define PR_LOCK_INDIR_BR_LP_STATUS 80 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sys.c b/kernel/sys.c index c434968e9f5d..91a1dc093c2a 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2340,6 +2340,21 @@ int __weak arch_lock_shadow_stack_status(struct task_struct *t, unsigned long st return -EINVAL; } +int __weak arch_get_indir_br_lp_status(struct task_struct *t, unsigned long __user *status) +{ + return -EINVAL; +} + +int __weak arch_set_indir_br_lp_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + +int __weak arch_lock_indir_br_lp_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + #define PR_IO_FLUSHER (PF_MEMALLOC_NOIO | PF_LOCAL_THROTTLE) #ifdef CONFIG_ANON_VMA_NAME @@ -2820,6 +2835,21 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, return -EINVAL; error = posixtimer_create_prctl(arg2); break; + case PR_GET_INDIR_BR_LP_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_get_indir_br_lp_status(me, (unsigned long __user *)arg2); + break; + case PR_SET_INDIR_BR_LP_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_set_indir_br_lp_status(me, arg2); + break; + case PR_LOCK_INDIR_BR_LP_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_lock_indir_br_lp_status(me, arg2); + break; default: trace_task_prctl_unknown(option, arg2, arg3, arg4, arg5); error = -EINVAL; From patchwork Wed Apr 30 00:16:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886830 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE44926156B for ; Wed, 30 Apr 2025 00:17:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972236; cv=none; b=aagCefdqZYJ0woTGA6VyQKFi8OZUbD/5FZJwgGOxEAVLN+TD0IRzhN734HJxEJyAGBiyDkNGZzrHvd3tKer08bfvZCRbzPUWbPqGTVX1pXofxm6/Zdpo75umuqTBo0jJ/f3kU8KunalsvJNRr7ZboeQhChS0nODCgWoy7A/Z324= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972236; c=relaxed/simple; bh=jQexnjOya22ysgOQudOcRfuyX2ffnh58PKctRMnRFxk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tj6quiKlO6sSOdFARKD8DDJOOc0ofN6KXJeA1FjyHR/w1iYjvOX6rjwTxlmROlp2eq8a17rOJ58rMNGhTKGjPss7nvIocjqSHNC/KNlQIb6pZMUKvr2aNfIr/bDElc4IthPfb5Q2D/vw+4eHYPUoq2mbRirZUbIEBV6qn9zJPuw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=lw6txBrU; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="lw6txBrU" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-223fd89d036so83503425ad.1 for ; Tue, 29 Apr 2025 17:17:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972234; x=1746577034; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=zzG5aBj4uT4Cxy0Ye5VbMroQI6g4J7udAW2U71MrLVg=; b=lw6txBrU0zW+NCIgtyExhOJFDwNCGAIBJ8FNAW7U0hDSrTnWuCt77e4s0QzsXJbOYo A6V2DjFT9PsGmnbKSm1MonIHTeJSyD90hYQZE94vntkTq9wifQgrtPasB/EyFyGGHhhD 5OiBSBMM6KEfVYMwAZxhRHw2IIeDdlSsjccTvYvlG0NmS7xkWHazWm+8Oqyj+jeYYP7m XO5p0Q/j6sPDSQ+yGSYZ2XAupmgYqudYy+jzGEnfnPgXw1ZSDofzGzpaZwS3P6JrBlqB bryYsqG8StOqVab6KdnU2kJj6gGJV+rYVuDskXCqMujmdXnhLEchrinZt8O31HOQEzKx b2sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972234; x=1746577034; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zzG5aBj4uT4Cxy0Ye5VbMroQI6g4J7udAW2U71MrLVg=; b=v7ZrPpzbB4jcbCa8/ylpUBStVvcYXU/QGXaeaOkCAmtZPvCckdSybObQV6PNJmK2xO /3GI2Mcr/lttxiZwFEeM8fV0iQML2aw+AHDWabwYKSACIcPTTb8jshALUYeEMK+reXXP FOqmuPvSMimcJ/hmHtq3RFSTWWXuV5Dkyvm2mJRjJ+LXmkTfBkzQ2BQOg+RAU7ZXcAfG kbOITV4PQiUDY95ty1i+LNijkKe+4aWBy7zONpuivE5V3LF7jtdNhHKfd+D1RkIYAgRx c2RXPtylyALOrlLpUzPvxpFX55h504bI9BR/IsmCaiNeZYgKh9vryH8IKNnPKij/ore5 DSgw== X-Forwarded-Encrypted: i=1; AJvYcCXoDVRUhibzDRte6syyrALc9TRWlcAtZXm5M9lu1GPC55topjZcmmcfYS6VHhgb2+u+da+8eAcFAPl00HPB/nY=@vger.kernel.org X-Gm-Message-State: AOJu0YykebOWUu6w/VMC1tK4MdWqKeF9lxKHV9UUVw4hV0gw/VL5LUgt XOWK5m1ltsAQu5IELvR5cPEIx05o2rIufvOr7yhWrlJ2dK7/pVOanGGDk2ajhe8= X-Gm-Gg: ASbGncuJb+A+lLBrVDS4HVRAmTqPL2lmCM65Rb0kxNaOGt6+aav4H/moaUqJDhL1UGD Ml/4F5Yv9FQM0PmB3CTIxla3uwkM3OcSnRed/2SFuKvdG49bzEGyV0r8z1P6+K5x6MRHbFuQTOI 3hO0nuHy38wTdjlknnIrEDirJlOp7DSXH/sQg33aw9n30iXgORsnRBsgvynz2GyKiXnVWTSsa9p 8UqW7H3SIWpRyh0//2YoxPreCZCxEARKVBNiGrY9fVZQlZMQmeOUkC5zeem3rkkVtZkMpLWNCXc +7bLweYaLV3HTOC0NnHu/Gv/Gu+8nhRAdr/CGtLHY3QQZ0h61/A= X-Google-Smtp-Source: AGHT+IEyV8PM+Op85J8MN8UIXzOwmO5AoH1n2zuPf2BQnUgr2EBtE46vqxBibis0+RspBTbObsPbvA== X-Received: by 2002:a17:902:e54c:b0:224:1c95:451e with SMTP id d9443c01a7336-22df5827bcdmr6954145ad.33.1745972234019; Tue, 29 Apr 2025 17:17:14 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:13 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:32 -0700 Subject: [PATCH v14 15/27] riscv/traps: Introduce software check exception Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-15-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 zicfiss / zicfilp introduces a new exception to priv isa `software check exception` with cause code = 18. This patch implements software check exception. Additionally it implements a cfi violation handler which checks for code in xtval. If xtval=2, it means that sw check exception happened because of an indirect branch not landing on 4 byte aligned PC or not landing on `lpad` instruction or label value embedded in `lpad` not matching label value setup in `x7`. If xtval=3, it means that sw check exception happened because of mismatch between link register (x1 or x5) and top of shadow stack (on execution of `sspopchk`). In case of cfi violation, SIGSEGV is raised with code=SEGV_CPERR. SEGV_CPERR was introduced by x86 shadow stack patches. Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/asm-prototypes.h | 1 + arch/riscv/include/asm/entry-common.h | 2 ++ arch/riscv/kernel/entry.S | 3 +++ arch/riscv/kernel/traps.c | 43 +++++++++++++++++++++++++++++++++ 4 files changed, 49 insertions(+) diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h index cd627ec289f1..5a27cefd7805 100644 --- a/arch/riscv/include/asm/asm-prototypes.h +++ b/arch/riscv/include/asm/asm-prototypes.h @@ -51,6 +51,7 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_u); DECLARE_DO_ERROR_INFO(do_trap_ecall_s); DECLARE_DO_ERROR_INFO(do_trap_ecall_m); DECLARE_DO_ERROR_INFO(do_trap_break); +DECLARE_DO_ERROR_INFO(do_trap_software_check); asmlinkage void handle_bad_stack(struct pt_regs *regs); asmlinkage void do_page_fault(struct pt_regs *regs); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm/entry-common.h index b28ccc6cdeea..34ed149af5d1 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -40,4 +40,6 @@ static inline int handle_misaligned_store(struct pt_regs *regs) } #endif +bool handle_user_cfi_violation(struct pt_regs *regs); + #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 978115567bca..8d25837a9384 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -474,6 +474,9 @@ SYM_DATA_START_LOCAL(excp_vect_table) RISCV_PTR do_page_fault /* load page fault */ RISCV_PTR do_trap_unknown RISCV_PTR do_page_fault /* store page fault */ + RISCV_PTR do_trap_unknown /* cause=16 */ + RISCV_PTR do_trap_unknown /* cause=17 */ + RISCV_PTR do_trap_software_check /* cause=18 is sw check exception */ SYM_DATA_END_LABEL(excp_vect_table, SYM_L_LOCAL, excp_vect_table_end) #ifndef CONFIG_MMU diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index 8ff8e8b36524..3f7709f4595a 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -354,6 +354,49 @@ void do_trap_ecall_u(struct pt_regs *regs) } +#define CFI_TVAL_FCFI_CODE 2 +#define CFI_TVAL_BCFI_CODE 3 +/* handle cfi violations */ +bool handle_user_cfi_violation(struct pt_regs *regs) +{ + bool ret = false; + unsigned long tval = csr_read(CSR_TVAL); + + if ((tval == CFI_TVAL_FCFI_CODE && cpu_supports_indirect_br_lp_instr()) || + (tval == CFI_TVAL_BCFI_CODE && cpu_supports_shadow_stack())) { + do_trap_error(regs, SIGSEGV, SEGV_CPERR, regs->epc, + "Oops - control flow violation"); + ret = true; + } + + return ret; +} + +/* + * software check exception is defined with risc-v cfi spec. Software check + * exception is raised when:- + * a) An indirect branch doesn't land on 4 byte aligned PC or `lpad` + * instruction or `label` value programmed in `lpad` instr doesn't + * match with value setup in `x7`. reported code in `xtval` is 2. + * b) `sspopchk` instruction finds a mismatch between top of shadow stack (ssp) + * and x1/x5. reported code in `xtval` is 3. + */ +asmlinkage __visible __trap_section void do_trap_software_check(struct pt_regs *regs) +{ + if (user_mode(regs)) { + irqentry_enter_from_user_mode(regs); + + /* not a cfi violation, then merge into flow of unknown trap handler */ + if (!handle_user_cfi_violation(regs)) + do_trap_unknown(regs); + + irqentry_exit_to_user_mode(regs); + } else { + /* sw check exception coming from kernel is a bug in kernel */ + die(regs, "Kernel BUG"); + } +} + #ifdef CONFIG_MMU asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs) { From patchwork Wed Apr 30 00:16:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886829 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1318B264FBC for ; Wed, 30 Apr 2025 00:17:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972244; cv=none; b=lg6dx/u5SdKOzBU2MQ12bqn8hP9oDpADeYrEVSc+yw+3uQPZtt9moYfEI6G3juCgwGM26LYUOCY/1lzDhyhhgupok3jMAg55LTb9rZ2+0GEnL44thxkphjTNKUMVukoMjRQKDkiEQH6ssQewPiihBKEdcjknlADoAuzaaECLRko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972244; c=relaxed/simple; bh=gg2vJdQgN9X+9gIUXwpUpI2GPYjg2wVSASLNhtMDlR4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ikRxI7WxWxw9PdVJXZiowpD/xsv27+vfwDFMEplEGD2vKCpsd3xQ4WwnBeZSSPcrjRT1GjgzTtNMS/8VkPiwoyBloo3SW3tvkMAYtGlTLbuFjv1uGAYStiC965O8FT0a2ZsuJgH3wt7ZCSbyxA6i5siOxkpXz6PSlY6rHJ2fEsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=W4/WeOPX; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="W4/WeOPX" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-22c33e4fdb8so71133965ad.2 for ; Tue, 29 Apr 2025 17:17:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972240; x=1746577040; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=5J/5mZxT+k+LmAKU7G8MkVD/8s7ODhQZCUl5WVjl9l4=; b=W4/WeOPXbPEFpr/7o60qaeEyv3FBWP/wnfPTRphxjueOGVydnE9B1XLD9cwxtVuSbW ON6k2iS60pm82DD03UnlDN05gBtt/qfDNN/R8e2DxTjX5unHKHfDur0CBoIF/+v49Wlh o4zudPf3L37DADpjEovY/eZpeJvTBnUFElcSgXLoWs9fJJwmoXygkGBgbxMmMXbWsuF8 wwHKCldv3Os/Q5wIrk1mKtPO/r3ACHitzCE4ImTDzawZKT7/iweHB749NRn/WG4ez61l +uFtiQjXWMBVlKv8Hv52pgAY/PvTweeecV9tly6A2lT/y9hM+wQ1/giMWRmDvK/O//YM +bIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972240; x=1746577040; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5J/5mZxT+k+LmAKU7G8MkVD/8s7ODhQZCUl5WVjl9l4=; b=UTiR3qJ85iub8unmwn9SaIUSAXGgpvK0T7woJeBgKsQafmkSGZ6Adm6WjEIvo6d1Li pZNKd9or+n4xovnQ2Ve8w7AGxu+0/tfmj9I8wkYH8OBMgZo2qTTgrLoEmkl+JXGxlLi6 2nch/AUUZjLPpIcYg4jorqxdLcINPUJfznp6dGP5yjsZqmNmC8M33eRu1ZxO73T78fyH b1pJ4Rx259zfI13dhj0ZxPVe8toqmyG09ky8S6JlyHp91QR0sDSQacOrfV8hhog75bDH h20jrAr/SEpEsy8mDuktws20Ev8bPHCSuyFdFo238TapoQiU9IxmcAgUXFQT8S0CCp8K b/cA== X-Forwarded-Encrypted: i=1; AJvYcCWfSodP5JKG1jOoZQp7OUzuQSuxfFSuL6Aepu0jRvoj4n6IQAlg50hAPbB+XrxrVqkoVsTSlNHvIAbqMT83c+k=@vger.kernel.org X-Gm-Message-State: AOJu0YxCro7gD0byuRKfX68RS0K3yfYJPAlPu1AKaR6+bAtsUsNrw0aO HWLnZpfM6OMGSofQWlNdiyuN0qzcarmxUwOZesTJorzp3HqtGnG0UOi5i1pBgfs= X-Gm-Gg: ASbGnct1L/n6ad6ibtpk5Cje7RHg1XPzxdpH4924F5gdUnll4Rv8kQkUZwLky8CYHHN FpWu6Uin77jD651NR9p0GoJWdLTUarM7Avk6bSv7QWICyo2/zgzb0Ef6vpbSAs8NjrI3VNdRaVn KKgAoQQM3nQJ7+0Ys0aYNbnifcysj+s5Umsvh4x2MS5MzsNOMgpBXI8H9WL/NAvuWV5KXk/jKOu 37jhH/BrRCpiwvIsHGg0s9V0RdVbCHWMHSIiConn19Gf7hHYGKIu4ZERyNpCkpiNXGhWemA7bO/ Mbbwd6TQfYAPBImTBiShse08vJTxlSqdM2T84IKhCFjmmTO369E= X-Google-Smtp-Source: AGHT+IHUsd3WrSJ9gmGyyVk1RxhM9S+Uh5Mwx3YhJbCfwhMaqV/yOZGZCO9itWeSGD50TlYFP578aw== X-Received: by 2002:a17:902:f611:b0:224:78e:4eb4 with SMTP id d9443c01a7336-22df356e711mr16382915ad.39.1745972240329; Tue, 29 Apr 2025 17:17:20 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:19 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:34 -0700 Subject: [PATCH v14 17/27] riscv/signal: save and restore of shadow stack for signal Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-17-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Deepak Gupta , Andy Chiu X-Mailer: b4 0.13.0 Save shadow stack pointer in sigcontext structure while delivering signal. Restore shadow stack pointer from sigcontext on sigreturn. As part of save operation, kernel uses `ssamoswap` to save snapshot of current shadow stack on shadow stack itself (can be called as a save token). During restore on sigreturn, kernel retrieves token from top of shadow stack and validates it. This allows that user mode can't arbitrary pivot to any shadow stack address without having a token and thus provide strong security assurance between signaly delivery and sigreturn window. Use ABI compatible way of saving/restoring shadow stack pointer into signal stack. This follows what Vector extension, where extra registers are placed in a form of extension header + extension body in the stack. The extension header indicates the size of the extra architectural states plus the size of header itself, and a magic identifier of the extension. Then, the extensions body contains the new architectural states in the form defined by uapi. Signed-off-by: Andy Chiu Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/usercfi.h | 10 ++++ arch/riscv/include/uapi/asm/ptrace.h | 4 ++ arch/riscv/include/uapi/asm/sigcontext.h | 1 + arch/riscv/kernel/signal.c | 86 ++++++++++++++++++++++++++++++++ arch/riscv/kernel/usercfi.c | 56 +++++++++++++++++++++ 5 files changed, 157 insertions(+) diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h index cea7908cdb3a..68da5b7b79fd 100644 --- a/arch/riscv/include/asm/usercfi.h +++ b/arch/riscv/include/asm/usercfi.h @@ -8,6 +8,7 @@ #ifndef __ASSEMBLY__ #include #include +#include struct task_struct; struct kernel_clone_args; @@ -34,6 +35,9 @@ bool is_shstk_locked(struct task_struct *task); bool is_shstk_allocated(struct task_struct *task); void set_shstk_lock(struct task_struct *task); void set_shstk_status(struct task_struct *task, bool enable); +unsigned long get_active_shstk(struct task_struct *task); +int restore_user_shstk(struct task_struct *tsk, unsigned long shstk_ptr); +int save_user_shstk(struct task_struct *tsk, unsigned long *saved_shstk_ptr); bool is_indir_lp_enabled(struct task_struct *task); bool is_indir_lp_locked(struct task_struct *task); void set_indir_lp_status(struct task_struct *task, bool enable); @@ -71,6 +75,12 @@ void set_indir_lp_lock(struct task_struct *task); #define set_indir_lp_lock(task) +#define restore_user_shstk(tsk, shstk_ptr) -EINVAL + +#define save_user_shstk(tsk, saved_shstk_ptr) -EINVAL + +#define get_active_shstk(task) 0UL + #endif /* CONFIG_RISCV_USER_CFI */ #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index a38268b19c3d..659ea3af5680 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -127,6 +127,10 @@ struct __riscv_v_regset_state { */ #define RISCV_MAX_VLENB (8192) +struct __sc_riscv_cfi_state { + unsigned long ss_ptr; /* shadow stack pointer */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ diff --git a/arch/riscv/include/uapi/asm/sigcontext.h b/arch/riscv/include/uapi/asm/sigcontext.h index cd4f175dc837..f37e4beffe03 100644 --- a/arch/riscv/include/uapi/asm/sigcontext.h +++ b/arch/riscv/include/uapi/asm/sigcontext.h @@ -10,6 +10,7 @@ /* The Magic number for signal context frame header. */ #define RISCV_V_MAGIC 0x53465457 +#define RISCV_ZICFISS_MAGIC 0x9487 #define END_MAGIC 0x0 /* The size of END signal context header. */ diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c index a5e3d54fe54b..1bcda11e0680 100644 --- a/arch/riscv/kernel/signal.c +++ b/arch/riscv/kernel/signal.c @@ -22,11 +22,13 @@ #include #include #include +#include unsigned long signal_minsigstksz __ro_after_init; extern u32 __user_rt_sigreturn[2]; static size_t riscv_v_sc_size __ro_after_init; +static size_t riscv_zicfiss_sc_size __ro_after_init; #define DEBUG_SIG 0 @@ -140,6 +142,62 @@ static long __restore_v_state(struct pt_regs *regs, void __user *sc_vec) return copy_from_user(current->thread.vstate.datap, datap, riscv_v_vsize); } +static long save_cfiss_state(struct pt_regs *regs, void __user *sc_cfi) +{ + struct __sc_riscv_cfi_state __user *state = sc_cfi; + unsigned long ss_ptr = 0; + long err = 0; + + if (!is_shstk_enabled(current)) + return 0; + + /* + * Save a pointer to shadow stack itself on shadow stack as a form of token. + * A token on shadow gives following properties + * - Safe save and restore for shadow stack switching. Any save of shadow stack + * must have had saved a token on shadow stack. Similarly any restore of shadow + * stack must check the token before restore. Since writing to shadow stack with + * address of shadow stack itself is not easily allowed. A restore without a save + * is quite difficult for an attacker to perform. + * - A natural break. A token in shadow stack provides a natural break in shadow stack + * So a single linear range can be bucketed into different shadow stack segments. Any + * sspopchk will detect the condition and fault to kernel as sw check exception. + */ + err |= save_user_shstk(current, &ss_ptr); + err |= __put_user(ss_ptr, &state->ss_ptr); + if (unlikely(err)) + return -EFAULT; + + return riscv_zicfiss_sc_size; +} + +static long __restore_cfiss_state(struct pt_regs *regs, void __user *sc_cfi) +{ + struct __sc_riscv_cfi_state __user *state = sc_cfi; + unsigned long ss_ptr = 0; + long err; + + /* + * Restore shadow stack as a form of token stored on shadow stack itself as a safe + * way to restore. + * A token on shadow gives following properties + * - Safe save and restore for shadow stack switching. Any save of shadow stack + * must have had saved a token on shadow stack. Similarly any restore of shadow + * stack must check the token before restore. Since writing to shadow stack with + * address of shadow stack itself is not easily allowed. A restore without a save + * is quite difficult for an attacker to perform. + * - A natural break. A token in shadow stack provides a natural break in shadow stack + * So a single linear range can be bucketed into different shadow stack segments. + * sspopchk will detect the condition and fault to kernel as sw check exception. + */ + err = __copy_from_user(&ss_ptr, &state->ss_ptr, sizeof(unsigned long)); + + if (unlikely(err)) + return err; + + return restore_user_shstk(current, ss_ptr); +} + struct arch_ext_priv { __u32 magic; long (*save)(struct pt_regs *regs, void __user *sc_vec); @@ -150,6 +208,10 @@ struct arch_ext_priv arch_ext_list[] = { .magic = RISCV_V_MAGIC, .save = &save_v_state, }, + { + .magic = RISCV_ZICFISS_MAGIC, + .save = &save_cfiss_state, + }, }; const size_t nr_arch_exts = ARRAY_SIZE(arch_ext_list); @@ -202,6 +264,12 @@ static long restore_sigcontext(struct pt_regs *regs, err = __restore_v_state(regs, sc_ext_ptr); break; + case RISCV_ZICFISS_MAGIC: + if (!is_shstk_enabled(current) || size != riscv_zicfiss_sc_size) + return -EINVAL; + + err = __restore_cfiss_state(regs, sc_ext_ptr); + break; default: return -EINVAL; } @@ -223,6 +291,16 @@ static size_t get_rt_frame_size(bool cal_all) total_context_size += riscv_v_sc_size; } + if (is_shstk_enabled(current)) + total_context_size += riscv_zicfiss_sc_size; + + /* + * Preserved a __riscv_ctx_hdr for END signal context header if an + * extension uses __riscv_extra_ext_header + */ + if (total_context_size) + total_context_size += sizeof(struct __riscv_ctx_hdr); + frame_size += total_context_size; frame_size = round_up(frame_size, 16); @@ -359,6 +437,11 @@ static int setup_rt_frame(struct ksignal *ksig, sigset_t *set, #ifdef CONFIG_MMU regs->ra = (unsigned long)VDSO_SYMBOL( current->mm->context.vdso, rt_sigreturn); + + /* if bcfi is enabled x1 (ra) and x5 (t0) must match. not sure if we need this? */ + if (is_shstk_enabled(current)) + regs->t0 = regs->ra; + #else /* * For the nommu case we don't have a VDSO. Instead we push two @@ -487,6 +570,9 @@ void __init init_rt_signal_env(void) { riscv_v_sc_size = sizeof(struct __riscv_ctx_hdr) + sizeof(struct __sc_riscv_v_state) + riscv_v_vsize; + + riscv_zicfiss_sc_size = sizeof(struct __riscv_ctx_hdr) + + sizeof(struct __sc_riscv_cfi_state); /* * Determine the stack space required for guaranteed signal delivery. * The signal_minsigstksz will be populated into the AT_MINSIGSTKSZ entry diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c index 2ebe789caa6b..8bc3e1e3f712 100644 --- a/arch/riscv/kernel/usercfi.c +++ b/arch/riscv/kernel/usercfi.c @@ -52,6 +52,11 @@ void set_active_shstk(struct task_struct *task, unsigned long shstk_addr) task->thread_info.user_cfi_state.user_shdw_stk = shstk_addr; } +unsigned long get_active_shstk(struct task_struct *task) +{ + return task->thread_info.user_cfi_state.user_shdw_stk; +} + void set_shstk_status(struct task_struct *task, bool enable) { if (!cpu_supports_shadow_stack()) @@ -169,6 +174,57 @@ static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) return 0; } +/* + * Save user shadow stack pointer on shadow stack itself and return pointer to saved location + * returns -EFAULT if operation was unsuccessful + */ +int save_user_shstk(struct task_struct *tsk, unsigned long *saved_shstk_ptr) +{ + unsigned long ss_ptr = 0; + unsigned long token_loc = 0; + int ret = 0; + + if (saved_shstk_ptr == NULL) + return -EINVAL; + + ss_ptr = get_active_shstk(tsk); + ret = create_rstor_token(ss_ptr, &token_loc); + + if (!ret) { + *saved_shstk_ptr = token_loc; + set_active_shstk(tsk, token_loc); + } + + return ret; +} + +/* + * Restores user shadow stack pointer from token on shadow stack for task `tsk` + * returns -EFAULT if operation was unsuccessful + */ +int restore_user_shstk(struct task_struct *tsk, unsigned long shstk_ptr) +{ + unsigned long token = 0; + + token = amo_user_shstk((unsigned long __user *)shstk_ptr, 0); + + if (token == -1) + return -EFAULT; + + /* invalid token, return EINVAL */ + if ((token - shstk_ptr) != SHSTK_ENTRY_SIZE) { + pr_info_ratelimited( + "%s[%d]: bad restore token in %s: pc=%p sp=%p, token=%p, shstk_ptr=%p\n", + tsk->comm, task_pid_nr(tsk), __func__, (void *)(task_pt_regs(tsk)->epc), + (void *)(task_pt_regs(tsk)->sp), (void *)token, (void *)shstk_ptr); + return -EINVAL; + } + + /* all checks passed, set active shstk and return success */ + set_active_shstk(tsk, token); + return 0; +} + static unsigned long allocate_shadow_stack(unsigned long addr, unsigned long size, unsigned long token_offset, bool set_tok) { From patchwork Wed Apr 30 00:16:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886828 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 989B627A44C for ; Wed, 30 Apr 2025 00:17:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972249; cv=none; b=pc6a4NH590oxxUBvkuvZ8HTVXnkiv/P/Xpc7RA70Kr5/WtLzcrkv2qXT8pX+swpHXgCfBTBmRnWvXvb12wgSdQnVfAUuZiv0QaN6HMCXRCpQLHeMCtnS/MCCz/4+cgRusGuPX3ZyaxwnIfS6papyn89eVEr5hYs1vYb9d5pU5Gw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972249; c=relaxed/simple; bh=DD8AZK8ySfZe7gMGHOO9A42rFShjR5wdUyvOAGK6saQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HcqbezGWxX233TIFPXnaxRkOVVIiMKJ2f8lcn4dwOD5LIs6P13tT8Cp2c3JXTyjCM3hYBhx7gBnBemkKs90dUduj6hY9qFRPNpoDhW6AAsPWYVbZkNu5PaKQsaOva84OqGHAO88aMc3qg9mo/qfHpbuPJLZIFQSed+k8CDGShaY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=EoXsCh2A; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="EoXsCh2A" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-301302a328bso9183433a91.2 for ; Tue, 29 Apr 2025 17:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972247; x=1746577047; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Ju4eTa3aeO/W78J21C+hNLwsAKoKH1MKseai5GOrm10=; b=EoXsCh2AMsaw9/Q2GwBcEb9iY+JTTZR4ERp9l1NOhfu2rGG3MYSogNffq4U7o0De1B WMfcMRkCaC1MiuUjACEvII0W1xBaqcb319ax5zHNgGNMAle2xrA5v5bjFBe1UPToS0sf cxJQzG/g6WD6wakvoEpe/H7EcZhasldBU9znQdA3L83oe9DmCBymQ0Xpai4w1LBdK5Yy 5J0KN/Y3SLuYVbkbpFOT6xjkjYWkq7Shklv11Mlz1Is/1mOR+RNUZoxxcd9pDcOna+/k ZpaokiMPjvSDJwRqz/o4mWX61wd1Rv9m0pLR2SUs8zs18WsnIh5ckibbIl1rCBYwVpHg HBrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972247; x=1746577047; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ju4eTa3aeO/W78J21C+hNLwsAKoKH1MKseai5GOrm10=; b=gLcikXTj9alxfpF6++inEBThG4h+9hQAZflEvWDbGMBsSYQQ+xbm8F1/3cYmmjs/kF y7lbMxtHsE2kXcw6esaBstgxPCZdHAsZS9j9fXFNjmBudkZ3Yj+v99nYF4qzTtTsPrgm dQ63ivE3+ijHCSP3qiRKtxOzjUXqrBspal0DzG69QtVDVjpGREME7NG7EMr1JXT7V7E4 FGC2plKmIHes+OTzY5lXUHxEzLnEQzp5H0bdEHpQK/vhsdUTlyMznKs93PgDFPJYaDkh fAboFfCgm8kOewopvI6MzlhMASutl2IEqUfl9a3w6n/GESCLCzZjBXQ36FJKU1fQAvTv 8A/g== X-Forwarded-Encrypted: i=1; AJvYcCWpjAzYF6EVrmclFwGRDHz4mJk1i7y4oAdJ/oWQnPswL3w305H+ZBaDPmOXSJFRXvnKu0tvY3mDRcaOMEgXPyE=@vger.kernel.org X-Gm-Message-State: AOJu0YyjhoNM53xor/qGvqk3esHI+cBpF0pt+FXPJujNbhf3aS7zoOXV oOthw8dq7uPEsN4X7u+zTZ8Mk4gNMsWX6vo5v5bq5O4andAimA9gUIyE0Q3vUX4= X-Gm-Gg: ASbGncuxn/4JLbflijDCD1sOslFjGKVMrtnLD5W1wPRJinzCpyGUQA1khaz9VE+tI3w YV0wWJ6XeGPsBp4jXN1+8Tg37UL/RIENgQzbZySipwVrTLtKsRviEjQ6Qh7CtY2BNcth8DxOTEq 1A7z9HUS9gEX5RQsm2MQ2AXE8uF7oujoSVNbQvuuh70zhOkxYNaylrnbHveVpBpu5NCHNY9JIhl PhhYwU74Sbb2viqapSavItiibyMkoigpLBduJt6lNvFF+wKQNQKaxboVB+FNogBwDbCPmF7xHkp hwUGEtaytsl2g4uRl17wPc/PQWs+kUNxxQSUslj8crypwpdhCpc= X-Google-Smtp-Source: AGHT+IHH5NBSIb8w2pDY2b1vP5J0WH8G+fGjXta1eAYEaxn/Svks1PLLA3C2BoMoZknQ17FEY5W4rQ== X-Received: by 2002:a17:90b:4fc2:b0:2fe:a545:4c85 with SMTP id 98e67ed59e1d1-30a33354fc9mr1556226a91.27.1745972246632; Tue, 29 Apr 2025 17:17:26 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:26 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:36 -0700 Subject: [PATCH v14 19/27] riscv/ptrace: riscv cfi status and state via ptrace and in core files Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-19-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Deepak Gupta X-Mailer: b4 0.13.0 Expose a new register type NT_RISCV_USER_CFI for risc-v cfi status and state. Intentionally both landing pad and shadow stack status and state are rolled into cfi state. Creating two different NT_RISCV_USER_XXX would not be useful and wastage of a note type. Enabling, disabling and locking of feature is not allowed via ptrace set interface. However setting `elp` state or setting shadow stack pointer are allowed via ptrace set interface . It is expected `gdb` might have use to fixup `elp` state or `shadow stack` pointer. Signed-off-by: Deepak Gupta --- arch/riscv/include/uapi/asm/ptrace.h | 30 ++++++++++++ arch/riscv/kernel/ptrace.c | 95 ++++++++++++++++++++++++++++++++++++ include/uapi/linux/elf.h | 2 + 3 files changed, 127 insertions(+) diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 659ea3af5680..42c3fc8bd513 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -131,6 +131,36 @@ struct __sc_riscv_cfi_state { unsigned long ss_ptr; /* shadow stack pointer */ }; +#define PTRACE_CFI_LP_EN_BIT 0 +#define PTRACE_CFI_LP_LOCK_BIT 1 +#define PTRACE_CFI_ELP_BIT 2 +#define PTRACE_CFI_SS_EN_BIT 3 +#define PTRACE_CFI_SS_LOCK_BIT 4 +#define PTRACE_CFI_SS_PTR_BIT 5 + +#define PTRACE_CFI_LP_EN_STATE (1 << PTRACE_CFI_LP_EN_BIT) +#define PTRACE_CFI_LP_LOCK_STATE (1 << PTRACE_CFI_LP_LOCK_BIT) +#define PTRACE_CFI_ELP_STATE (1 << PTRACE_CFI_ELP_BIT) +#define PTRACE_CFI_SS_EN_STATE (1 << PTRACE_CFI_SS_EN_BIT) +#define PTRACE_CFI_SS_LOCK_STATE (1 << PTRACE_CFI_SS_LOCK_BIT) +#define PTRACE_CFI_SS_PTR_STATE (1 << PTRACE_CFI_SS_PTR_BIT) + +#define PRACE_CFI_STATE_INVALID_MASK ~(PTRACE_CFI_LP_EN_STATE | \ + PTRACE_CFI_LP_LOCK_STATE | \ + PTRACE_CFI_ELP_STATE | \ + PTRACE_CFI_SS_EN_STATE | \ + PTRACE_CFI_SS_LOCK_STATE | \ + PTRACE_CFI_SS_PTR_STATE) + +struct __cfi_status { + __u64 cfi_state; +}; + +struct user_cfi_state { + struct __cfi_status cfi_status; + __u64 shstk_ptr; +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c index ea67e9fb7a58..933a3d26d33c 100644 --- a/arch/riscv/kernel/ptrace.c +++ b/arch/riscv/kernel/ptrace.c @@ -19,6 +19,7 @@ #include #include #include +#include enum riscv_regset { REGSET_X, @@ -31,6 +32,9 @@ enum riscv_regset { #ifdef CONFIG_RISCV_ISA_SUPM REGSET_TAGGED_ADDR_CTRL, #endif +#ifdef CONFIG_RISCV_USER_CFI + REGSET_CFI, +#endif }; static int riscv_gpr_get(struct task_struct *target, @@ -184,6 +188,87 @@ static int tagged_addr_ctrl_set(struct task_struct *target, } #endif +#ifdef CONFIG_RISCV_USER_CFI +static int riscv_cfi_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct user_cfi_state user_cfi; + struct pt_regs *regs; + + memset(&user_cfi, 0, sizeof(user_cfi)); + regs = task_pt_regs(target); + + if (is_indir_lp_enabled(target)) { + user_cfi.cfi_status.cfi_state |= PTRACE_CFI_LP_EN_STATE; + user_cfi.cfi_status.cfi_state |= is_indir_lp_locked(target) ? + PTRACE_CFI_LP_LOCK_STATE : 0; + user_cfi.cfi_status.cfi_state |= (regs->status & SR_ELP) ? + PTRACE_CFI_ELP_STATE : 0; + } + + if (is_shstk_enabled(target)) { + user_cfi.cfi_status.cfi_state |= (PTRACE_CFI_SS_EN_STATE | + PTRACE_CFI_SS_PTR_STATE); + user_cfi.cfi_status.cfi_state |= is_shstk_locked(target) ? + PTRACE_CFI_SS_LOCK_STATE : 0; + user_cfi.shstk_ptr = get_active_shstk(target); + } + + return membuf_write(&to, &user_cfi, sizeof(user_cfi)); +} + +/* + * Does it make sense to allowing enable / disable of cfi via ptrace? + * Not allowing enable / disable / locking control via ptrace for now. + * Setting shadow stack pointer is allowed. GDB might use it to unwind or + * some other fixup. Similarly gdb might want to suppress elp and may want + * to reset elp state. + */ +static int riscv_cfi_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + int ret; + struct user_cfi_state user_cfi; + struct pt_regs *regs; + + regs = task_pt_regs(target); + + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &user_cfi, 0, -1); + if (ret) + return ret; + + /* + * Not allowing enabling or locking shadow stack or landing pad + * There is no disabling of shadow stack or landing pad via ptrace + * rsvd field should be set to zero so that if those fields are needed in future + */ + if ((user_cfi.cfi_status.cfi_state & + (PTRACE_CFI_LP_EN_STATE | PTRACE_CFI_LP_LOCK_STATE | + PTRACE_CFI_SS_EN_STATE | PTRACE_CFI_SS_LOCK_STATE)) || + (user_cfi.cfi_status.cfi_state & PRACE_CFI_STATE_INVALID_MASK)) + return -EINVAL; + + /* If lpad is enabled on target and ptrace requests to set / clear elp, do that */ + if (is_indir_lp_enabled(target)) { + if (user_cfi.cfi_status.cfi_state & + PTRACE_CFI_ELP_STATE) /* set elp state */ + regs->status |= SR_ELP; + else + regs->status &= ~SR_ELP; /* clear elp state */ + } + + /* If shadow stack enabled on target, set new shadow stack pointer */ + if (is_shstk_enabled(target) && + (user_cfi.cfi_status.cfi_state & PTRACE_CFI_SS_PTR_STATE)) + set_active_shstk(target, user_cfi.shstk_ptr); + + return 0; +} +#endif + static const struct user_regset riscv_user_regset[] = { [REGSET_X] = { .core_note_type = NT_PRSTATUS, @@ -224,6 +309,16 @@ static const struct user_regset riscv_user_regset[] = { .set = tagged_addr_ctrl_set, }, #endif +#ifdef CONFIG_RISCV_USER_CFI + [REGSET_CFI] = { + .core_note_type = NT_RISCV_USER_CFI, + .align = sizeof(__u64), + .n = sizeof(struct user_cfi_state) / sizeof(__u64), + .size = sizeof(__u64), + .regset_get = riscv_cfi_get, + .set = riscv_cfi_set, + }, +#endif }; static const struct user_regset_view riscv_user_native_view = { diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h index 819ded2d39de..ee30dcd80901 100644 --- a/include/uapi/linux/elf.h +++ b/include/uapi/linux/elf.h @@ -545,6 +545,8 @@ typedef struct elf64_shdr { #define NT_RISCV_VECTOR 0x901 /* RISC-V vector registers */ #define NN_RISCV_TAGGED_ADDR_CTRL "LINUX" #define NT_RISCV_TAGGED_ADDR_CTRL 0x902 /* RISC-V tagged address control (prctl()) */ +#define NN_RISCV_USER_CFI "LINUX" +#define NT_RISCV_USER_CFI 0x903 /* RISC-V shadow stack state */ #define NN_LOONGARCH_CPUCFG "LINUX" #define NT_LOONGARCH_CPUCFG 0xa00 /* LoongArch CPU config registers */ #define NN_LOONGARCH_CSR "LINUX" From patchwork Wed Apr 30 00:16:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886827 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D85927E7F4 for ; Wed, 30 Apr 2025 00:17:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972255; cv=none; b=UYufl6v8PGe8CdXW71YLLcZgAZyHpbGbgU4dvh3m5VmwGYaODY+jJf4iMqgUxuCl7+v+Gba2WQOOmeq1aMW4wJdfXyS/u6MX1qvvvbetxsNliHSmMlYO3UpZcVSr2zfx3dJrDqy8ABOaErbm8xUy7ZFWXUyRtJ6KOL/3A0ps17A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972255; c=relaxed/simple; bh=LEwg+PEEXgk17kSZjUwQnMqL3CegoSsO5m1ICVe2uCA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=drceEpeQBVsPDGSuhU3Jn36u9DV4dJlu7hrnfRg3tiCa5isYH7E4+dRrUiGRGTmM0T8PzXW/kbwYL8CrJ/3JEbvxST2u0MD8yIaeh6PBGsKGW5Qn0uLQfrTn7i4feChqDElbRS3n6xU6ZpTd1TJn0JWO0RXcvKvNXJLuCgWa5zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=xEXtSJ/K; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="xEXtSJ/K" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-22423adf751so67172915ad.2 for ; Tue, 29 Apr 2025 17:17:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972253; x=1746577053; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=aI596RAmm7Lv4XL+SxRun/ZHSsQZoWKcT7qQq9RSlTc=; b=xEXtSJ/KhdFcEzrjOeIcCwrbVfY8P4U3Ah2y97fCgArnnILdQR9oJhyCQy029jipHQ /Q2sR47ierNCFPJKSHsx/feXK+6uzqzVigXsTHEDiyqcrEI5/va8Lqvo45qY6/FJRiwr vIgrOUgCBLdBSkMwCZvOoIkVII9COu8c+IiOjTf6BMMXteS9oIhWvDJon73sExorjkQr neY0PoH3TGgWzlsQ07HD4d3BENp15DENbNyiZEHu9BZ7xqbiMU5kSlKf6HhVhJXmCClJ w3G8q8rcOtonW+FvaFL4phFRMovX99CFvWCVdtffnQJoxKzmT0uYX/hA8oyTQrx+tghD ETbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972253; x=1746577053; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aI596RAmm7Lv4XL+SxRun/ZHSsQZoWKcT7qQq9RSlTc=; b=JtJZe2WTo9txcPhBA747UK73lJkOkujzqI6MKrJ/DJ7KxEtHsZ7+dLcX0n/4/J0Z5p nxmrQ2xC6Htigw0OJVJ46ysGQzLNCsHrHvfHataeEHzJ+YCE+H1r8YceMpEjkG2og+ur +k0UafQygZESvkOeTIiJz+MzjNVhSn6DlSF+er5k7zyPYh2MF2ZM7uiM+LPLSDEIIUZg GOs79BjnHyJy8rr6TIvTqLt390/aYDrirQT1aZeyqTkO99vgyFAf9b1g3gGTGYwuR2n4 4JRXtsh/ej0329dI5sKd1ib1oAhJXIjhuA8tvq3z3l/7N+zIJW/+VdlCLsPODtgfmhUs yyYw== X-Forwarded-Encrypted: i=1; AJvYcCUw7S/Tl/iqKV77fNfgHBF0sgAwXqbw37482eVV2ewk40vrdVigQYRersoByXYnesom7Q17C4ZxGoIeZwTEliM=@vger.kernel.org X-Gm-Message-State: AOJu0YxNheVKA1tTtdbpF/QIStEriN2U0orK+e1A/pAprXWNhtdi7evc 9kA+aFox2T5mKdtDHrZLQTVLuzlwdAfFhvlCrEts7ezjbb7GyDhYlGnZ2IgxzVY= X-Gm-Gg: ASbGncva4QIV3kUBI4Pok91Zt/bcdEbY9GeYBVKl7o6X5LvN9sSPJcPn72lCx9SnwpX 0m4mj6QJcbQBTnI+t1LsqPvUVnfij2D72aJXk39r8eniKWtrv7gIIlCgDHmxTGMRToGrWDNdcIM 2Oh1FIanmCBNGHmR+g+1o2m8wXhQ5u7Vtu5cOudx94r4n2mivWO4M+HjNBif6d4p06tidPyYiiu Gumq8dvp6nCl587fiFvA8VlPKtlldeQlauJD52ftTn6S2OV0lpZpHFZSVQNP9H7w1Wg9TXLRvrO /CHb+B4gvDwfpJjHoZr3I+KiebSc59nBvqjiwMljFH1W/xxENE41k3UeLq0eYg== X-Google-Smtp-Source: AGHT+IE1ImDWArA4vAOMZ3tPvYABZbcFyPq43rxtUjG5ptU/GyltCdFbvr8IQ/OkfIGhaCQBtfBIRg== X-Received: by 2002:a17:902:d2cb:b0:224:fa0:36d2 with SMTP id d9443c01a7336-22df350917emr17573875ad.26.1745972252926; Tue, 29 Apr 2025 17:17:32 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:32 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:38 -0700 Subject: [PATCH v14 21/27] riscv: kernel command line option to opt out of user cfi Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-21-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Deepak Gupta X-Mailer: b4 0.13.0 This commit adds a kernel command line option using which user cfi can be disabled. User backward cfi and forward cfi can be enabled independently. Kernel command line parameter "riscv_nousercfi" can take below values: - "all" : Disable forward and backward cfi both. - "bcfi" : Disable backward cfi. - "fcfi" : Disable forward cfi Signed-off-by: Deepak Gupta --- Documentation/admin-guide/kernel-parameters.txt | 8 ++++ arch/riscv/include/asm/usercfi.h | 7 +++ arch/riscv/kernel/usercfi.c | 59 ++++++++++++++++++++----- 3 files changed, 63 insertions(+), 11 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 76e538c77e31..f75d50420a56 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -6237,6 +6237,14 @@ replacement properties are not found. See the Kconfig entry for RISCV_ISA_FALLBACK. + riscv_nousercfi= + all Disable user cfi ABI to userspace even if cpu extension + are available. + bcfi Disable user backward cfi ABI to userspace even if + shadow stack extension is available. + fcfi Disable user forward cfi ABI to userspace even if landing + pad extension is available. + ro [KNL] Mount root device read-only on boot rodata= [KNL,EARLY] diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h index 68da5b7b79fd..6867ba6bd5a5 100644 --- a/arch/riscv/include/asm/usercfi.h +++ b/arch/riscv/include/asm/usercfi.h @@ -5,6 +5,10 @@ #ifndef _ASM_RISCV_USERCFI_H #define _ASM_RISCV_USERCFI_H +#define CMDLINE_DISABLE_RISCV_USERCFI_FCFI 1 +#define CMDLINE_DISABLE_RISCV_USERCFI_BCFI 2 +#define CMDLINE_DISABLE_RISCV_USERCFI 3 + #ifndef __ASSEMBLY__ #include #include @@ -83,6 +87,9 @@ void set_indir_lp_lock(struct task_struct *task); #endif /* CONFIG_RISCV_USER_CFI */ +bool is_user_shstk_enabled(void); +bool is_user_lpad_enabled(void); + #endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_USERCFI_H */ diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c index 8bc3e1e3f712..5ef357f43ad7 100644 --- a/arch/riscv/kernel/usercfi.c +++ b/arch/riscv/kernel/usercfi.c @@ -17,6 +17,8 @@ #include #include +unsigned int riscv_nousercfi; + #define SHSTK_ENTRY_SIZE sizeof(void *) bool is_shstk_enabled(struct task_struct *task) @@ -59,7 +61,7 @@ unsigned long get_active_shstk(struct task_struct *task) void set_shstk_status(struct task_struct *task, bool enable) { - if (!cpu_supports_shadow_stack()) + if (!is_user_shstk_enabled()) return; task->thread_info.user_cfi_state.ubcfi_en = enable ? 1 : 0; @@ -89,7 +91,7 @@ bool is_indir_lp_locked(struct task_struct *task) void set_indir_lp_status(struct task_struct *task, bool enable) { - if (!cpu_supports_indirect_br_lp_instr()) + if (!is_user_lpad_enabled()) return; task->thread_info.user_cfi_state.ufcfi_en = enable ? 1 : 0; @@ -259,7 +261,7 @@ SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsi bool set_tok = flags & SHADOW_STACK_SET_TOKEN; unsigned long aligned_size = 0; - if (!cpu_supports_shadow_stack()) + if (!is_user_shstk_enabled()) return -EOPNOTSUPP; /* Anything other than set token should result in invalid param */ @@ -306,7 +308,7 @@ unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long addr, size; /* If shadow stack is not supported, return 0 */ - if (!cpu_supports_shadow_stack()) + if (!is_user_shstk_enabled()) return 0; /* @@ -352,7 +354,7 @@ void shstk_release(struct task_struct *tsk) { unsigned long base = 0, size = 0; /* If shadow stack is not supported or not enabled, nothing to release */ - if (!cpu_supports_shadow_stack() || !is_shstk_enabled(tsk)) + if (!is_user_shstk_enabled() || !is_shstk_enabled(tsk)) return; /* @@ -381,7 +383,7 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st { unsigned long bcfi_status = 0; - if (!cpu_supports_shadow_stack()) + if (!is_user_shstk_enabled()) return -EINVAL; /* this means shadow stack is enabled on the task */ @@ -395,7 +397,7 @@ int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status) unsigned long size = 0, addr = 0; bool enable_shstk = false; - if (!cpu_supports_shadow_stack()) + if (!is_user_shstk_enabled()) return -EINVAL; /* Reject unknown flags */ @@ -448,7 +450,7 @@ int arch_lock_shadow_stack_status(struct task_struct *task, unsigned long arg) { /* If shtstk not supported or not enabled on task, nothing to lock here */ - if (!cpu_supports_shadow_stack() || + if (!is_user_shstk_enabled() || !is_shstk_enabled(task) || arg != 0) return -EINVAL; @@ -461,7 +463,7 @@ int arch_get_indir_br_lp_status(struct task_struct *t, unsigned long __user *sta { unsigned long fcfi_status = 0; - if (!cpu_supports_indirect_br_lp_instr()) + if (!is_user_lpad_enabled()) return -EINVAL; /* indirect branch tracking is enabled on the task or not */ @@ -474,7 +476,7 @@ int arch_set_indir_br_lp_status(struct task_struct *t, unsigned long status) { bool enable_indir_lp = false; - if (!cpu_supports_indirect_br_lp_instr()) + if (!is_user_lpad_enabled()) return -EINVAL; /* indirect branch tracking is locked and further can't be modified by user */ @@ -498,7 +500,7 @@ int arch_lock_indir_br_lp_status(struct task_struct *task, * If indirect branch tracking is not supported or not enabled on task, * nothing to lock here */ - if (!cpu_supports_indirect_br_lp_instr() || + if (!is_user_lpad_enabled() || !is_indir_lp_enabled(task) || arg != 0) return -EINVAL; @@ -506,3 +508,38 @@ int arch_lock_indir_br_lp_status(struct task_struct *task, return 0; } + +bool is_user_shstk_enabled(void) +{ + return (cpu_supports_shadow_stack() && + !(riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_BCFI)); +} + +bool is_user_lpad_enabled(void) +{ + return (cpu_supports_indirect_br_lp_instr() && + !(riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_FCFI)); +} + +static int __init setup_global_riscv_enable(char *str) +{ + if (strcmp(str, "all") == 0) + riscv_nousercfi = CMDLINE_DISABLE_RISCV_USERCFI; + + if (strcmp(str, "fcfi") == 0) + riscv_nousercfi |= CMDLINE_DISABLE_RISCV_USERCFI_FCFI; + + if (strcmp(str, "bcfi") == 0) + riscv_nousercfi |= CMDLINE_DISABLE_RISCV_USERCFI_BCFI; + + if (riscv_nousercfi) + pr_info("riscv user cfi disabled via cmdline" + "shadow stack status : %s, landing pad status : %s\n", + (riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_BCFI) ? "disabled" : + "enabled", (riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_FCFI) ? + "disabled" : "enabled"); + + return 1; +} + +__setup("riscv_nousercfi=", setup_global_riscv_enable); From patchwork Wed Apr 30 00:16:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886826 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D6A2283FEB for ; Wed, 30 Apr 2025 00:17:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972263; cv=none; b=Tz5Mxg3l0F7ZAwHyxDLVYMomUCKLYaYk/3KrdY7rl/bCS6TepUEFDoEqfgHopMMnfDN0Iraf49CFgjMO1gMeC69Pi5Dvs4WYY11K/xLU7EF58utUkjZJnDSD+nm9qLkTXhMy9PZIcIoSktzg2/hwL2HQBIiFtQaeMIyhXl/V4gw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972263; c=relaxed/simple; bh=9Swyph29QY0aTO9lGr2kIMJSXQ3XHyQYbPKd5gqQ4tQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jgT/BuADmQBgi7XEqeeecNk3kZNILH/0A1xN8GzAaCTEewNE9SzEG0o0fn7Bl8eP3fDI8rdret32/6iigTvdynMP+EDNo/QOfyXo3oRHwfKsJPj9uAxxTq1WVOBPuXBnFCXJolGxk2OVYvKKDRbGcx8VzHv+9Vr0IU7coQbhkQQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=SxyJDKyR; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="SxyJDKyR" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-306b6ae4fb2so6094488a91.3 for ; Tue, 29 Apr 2025 17:17:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972259; x=1746577059; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=C5ZEZwH3aQrzLfVWdE9j1t2G3YTj0WprbxNkrWh/BaE=; b=SxyJDKyRYOUSHMIQntnSqQBj495aY7jFVR/tqQCF+CnU6/fQm5nqIGpHsVvxyyVxYH ewuIkTj4fAYwJDQ9Kti1/RTFE3WhyFC2u6v2TBx0AqeCCxzdd64Lt7R+VOuE6LqYolyA qjJtC/43UO8L3wMOFmpom5SHJU3L4vqo8uk2suv+njM0mT+jLSTf/DEE7nQJ/vCk1QEv 0jsZyz/1qQnNRYp0y1GeEzB/6V9QQo6e0WeIb0V7DbtM2rQHB9RL4q89p4ZqcIYoKtHm XP9YFoPr2rRigvEHJCqYPrZJVHXlpG4Vo57q0nsBvd/s/NS56Mi3toOvXwpFAhVkNBZl XWKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972259; x=1746577059; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5ZEZwH3aQrzLfVWdE9j1t2G3YTj0WprbxNkrWh/BaE=; b=K1oNFhZ7T+cSXxWNNfq/8HmDsED0udNta4Tds0uWODEouCD5X8iMJrDhi4u+FcJ9b6 xEH62ZESNvvQIuf3EtS7kRR4bd0qtvICob4S9O/R2i40LOT3C1lIQKngBf/bgW7yswq0 YnXrW+QaSZHtOLXQIE2D09+gLcrMpUjF9UXhtqFvTqR8eMqQTt1YBVUXtnou8LFIFWYi TXUvt9m2ZrHNctl5qH5D+TeVHEBoU//tbqtl3JMoYuPX2MBNeAMmz2p4mZUdesgiCrgS gXzhf5FYymU1r06AywgtAn0zDQQ6niWkBrOGuiEKiCkDn1ONvyoJoOEORkFSsFW3k2vO EahQ== X-Forwarded-Encrypted: i=1; AJvYcCXtL0QCbA19m2FdrEsSp43N9IfVJxsmkWUA4AE5ObP0zFC4Tj7K0r/23XcQ+0yMKIsMRM5MTAXz21aK+19bTLM=@vger.kernel.org X-Gm-Message-State: AOJu0Yy5iEkA0s5027Ifz4OH7R5MIgl8iy/x2EtvD1RzpSs2GYCdR39r 1hYgbrSuFI6xlYrkJHpoexkg+onpuF+QlFVlGo9OSzA0S0UCzGJnD+Qz/gm5Abs= X-Gm-Gg: ASbGnctxm8F93E6MMRgS97HmOOugNj2V7I0BIDT11dXq1khvbpnSAtlxwJmclEcuPya KZP9A3HkTqHvumIFk06/GLZBmZxD8GaOHcyoMfXUcizY3/arij9A9winBK23+MDxvhRM5wAFI0q kZbJSSUF6r1LLDCXByV44yPtNe2I/tp04vAzioy+Ue5QEZAbbFxZkkqZCR2wF0HCoCnOK/qzMjz sgyjHFhrXGTXzPZ9+g/0gcYZP5B/z2n0wCwsIR9DprLwRdQagl+sJyciocXT8KexITGIKskBQgz xnFtsW4BElYhqQn7UWCu8Y/F0XOwUHnjf3+kIXn86n1Izdhy7So= X-Google-Smtp-Source: AGHT+IFAMLIyLPT3Vyub7NbksFuJzHx0Dg3VrKlspZ+PnLCueluan8OlRjrHsN8k6+RcRQ9h1pRZQA== X-Received: by 2002:a17:90b:5824:b0:2ff:618c:a1e0 with SMTP id 98e67ed59e1d1-30a333608d8mr1421802a91.29.1745972259317; Tue, 29 Apr 2025 17:17:39 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:38 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:40 -0700 Subject: [PATCH v14 23/27] arch/riscv: compile vdso with landing pad Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-23-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 From: Jim Shu user mode tasks compiled with zicfilp may call indirectly into vdso (like hwprobe indirect calls). Add landing pad compile support in vdso. vdso with landing pad in it will be nop for tasks which have not enabled landing pad. This patch allows to run user mode tasks with cfi eanbled and do no harm. Future work can be done on this to do below - labeled landing pad on vdso functions (whenever labeling support shows up in gnu-toolchain) - emit shadow stack instructions only in vdso compiled objects as part of kernel compile. Signed-off-by: Jim Shu Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- arch/riscv/Makefile | 5 +++- arch/riscv/include/asm/assembler.h | 44 +++++++++++++++++++++++++++++++++++ arch/riscv/kernel/vdso/Makefile | 6 +++++ arch/riscv/kernel/vdso/flush_icache.S | 4 ++++ arch/riscv/kernel/vdso/getcpu.S | 4 ++++ arch/riscv/kernel/vdso/rt_sigreturn.S | 4 ++++ arch/riscv/kernel/vdso/sys_hwprobe.S | 4 ++++ 7 files changed, 70 insertions(+), 1 deletion(-) diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 539d2aef5cab..c2dd09bb9db3 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -88,9 +88,12 @@ riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZACAS) := $(riscv-march-y)_zacas # Check if the toolchain supports Zabha riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZABHA) := $(riscv-march-y)_zabha +KBUILD_BASE_ISA = -march=$(shell echo $(riscv-march-y) | sed -E 's/(rv32ima|rv64ima)fd([^v_]*)v?/\1\2/') +export KBUILD_BASE_ISA + # Remove F,D,V from isa string for all. Keep extensions between "fd" and "v" by # matching non-v and non-multi-letter extensions out with the filter ([^v_]*) -KBUILD_CFLAGS += -march=$(shell echo $(riscv-march-y) | sed -E 's/(rv32ima|rv64ima)fd([^v_]*)v?/\1\2/') +KBUILD_CFLAGS += $(KBUILD_BASE_ISA) KBUILD_AFLAGS += -march=$(riscv-march-y) diff --git a/arch/riscv/include/asm/assembler.h b/arch/riscv/include/asm/assembler.h index 44b1457d3e95..a058ea5e9c58 100644 --- a/arch/riscv/include/asm/assembler.h +++ b/arch/riscv/include/asm/assembler.h @@ -80,3 +80,47 @@ .endm #endif /* __ASM_ASSEMBLER_H */ + +#if defined(CONFIG_RISCV_USER_CFI) && (__riscv_xlen == 64) +.macro vdso_lpad +lpad 0 +.endm +#else +.macro vdso_lpad +.endm +#endif + +/* + * This macro emits a program property note section identifying + * architecture features which require special handling, mainly for + * use in assembly files included in the VDSO. + */ +#define NT_GNU_PROPERTY_TYPE_0 5 +#define GNU_PROPERTY_RISCV_FEATURE_1_AND 0xc0000000 + +#define GNU_PROPERTY_RISCV_FEATURE_1_ZICFILP (1U << 0) +#define GNU_PROPERTY_RISCV_FEATURE_1_ZICFISS (1U << 1) + +#if defined(CONFIG_RISCV_USER_CFI) && (__riscv_xlen == 64) +#define GNU_PROPERTY_RISCV_FEATURE_1_DEFAULT \ + (GNU_PROPERTY_RISCV_FEATURE_1_ZICFILP) +#endif + +#ifdef GNU_PROPERTY_RISCV_FEATURE_1_DEFAULT +.macro emit_riscv_feature_1_and, feat = GNU_PROPERTY_RISCV_FEATURE_1_DEFAULT + .pushsection .note.gnu.property, "a" + .p2align 3 + .word 4 + .word 16 + .word NT_GNU_PROPERTY_TYPE_0 + .asciz "GNU" + .word GNU_PROPERTY_RISCV_FEATURE_1_AND + .word 4 + .word \feat + .word 0 + .popsection +.endm +#else +.macro emit_riscv_feature_1_and, feat = 0 +.endm +#endif diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile index ad73607abc28..8e771d316938 100644 --- a/arch/riscv/kernel/vdso/Makefile +++ b/arch/riscv/kernel/vdso/Makefile @@ -13,12 +13,18 @@ vdso-syms += flush_icache vdso-syms += hwprobe vdso-syms += sys_hwprobe +ifdef CONFIG_RISCV_USER_CFI +LPAD_MARCH = _zicfilp +endif + # Files to link into the vdso obj-vdso = $(patsubst %, %.o, $(vdso-syms)) note.o ccflags-y := -fno-stack-protector ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -fno-builtin +ccflags-y += $(KBUILD_BASE_ISA)$(LPAD_MARCH) +asflags-y += $(KBUILD_BASE_ISA)$(LPAD_MARCH) ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday.o += -fPIC -include $(c-gettimeofday-y) diff --git a/arch/riscv/kernel/vdso/flush_icache.S b/arch/riscv/kernel/vdso/flush_icache.S index 8f884227e8bc..e4c56970905e 100644 --- a/arch/riscv/kernel/vdso/flush_icache.S +++ b/arch/riscv/kernel/vdso/flush_icache.S @@ -5,11 +5,13 @@ #include #include +#include .text /* int __vdso_flush_icache(void *start, void *end, unsigned long flags); */ SYM_FUNC_START(__vdso_flush_icache) .cfi_startproc + vdso_lpad #ifdef CONFIG_SMP li a7, __NR_riscv_flush_icache ecall @@ -20,3 +22,5 @@ SYM_FUNC_START(__vdso_flush_icache) ret .cfi_endproc SYM_FUNC_END(__vdso_flush_icache) + +emit_riscv_feature_1_and diff --git a/arch/riscv/kernel/vdso/getcpu.S b/arch/riscv/kernel/vdso/getcpu.S index 9c1bd531907f..5c1ecc4e1465 100644 --- a/arch/riscv/kernel/vdso/getcpu.S +++ b/arch/riscv/kernel/vdso/getcpu.S @@ -5,14 +5,18 @@ #include #include +#include .text /* int __vdso_getcpu(unsigned *cpu, unsigned *node, void *unused); */ SYM_FUNC_START(__vdso_getcpu) .cfi_startproc + vdso_lpad /* For now, just do the syscall. */ li a7, __NR_getcpu ecall ret .cfi_endproc SYM_FUNC_END(__vdso_getcpu) + +emit_riscv_feature_1_and diff --git a/arch/riscv/kernel/vdso/rt_sigreturn.S b/arch/riscv/kernel/vdso/rt_sigreturn.S index 3dc022aa8931..e82987dc3739 100644 --- a/arch/riscv/kernel/vdso/rt_sigreturn.S +++ b/arch/riscv/kernel/vdso/rt_sigreturn.S @@ -5,12 +5,16 @@ #include #include +#include .text SYM_FUNC_START(__vdso_rt_sigreturn) .cfi_startproc .cfi_signal_frame + vdso_lpad li a7, __NR_rt_sigreturn ecall .cfi_endproc SYM_FUNC_END(__vdso_rt_sigreturn) + +emit_riscv_feature_1_and diff --git a/arch/riscv/kernel/vdso/sys_hwprobe.S b/arch/riscv/kernel/vdso/sys_hwprobe.S index 77e57f830521..f1694451a60c 100644 --- a/arch/riscv/kernel/vdso/sys_hwprobe.S +++ b/arch/riscv/kernel/vdso/sys_hwprobe.S @@ -3,13 +3,17 @@ #include #include +#include .text SYM_FUNC_START(riscv_hwprobe) .cfi_startproc + vdso_lpad li a7, __NR_riscv_hwprobe ecall ret .cfi_endproc SYM_FUNC_END(riscv_hwprobe) + +emit_riscv_feature_1_and From patchwork Wed Apr 30 00:16:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886825 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 830FC289E12 for ; Wed, 30 Apr 2025 00:17:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972268; cv=none; b=PbXxwQdTfr5Q64805GHUFmLL1N2T1U2gESfNcfiYk7AFeXaoOKsiKpC5Ra9X3xPDYWt2CM0bNylKmpACUcGs0DAbsYaGuI57k/vHym1SQJ3N7FRYuEUxFZFdKcrVXVwHbxOj5xQcIW4OB78vjeqq1YV07Yh+8lm4FWN3UBYgEgE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972268; c=relaxed/simple; bh=rpxI7deGxR1b1p2DEnatELgrO8HyrKziLzhoGXBzZ0c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=peg+FSVTJsKFrbtmtVrSFsh/fzRpi8EojcJSkHcSUx2jILH7V/wz6Ji8tkkYvFNd2h1/U5BJeLrP/sjyeizCPyuXcOJcClmVUne++t6vljYoTkJK/FOGAMe+xxfpWJlshHzE7q02uHAfc8ZsmGfbDOT1n6hUymahAX7lzp+SFg8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=FV+euEks; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="FV+euEks" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-22d95f0dda4so96384585ad.2 for ; Tue, 29 Apr 2025 17:17:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972266; x=1746577066; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=04baGaWIOcpAM10BsvDKc2YnzV44Y+9Sv7IQZJnFcq0=; b=FV+euEksBuu9zMDYvRUYuScJkPUYLbd66fSJAOM47T45Ix1xauoZ4jyVUjW/CDjLFv iOhOvFsivP70iCg+iyEPeiMMIJrgTxulG7b5oRm5y26fBVOfJOw8L5vSjp+g63hEcJEM 3VsCmMTlmmjrLa+SEmuU12lVRQcdnR+H3BBhJyOYTPe8yAifmcXQq+xX/DcT4JVt3uWb iT+4jYxJ8Hn5bJmhSzI3XqPgGxyTJ/PQUTWSWh3SVJ3JfK1SCD3t2lqm88Zeys//cAfr 7/GFOTsjNtPAOxSt+DwvDZy96pl7F6qyTjHSk+zI23vwqxzs9j7jCGkzoXcnm7JEAhJu cayQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972266; x=1746577066; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=04baGaWIOcpAM10BsvDKc2YnzV44Y+9Sv7IQZJnFcq0=; b=k8pqvQ+evJYVIH5uWHcVOShJyz5g25qF2GPsT8hLKl2shKid6sQqmc4gel4sWQvoKo eAfm5VYR1v/g4LENcvmeuSo9eT7SCyu9pCoU2dfFJmJ0+vH7F6DJbMkfD9peQwQE9GyK BFccUE55IIm5U+Y7M9QxtA1MvTf5bm9zAyQG2mJtZ7kyByEfgXKWfgEK1OMDtEm8bYwU v74JZjZoSOe1cZQti1uNRGWFnKS+IBy8s2UDQMgz57JTYK1YTFGN0g0Kolnk4DxzCPWc fn8YpfClJPRq2C4YTIOp/ul2UgSo44gc/ihFx1/Id6v9kZbr5ZiL+1mbwaMQ+vVyi6hx Pfgg== X-Forwarded-Encrypted: i=1; AJvYcCUbya4nKTyW+i94WXdLYl4t7Ow/IXycGYHOiPS7wibBgrjkzs+4fhrNkyzigCVdFk+T3zzcKAg9iTxHe7syte0=@vger.kernel.org X-Gm-Message-State: AOJu0YzM+i6Eld8hGuwb2dUQNASbloxUCgFp/zv9vMQEqG3PEnq5BwQL yLxCa4VMQ363UGfgwqo9w5zsO+dB3RLEboVbzaMeKGiPysmuuG6iTBcTWwCWcBU= X-Gm-Gg: ASbGncvFUoWzPAE/VE6XbkAJqCUaWk1RHcNX/fHpO8QbwgOQ2xdbRVRRr2O1K7sDc8i dAxOy4AfQZYRetUA0pVAZ9KHiy1L31l9FTmCl6e7yROoU8kvvPHcbNB9wsvGi4eYd2B5F29ED9r JFNnJRkRsORcxEN/GuDLMNzvNAZDqzqBxmQ4eti57yvwzZdbG4AIVKUsCfi5pc6c6Ar7h/Gf+kF ArJuHvmdaQHaE+7K8/mw2EBWwjIjmir8SSoTWRQloteyMayBzkViZnHkNsJ56p+vy7SpfDNQg3d dZWd79XYfmzkto6IISJXqAejpymFLKU5D1C3DNOvE8oWQWygnu0= X-Google-Smtp-Source: AGHT+IElsYHuL6uNpuF98grZ32cMaP4x/B+n9NaNOM+GYOmHv0lBJAPxppghFLiR02kYnVeXaWC46g== X-Received: by 2002:a17:902:e80d:b0:224:76f:9e44 with SMTP id d9443c01a7336-22df34b848cmr16103735ad.8.1745972265738; Tue, 29 Apr 2025 17:17:45 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:45 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:42 -0700 Subject: [PATCH v14 25/27] riscv: Documentation for landing pad / indirect branch tracking Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-25-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Zong Li , Deepak Gupta X-Mailer: b4 0.13.0 Adding documentation on landing pad aka indirect branch tracking on riscv and kernel interfaces exposed so that user tasks can enable it. Reviewed-by: Zong Li Signed-off-by: Deepak Gupta --- Documentation/arch/riscv/index.rst | 1 + Documentation/arch/riscv/zicfilp.rst | 115 +++++++++++++++++++++++++++++++++++ 2 files changed, 116 insertions(+) diff --git a/Documentation/arch/riscv/index.rst b/Documentation/arch/riscv/index.rst index eecf347ce849..be7237b69682 100644 --- a/Documentation/arch/riscv/index.rst +++ b/Documentation/arch/riscv/index.rst @@ -14,6 +14,7 @@ RISC-V architecture uabi vector cmodx + zicfilp features diff --git a/Documentation/arch/riscv/zicfilp.rst b/Documentation/arch/riscv/zicfilp.rst new file mode 100644 index 000000000000..dcf43de73a06 --- /dev/null +++ b/Documentation/arch/riscv/zicfilp.rst @@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: GPL-2.0 + +:Author: Deepak Gupta +:Date: 12 January 2024 + +==================================================== +Tracking indirect control transfers on RISC-V Linux +==================================================== + +This document briefly describes the interface provided to userspace by Linux +to enable indirect branch tracking for user mode applications on RISC-V + +1. Feature Overview +-------------------- + +Memory corruption issues usually result into crashes, however when in hands of +an adversary and if used creatively can result into a variety security issues. + +One of those security issues can be code re-use attacks on program where adversary +can use corrupt function pointers and chain them together to perform jump oriented +programming (JOP) or call oriented programming (COP) and thus compromising control +flow integrity (CFI) of the program. + +Function pointers live in read-write memory and thus are susceptible to corruption +and allows an adversary to reach any program counter (PC) in address space. On +RISC-V zicfilp extension enforces a restriction on such indirect control +transfers: + +- indirect control transfers must land on a landing pad instruction ``lpad``. + There are two exception to this rule: + + - rs1 = x1 or rs1 = x5, i.e. a return from a function and returns are + protected using shadow stack (see zicfiss.rst) + + - rs1 = x7. On RISC-V compiler usually does below to reach function + which is beyond the offset possible J-type instruction:: + + auipc x7, + jalr (x7) + + Such form of indirect control transfer are still immutable and don't rely + on memory and thus rs1=x7 is exempted from tracking and considered software + guarded jumps. + +``lpad`` instruction is pseudo of ``auipc rd, `` with ``rd=x0`` and +is a HINT nop. ``lpad`` instruction must be aligned on 4 byte boundary and +compares 20 bit immediate with x7. If ``imm_20bit`` == 0, CPU doesn't perform +any comparision with ``x7``. If ``imm_20bit`` != 0, then ``imm_20bit`` must +match ``x7`` else CPU will raise ``software check exception`` (``cause=18``) +with ``*tval = 2``. + +Compiler can generate a hash over function signatures and setup them (truncated +to 20bit) in x7 at callsites and function prologues can have ``lpad`` with same +function hash. This further reduces number of program counters a call site can +reach. + +2. ELF and psABI +----------------- + +Toolchain sets up :c:macro:`GNU_PROPERTY_RISCV_FEATURE_1_FCFI` for property +:c:macro:`GNU_PROPERTY_RISCV_FEATURE_1_AND` in notes section of the object file. + +3. Linux enabling +------------------ + +User space programs can have multiple shared objects loaded in its address space +and it's a difficult task to make sure all the dependencies have been compiled +with support of indirect branch. Thus it's left to dynamic loader to enable +indirect branch tracking for the program. + +4. prctl() enabling +-------------------- + +:c:macro:`PR_SET_INDIR_BR_LP_STATUS` / :c:macro:`PR_GET_INDIR_BR_LP_STATUS` / +:c:macro:`PR_LOCK_INDIR_BR_LP_STATUS` are three prctls added to manage indirect +branch tracking. prctls are arch agnostic and returns -EINVAL on other arches. + +* prctl(PR_SET_INDIR_BR_LP_STATUS, unsigned long arg) + +If arg1 is :c:macro:`PR_INDIR_BR_LP_ENABLE` and if CPU supports ``zicfilp`` +then kernel will enable indirect branch tracking for the task. Dynamic loader +can issue this :c:macro:`prctl` once it has determined that all the objects +loaded in address space support indirect branch tracking. Additionally if there +is a `dlopen` to an object which wasn't compiled with ``zicfilp``, dynamic +loader can issue this prctl with arg1 set to 0 (i.e. +:c:macro:`PR_INDIR_BR_LP_ENABLE` being clear) + +* prctl(PR_GET_INDIR_BR_LP_STATUS, unsigned long arg) + +Returns current status of indirect branch tracking. If enabled it'll return +:c:macro:`PR_INDIR_BR_LP_ENABLE` + +* prctl(PR_LOCK_INDIR_BR_LP_STATUS, unsigned long arg) + +Locks current status of indirect branch tracking on the task. User space may +want to run with strict security posture and wouldn't want loading of objects +without ``zicfilp`` support in it and thus would want to disallow disabling of +indirect branch tracking. In that case user space can use this prctl to lock +current settings. + +5. violations related to indirect branch tracking +-------------------------------------------------- + +Pertaining to indirect branch tracking, CPU raises software check exception in +following conditions: + +- missing ``lpad`` after indirect call / jmp +- ``lpad`` not on 4 byte boundary +- ``imm_20bit`` embedded in ``lpad`` instruction doesn't match with ``x7`` + +In all 3 cases, ``*tval = 2`` is captured and software check exception is +raised (``cause=18``) + +Linux kernel will treat this as :c:macro:`SIGSEV`` with code = +:c:macro:`SEGV_CPERR` and follow normal course of signal delivery. From patchwork Wed Apr 30 00:16:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 886824 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2666D29375C for ; Wed, 30 Apr 2025 00:17:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972275; cv=none; b=DdhEgoBCtKhF3/RIxlCV4sW3c+gNOXbFRXmZLI/dJmubNQiqkC8Suus1GkuoSAQvNMBfGpTKRq7XeGJXn/IdDbH5+luB/fP8+S3NuG9QTQ39h5tmTHawh+6nudAnyb8lrgg32TuxiqmobUaSqfiHxSTgiLFr64Z1nWZne+dZqls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745972275; c=relaxed/simple; bh=azX4l+9eUPXXiA2MWmjdsBQzljDuEVuBqF2RFHU+8z4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oytJfgOdBwJbi0JJOn9aNPOASLOuGXkb8oNt/wt4FuGiEeP7nCxEXxRqzBZQ/yrRcCMHoTNoqxRSxA/Q7lu+meEo7/zwc0Y+hm3TPyH7gAAdMc5S9sdE0HQU19+YmTpw2Lo8XVIGirpuiQ6jyvjaxoMirXLXNA1FMOOsdInwJdY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=wHxY51zr; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="wHxY51zr" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2295d78b45cso94620695ad.0 for ; Tue, 29 Apr 2025 17:17:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1745972272; x=1746577072; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=1m7Q8DqiKaz+P5fbk0v699raT+/1ibH9Sjx2HpamE1Q=; b=wHxY51zr9IeYUrDKl+wI+WxPOI7IL9gTKvBuWJt592PyOUMbS5cSFGfUw4PmdaCkOy 5m7kyXdV1l1iqp1/F/2zUNymzcnqi9BZ/+wGdpqGAbXuQHRP/pbl+NvOGxM728LX7HSr fl/mTwUzLrxi0WWeYScPnyekZDtpgK4RjkKVKX++YxZe47uwz3yOHujNlidFl3RluwwM XFsqknsEflyZiaNSZ/6yq9X4IDBplIk0nnarsvzT+Pslv956lGpRWPYSBoB8EgxKmGNn HSVwiBdiB3xAyGxs94ZYhKQwk/EpjLcuMVPNlsv45T3Xf6/gwbzy/j/wPAznKsx2kI72 ws8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745972272; x=1746577072; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1m7Q8DqiKaz+P5fbk0v699raT+/1ibH9Sjx2HpamE1Q=; b=aek2hS/mBzVdtviFo7I1GdT9ILCJsbtjIO0+jQ1PoMB4Mw2M7kWIpArR8hlgVLesgV C6zo/ZW9a27gRmHhHpIVx7udSxR6r85XL99WNaOcS+FbHFw55LjWOnKqOyzLw4JwgkRS uTkhGsqh4m1430Uk4BK8f9It9LFAhD+LNzm10H8VXqQ7tI2qUr3noyltp7xmojtqICju 4uC2lnMTxWLvDdQnoTG7cCa7IOOcjsVQpn7Oy9a5tM+LXyGxaGD4GMbby+rotA/kffPp PRcL8HzfxxoXX6XH9bkDnBoJhpY2x2oG+e8hk0B7KJCxV0gC0pSvV+QMmEgyZR/T2CTp qWtg== X-Forwarded-Encrypted: i=1; AJvYcCWBp02ubfLEewM3u1wohvb3fr34G5+QbxC9Ka9zOkg8icx5fUL0B/4ByM1Mo3v0lAnuehOmM/m4YsnkBlp7vkk=@vger.kernel.org X-Gm-Message-State: AOJu0YzsXL72vWiRNVMAKjUtD3vJDSPe7PWC92GSfB+0d15EnDf8wbQy U46+qi0hAO0+Zp/k43zCWSLJoXaYMZVkCAhoK4+aG83Rz/ATROCtwDIzN4DKHgI= X-Gm-Gg: ASbGnct9GZdTk9x613qpDQxWol2716FLp+FE9+egdO2NCmQSoQ0MGLLp9xK9b/jynUw 9lR72Hx7BexAWFHGKfVGAOIOt7DLnKLahIJny0WtMgfVXWqOspQlU/ljCuxTVUapYDJluYLCWe8 znH3bcKEybPUOEFRNPB84tnGeX50nKbhwXpXoZdhmLbInhIrWUpu80zyBFJcieK7Etupk+EegzL XW1dCpN21SMK7yZSabmzr6JQck6B0n5r5O468FMJubY30+CWWo7+hXS5E/27i7uztWVqh0XVJFI Q8HhUrzU7TywuT+KbV1hD/urxxo3dqaOnxyII3CzsEZ0xxdZaIk= X-Google-Smtp-Source: AGHT+IENCFqRAcvQmAU+rr2bcAnacyefXUmgaAWXi+FQ/dx4DuH96f67lexQ60/+hxd9kG7nDpbYAg== X-Received: by 2002:a17:902:ccd1:b0:223:f639:69df with SMTP id d9443c01a7336-22df5835e57mr7152465ad.41.1745972272210; Tue, 29 Apr 2025 17:17:52 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22db4d770d6sm109386035ad.17.2025.04.29.17.17.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 17:17:51 -0700 (PDT) From: Deepak Gupta Date: Tue, 29 Apr 2025 17:16:44 -0700 Subject: [PATCH v14 27/27] kselftest/riscv: kselftest for user mode cfi Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250429-v5_user_cfi_series-v14-27-5239410d012a@rivosinc.com> References: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> In-Reply-To: <20250429-v5_user_cfi_series-v14-0-5239410d012a@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, rust-for-linux@vger.kernel.org, Deepak Gupta X-Mailer: b4 0.13.0 Adds kselftest for RISC-V control flow integrity implementation for user mode. There is not a lot going on in kernel for enabling landing pad for user mode. cfi selftest are intended to be compiled with zicfilp and zicfiss enabled compiler. Thus kselftest simply checks if landing pad / shadow stack for the process are enabled or not and executes ptrace selftests on cfi. selftest then register a signal handler for SIGSEGV. Any control flow violation are reported as SIGSEGV with si_code = SEGV_CPERR. Test will fail on receiving any SEGV_CPERR. Shadow stack part has more changes in kernel and thus there are separate tests for that - Exercise `map_shadow_stack` syscall - `fork` test to make sure COW works for shadow stack pages - gup tests Kernel uses FOLL_FORCE when access happens to memory via /proc//mem. Not breaking that for shadow stack. - signal test. Make sure signal delivery results in token creation on shadow stack and consumes (and verifies) token on sigreturn - shadow stack protection test. attempts to write using regular store instruction on shadow stack memory must result in access faults Test outut ========== """ TAP version 13 1..5 This is to ensure shadow stack is indeed enabled and working This is to ensure shadow stack is indeed enabled and working ok 1 shstk fork test ok 2 map shadow stack syscall ok 3 shadow stack gup tests ok 4 shadow stack signal tests ok 5 memory protections of shadow stack memory """ Signed-off-by: Deepak Gupta --- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/cfi/.gitignore | 3 + tools/testing/selftests/riscv/cfi/Makefile | 10 + tools/testing/selftests/riscv/cfi/cfi_rv_test.h | 82 +++++ tools/testing/selftests/riscv/cfi/riscv_cfi_test.c | 171 +++++++++ tools/testing/selftests/riscv/cfi/shadowstack.c | 385 +++++++++++++++++++++ tools/testing/selftests/riscv/cfi/shadowstack.h | 27 ++ 7 files changed, 679 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftests/riscv/Makefile index 099b8c1f46f8..5671b4405a12 100644 --- a/tools/testing/selftests/riscv/Makefile +++ b/tools/testing/selftests/riscv/Makefile @@ -5,7 +5,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),riscv)) -RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector +RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector cfi else RISCV_SUBTARGETS := endif diff --git a/tools/testing/selftests/riscv/cfi/.gitignore b/tools/testing/selftests/riscv/cfi/.gitignore new file mode 100644 index 000000000000..82545863bac6 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/.gitignore @@ -0,0 +1,3 @@ +cfitests +riscv_cfi_test +shadowstack diff --git a/tools/testing/selftests/riscv/cfi/Makefile b/tools/testing/selftests/riscv/cfi/Makefile new file mode 100644 index 000000000000..b65f7ff38a32 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/Makefile @@ -0,0 +1,10 @@ +CFLAGS += -I$(top_srcdir)/tools/include + +CFLAGS += -march=rv64gc_zicfilp_zicfiss + +TEST_GEN_PROGS := cfitests + +include ../../lib.mk + +$(OUTPUT)/cfitests: riscv_cfi_test.c shadowstack.c + $(CC) -o$@ $(CFLAGS) $(LDFLAGS) $^ diff --git a/tools/testing/selftests/riscv/cfi/cfi_rv_test.h b/tools/testing/selftests/riscv/cfi/cfi_rv_test.h new file mode 100644 index 000000000000..1c8043f2b778 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/cfi_rv_test.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef SELFTEST_RISCV_CFI_H +#define SELFTEST_RISCV_CFI_H +#include +#include +#include "shadowstack.h" + +#define CHILD_EXIT_CODE_SSWRITE 10 +#define CHILD_EXIT_CODE_SIG_TEST 11 + +#define my_syscall5(num, arg1, arg2, arg3, arg4, arg5) \ +({ \ + register long _num __asm__ ("a7") = (num); \ + register long _arg1 __asm__ ("a0") = (long)(arg1); \ + register long _arg2 __asm__ ("a1") = (long)(arg2); \ + register long _arg3 __asm__ ("a2") = (long)(arg3); \ + register long _arg4 __asm__ ("a3") = (long)(arg4); \ + register long _arg5 __asm__ ("a4") = (long)(arg5); \ + \ + __asm__ volatile( \ + "ecall\n" \ + : "+r" \ + (_arg1) \ + : "r"(_arg2), "r"(_arg3), "r"(_arg4), "r"(_arg5), \ + "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +#define my_syscall3(num, arg1, arg2, arg3) \ +({ \ + register long _num __asm__ ("a7") = (num); \ + register long _arg1 __asm__ ("a0") = (long)(arg1); \ + register long _arg2 __asm__ ("a1") = (long)(arg2); \ + register long _arg3 __asm__ ("a2") = (long)(arg3); \ + \ + __asm__ volatile( \ + "ecall\n" \ + : "+r" (_arg1) \ + : "r"(_arg2), "r"(_arg3), \ + "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +#ifndef __NR_prctl +#define __NR_prctl 167 +#endif + +#ifndef __NR_map_shadow_stack +#define __NR_map_shadow_stack 453 +#endif + +#define CSR_SSP 0x011 + +#ifdef __ASSEMBLY__ +#define __ASM_STR(x) x +#else +#define __ASM_STR(x) #x +#endif + +#define csr_read(csr) \ +({ \ + register unsigned long __v; \ + __asm__ __volatile__ ("csrr %0, " __ASM_STR(csr) \ + : "=r" (__v) : \ + : "memory"); \ + __v; \ +}) + +#define csr_write(csr, val) \ +({ \ + unsigned long __v = (unsigned long)(val); \ + __asm__ __volatile__ ("csrw " __ASM_STR(csr) ", %0" \ + : : "rK" (__v) \ + : "memory"); \ +}) + +#endif diff --git a/tools/testing/selftests/riscv/cfi/riscv_cfi_test.c b/tools/testing/selftests/riscv/cfi/riscv_cfi_test.c new file mode 100644 index 000000000000..ac8b55ef43c6 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/riscv_cfi_test.c @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../../kselftest.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "cfi_rv_test.h" + +/* do not optimize cfi related test functions */ +#pragma GCC push_options +#pragma GCC optimize("O0") + +void sigsegv_handler(int signum, siginfo_t *si, void *uc) +{ + struct ucontext *ctx = (struct ucontext *)uc; + + if (si->si_code == SEGV_CPERR) { + ksft_print_msg("Control flow violation happened somewhere\n"); + ksft_print_msg("PC where violation happened %lx\n", ctx->uc_mcontext.gregs[0]); + exit(-1); + } + + /* all other cases are expected to be of shadow stack write case */ + exit(CHILD_EXIT_CODE_SSWRITE); +} + +bool register_signal_handler(void) +{ + struct sigaction sa = {}; + + sa.sa_sigaction = sigsegv_handler; + sa.sa_flags = SA_SIGINFO; + if (sigaction(SIGSEGV, &sa, NULL)) { + ksft_print_msg("Registering signal handler for landing pad violation failed\n"); + return false; + } + + return true; +} + +bool cfi_ptrace_test(void) +{ + pid_t pid; + int status, ret = 0; + unsigned long ptrace_test_num = 0, total_ptrace_tests = 2; + + struct user_cfi_state cfi_reg; + struct iovec iov; + + pid = fork(); + + if (pid == -1) { + ksft_exit_fail_msg("%s: fork failed\n", __func__); + exit(1); + } + + if (pid == 0) { + /* allow to be traced */ + ptrace(PTRACE_TRACEME, 0, NULL, NULL); + raise(SIGSTOP); + asm volatile ( + "la a5, 1f\n" + "jalr a5 \n" + "nop \n" + "nop \n" + "1: nop\n" + : : : "a5"); + exit(11); + /* child shouldn't go beyond here */ + } + + /* parent's code goes here */ + iov.iov_base = &cfi_reg; + iov.iov_len = sizeof(cfi_reg); + + while (ptrace_test_num < total_ptrace_tests) { + memset(&cfi_reg, 0, sizeof(cfi_reg)); + waitpid(pid, &status, 0); + if (WIFSTOPPED(status)) { + errno = 0; + ret = ptrace(PTRACE_GETREGSET, pid, NT_RISCV_USER_CFI, &iov); + if (ret == -1 && errno) + ksft_exit_fail_msg("%s: PTRACE_GETREGSET failed\n", __func__); + } else + ksft_exit_fail_msg("%s: child didn't stop, failed\n", __func__); + + switch (ptrace_test_num) { +#define CFI_ENABLE_MASK (PTRACE_CFI_LP_EN_STATE | \ + PTRACE_CFI_SS_EN_STATE | \ + PTRACE_CFI_SS_PTR_STATE) + case 0: + if ((cfi_reg.cfi_status.cfi_state & CFI_ENABLE_MASK) != CFI_ENABLE_MASK) + ksft_exit_fail_msg("%s: ptrace_getregset failed, %llu\n", __func__, + cfi_reg.cfi_status.cfi_state); + if (!cfi_reg.shstk_ptr) + ksft_exit_fail_msg("%s: NULL shadow stack pointer, test failed\n", + __func__); + break; + case 1: + if (!(cfi_reg.cfi_status.cfi_state & PTRACE_CFI_ELP_STATE)) + ksft_exit_fail_msg("%s: elp must have been set\n", __func__); + /* clear elp state. not interested in anything else */ + cfi_reg.cfi_status.cfi_state = 0; + + ret = ptrace(PTRACE_SETREGSET, pid, NT_RISCV_USER_CFI, &iov); + if (ret == -1 && errno) + ksft_exit_fail_msg("%s: PTRACE_GETREGSET failed\n", __func__); + break; + default: + ksft_exit_fail_msg("%s: unreachable switch case\n", __func__); + break; + } + ptrace(PTRACE_CONT, pid, NULL, NULL); + ptrace_test_num++; + } + + waitpid(pid, &status, 0); + if (WEXITSTATUS(status) != 11) + ksft_print_msg("%s, bad return code from child\n", __func__); + + ksft_print_msg("%s, ptrace test succeeded\n", __func__); + return true; +} + +int main(int argc, char *argv[]) +{ + int ret = 0; + unsigned long lpad_status = 0, ss_status = 0; + + ksft_print_header(); + + ksft_print_msg("Starting risc-v tests\n"); + + /* + * Landing pad test. Not a lot of kernel changes to support landing + * pad for user mode except lighting up a bit in senvcfg via a prctl + * Enable landing pad through out the execution of test binary + */ + ret = my_syscall5(__NR_prctl, PR_GET_INDIR_BR_LP_STATUS, &lpad_status, 0, 0, 0); + if (ret) + ksft_exit_fail_msg("Get landing pad status failed with %d\n", ret); + + if (!(lpad_status & PR_INDIR_BR_LP_ENABLE)) + ksft_exit_fail_msg("Landing pad is not enabled, should be enabled via glibc\n"); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, &ss_status, 0, 0, 0); + if (ret) + ksft_exit_fail_msg("Get shadow stack failed with %d\n", ret); + + if (!(ss_status & PR_SHADOW_STACK_ENABLE)) + ksft_exit_fail_msg("Shadow stack is not enabled, should be enabled via glibc\n"); + + if (!register_signal_handler()) + ksft_exit_fail_msg("Registering signal handler for SIGSEGV failed\n"); + + ksft_print_msg("Landing pad and shadow stack are enabled for binary\n"); + cfi_ptrace_test(); + + execute_shadow_stack_tests(); + + return 0; +} + +#pragma GCC pop_options diff --git a/tools/testing/selftests/riscv/cfi/shadowstack.c b/tools/testing/selftests/riscv/cfi/shadowstack.c new file mode 100644 index 000000000000..53387dbd9cf5 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/shadowstack.c @@ -0,0 +1,385 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../../kselftest.h" +#include +#include +#include +#include +#include +#include "shadowstack.h" +#include "cfi_rv_test.h" + +static struct shadow_stack_tests shstk_tests[] = { + { "shstk fork test\n", shadow_stack_fork_test }, + { "map shadow stack syscall\n", shadow_stack_map_test }, + { "shadow stack gup tests\n", shadow_stack_gup_tests }, + { "shadow stack signal tests\n", shadow_stack_signal_test}, + { "memory protections of shadow stack memory\n", shadow_stack_protection_test } +}; + +#define RISCV_SHADOW_STACK_TESTS ARRAY_SIZE(shstk_tests) + +/* do not optimize shadow stack related test functions */ +#pragma GCC push_options +#pragma GCC optimize("O0") + +void zar(void) +{ + unsigned long ssp = 0; + + ssp = csr_read(CSR_SSP); + ksft_print_msg("Spewing out shadow stack ptr: %lx\n" + " This is to ensure shadow stack is indeed enabled and working\n", + ssp); +} + +void bar(void) +{ + zar(); +} + +void foo(void) +{ + bar(); +} + +void zar_child(void) +{ + unsigned long ssp = 0; + + ssp = csr_read(CSR_SSP); + ksft_print_msg("Spewing out shadow stack ptr: %lx\n" + " This is to ensure shadow stack is indeed enabled and working\n", + ssp); +} + +void bar_child(void) +{ + zar_child(); +} + +void foo_child(void) +{ + bar_child(); +} + +typedef void (call_func_ptr)(void); +/* + * call couple of functions to test push pop. + */ +int shadow_stack_call_tests(call_func_ptr fn_ptr, bool parent) +{ + ksft_print_msg("dummy calls for sspush and sspopchk in context of %s\n", + parent ? "parent" : "child"); + + (fn_ptr)(); + + return 0; +} + +/* forks a thread, and ensure shadow stacks fork out */ +bool shadow_stack_fork_test(unsigned long test_num, void *ctx) +{ + int pid = 0, child_status = 0, parent_pid = 0, ret = 0; + unsigned long ss_status = 0; + + ksft_print_msg("Exercising shadow stack fork test\n"); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, &ss_status, 0, 0, 0); + if (ret) { + ksft_exit_skip("Shadow stack get status prctl failed with errorcode %d\n", ret); + return false; + } + + if (!(ss_status & PR_SHADOW_STACK_ENABLE)) + ksft_exit_skip("Shadow stack is not enabled, should be enabled via glibc\n"); + + parent_pid = getpid(); + pid = fork(); + + if (pid) { + ksft_print_msg("Parent pid %d and child pid %d\n", parent_pid, pid); + shadow_stack_call_tests(&foo, true); + } else { + shadow_stack_call_tests(&foo_child, false); + } + + if (pid) { + ksft_print_msg("Waiting on child to finish\n"); + wait(&child_status); + } else { + /* exit child gracefully */ + exit(0); + } + + if (pid && WIFSIGNALED(child_status)) { + ksft_print_msg("Child faulted, fork test failed\n"); + return false; + } + + return true; +} + +/* exercise `map_shadow_stack`, pivot to it and call some functions to ensure it works */ +#define SHADOW_STACK_ALLOC_SIZE 4096 +bool shadow_stack_map_test(unsigned long test_num, void *ctx) +{ + unsigned long shdw_addr; + int ret = 0; + + ksft_print_msg("Exercising shadow stack map test\n"); + + shdw_addr = my_syscall3(__NR_map_shadow_stack, NULL, SHADOW_STACK_ALLOC_SIZE, 0); + + if (((long)shdw_addr) <= 0) { + ksft_print_msg("map_shadow_stack failed with error code %d\n", + (int)shdw_addr); + return false; + } + + ret = munmap((void *)shdw_addr, SHADOW_STACK_ALLOC_SIZE); + + if (ret) { + ksft_print_msg("munmap failed with error code %d\n", ret); + return false; + } + + return true; +} + +/* + * shadow stack protection tests. map a shadow stack and + * validate all memory protections work on it + */ +bool shadow_stack_protection_test(unsigned long test_num, void *ctx) +{ + unsigned long shdw_addr; + unsigned long *write_addr = NULL; + int ret = 0, pid = 0, child_status = 0; + + ksft_print_msg("Exercising shadow stack protection test (WPT)\n"); + + shdw_addr = my_syscall3(__NR_map_shadow_stack, NULL, SHADOW_STACK_ALLOC_SIZE, 0); + + if (((long)shdw_addr) <= 0) { + ksft_print_msg("map_shadow_stack failed with error code %d\n", + (int)shdw_addr); + return false; + } + + write_addr = (unsigned long *)shdw_addr; + pid = fork(); + + /* no child was created, return false */ + if (pid == -1) + return false; + + /* + * try to perform a store from child on shadow stack memory + * it should result in SIGSEGV + */ + if (!pid) { + /* below write must lead to SIGSEGV */ + *write_addr = 0xdeadbeef; + } else { + wait(&child_status); + } + + /* test fail, if 0xdeadbeef present on shadow stack address */ + if (*write_addr == 0xdeadbeef) { + ksft_print_msg("Shadow stack WPT failed\n"); + return false; + } + + /* if child reached here, then fail */ + if (!pid) { + ksft_print_msg("Shadow stack WPT failed: child reached unreachable state\n"); + return false; + } + + /* if child exited via signal handler but not for write on ss */ + if (WIFEXITED(child_status) && + WEXITSTATUS(child_status) != CHILD_EXIT_CODE_SSWRITE) { + ksft_print_msg("Shadow stack WPT failed: child wasn't signaled for write\n"); + return false; + } + + ret = munmap(write_addr, SHADOW_STACK_ALLOC_SIZE); + if (ret) { + ksft_print_msg("Shadow stack WPT failed: munmap failed, error code %d\n", + ret); + return false; + } + + return true; +} + +#define SS_MAGIC_WRITE_VAL 0xbeefdead + +int gup_tests(int mem_fd, unsigned long *shdw_addr) +{ + unsigned long val = 0; + + lseek(mem_fd, (unsigned long)shdw_addr, SEEK_SET); + if (read(mem_fd, &val, sizeof(val)) < 0) { + ksft_print_msg("Reading shadow stack mem via gup failed\n"); + return 1; + } + + val = SS_MAGIC_WRITE_VAL; + lseek(mem_fd, (unsigned long)shdw_addr, SEEK_SET); + if (write(mem_fd, &val, sizeof(val)) < 0) { + ksft_print_msg("Writing shadow stack mem via gup failed\n"); + return 1; + } + + if (*shdw_addr != SS_MAGIC_WRITE_VAL) { + ksft_print_msg("GUP write to shadow stack memory failed\n"); + return 1; + } + + return 0; +} + +bool shadow_stack_gup_tests(unsigned long test_num, void *ctx) +{ + unsigned long shdw_addr = 0; + unsigned long *write_addr = NULL; + int fd = 0; + bool ret = false; + + ksft_print_msg("Exercising shadow stack gup tests\n"); + shdw_addr = my_syscall3(__NR_map_shadow_stack, NULL, SHADOW_STACK_ALLOC_SIZE, 0); + + if (((long)shdw_addr) <= 0) { + ksft_print_msg("map_shadow_stack failed with error code %d\n", (int)shdw_addr); + return false; + } + + write_addr = (unsigned long *)shdw_addr; + + fd = open("/proc/self/mem", O_RDWR); + if (fd == -1) + return false; + + if (gup_tests(fd, write_addr)) { + ksft_print_msg("gup tests failed\n"); + goto out; + } + + ret = true; +out: + if (shdw_addr && munmap(write_addr, SHADOW_STACK_ALLOC_SIZE)) { + ksft_print_msg("munmap failed with error code %d\n", ret); + ret = false; + } + + return ret; +} + +volatile bool break_loop; + +void sigusr1_handler(int signo) +{ + break_loop = true; +} + +bool sigusr1_signal_test(void) +{ + struct sigaction sa = {}; + + sa.sa_handler = sigusr1_handler; + sa.sa_flags = 0; + sigemptyset(&sa.sa_mask); + if (sigaction(SIGUSR1, &sa, NULL)) { + ksft_print_msg("Registering signal handler for SIGUSR1 failed\n"); + return false; + } + + return true; +} + +/* + * shadow stack signal test. shadow stack must be enabled. + * register a signal, fork another thread which is waiting + * on signal. Send a signal from parent to child, verify + * that signal was received by child. If not test fails + */ +bool shadow_stack_signal_test(unsigned long test_num, void *ctx) +{ + int pid = 0, child_status = 0, ret = 0; + unsigned long ss_status = 0; + + ksft_print_msg("Exercising shadow stack signal test\n"); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, &ss_status, 0, 0, 0); + if (ret) { + ksft_print_msg("Shadow stack get status prctl failed with errorcode %d\n", ret); + return false; + } + + if (!(ss_status & PR_SHADOW_STACK_ENABLE)) + ksft_print_msg("Shadow stack is not enabled, should be enabled via glibc\n"); + + /* this should be caught by signal handler and do an exit */ + if (!sigusr1_signal_test()) { + ksft_print_msg("Registering sigusr1 handler failed\n"); + exit(-1); + } + + pid = fork(); + + if (pid == -1) { + ksft_print_msg("Signal test: fork failed\n"); + goto out; + } + + if (pid == 0) { + while (!break_loop) + sleep(1); + + exit(11); + /* child shouldn't go beyond here */ + } + + /* send SIGUSR1 to child */ + kill(pid, SIGUSR1); + wait(&child_status); + +out: + + return (WIFEXITED(child_status) && + WEXITSTATUS(child_status) == 11); +} + +int execute_shadow_stack_tests(void) +{ + int ret = 0; + unsigned long test_count = 0; + unsigned long shstk_status = 0; + bool test_pass = false; + + ksft_print_msg("Executing RISC-V shadow stack self tests\n"); + ksft_set_plan(RISCV_SHADOW_STACK_TESTS); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, &shstk_status, 0, 0, 0); + + if (ret != 0) + ksft_exit_fail_msg("Get shadow stack status failed with %d\n", ret); + + /* + * If we are here that means get shadow stack status succeeded and + * thus shadow stack support is baked in the kernel. + */ + while (test_count < RISCV_SHADOW_STACK_TESTS) { + test_pass = (*shstk_tests[test_count].t_func)(test_count, NULL); + ksft_test_result(test_pass, shstk_tests[test_count].name); + test_count++; + } + + ksft_finished(); + + return 0; +} + +#pragma GCC pop_options diff --git a/tools/testing/selftests/riscv/cfi/shadowstack.h b/tools/testing/selftests/riscv/cfi/shadowstack.h new file mode 100644 index 000000000000..0be510167de3 --- /dev/null +++ b/tools/testing/selftests/riscv/cfi/shadowstack.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef SELFTEST_SHADOWSTACK_TEST_H +#define SELFTEST_SHADOWSTACK_TEST_H +#include +#include + +/* + * a cfi test returns true for success or false for fail + * takes a number for test number to index into array and void pointer. + */ +typedef bool (*shstk_test_func)(unsigned long test_num, void *); + +struct shadow_stack_tests { + char *name; + shstk_test_func t_func; +}; + +bool shadow_stack_fork_test(unsigned long test_num, void *ctx); +bool shadow_stack_map_test(unsigned long test_num, void *ctx); +bool shadow_stack_protection_test(unsigned long test_num, void *ctx); +bool shadow_stack_gup_tests(unsigned long test_num, void *ctx); +bool shadow_stack_signal_test(unsigned long test_num, void *ctx); + +int execute_shadow_stack_tests(void); + +#endif